Search in sources :

Example 26 with NETWORK

use of org.apache.cassandra.distributed.api.Feature.NETWORK in project cassandra by apache.

the class ReprepareTestBase method testReprepareTwoKeyspaces.

public void testReprepareTwoKeyspaces(BiConsumer<ClassLoader, Integer> instanceInitializer) throws Throwable {
    try (ICluster<IInvokableInstance> c = init(builder().withNodes(2).withConfig(config -> config.with(GOSSIP, NETWORK, NATIVE_PROTOCOL)).withInstanceInitializer(instanceInitializer).start())) {
        c.schemaChange(withKeyspace("CREATE KEYSPACE %s2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 2};"));
        c.schemaChange(withKeyspace("CREATE TABLE %s.tbl (pk int, ck int, v int, PRIMARY KEY (pk, ck));"));
        ForceHostLoadBalancingPolicy lbp = new ForceHostLoadBalancingPolicy();
        for (int firstContact : new int[] { 1, 2 }) try (com.datastax.driver.core.Cluster cluster = com.datastax.driver.core.Cluster.builder().addContactPoint("127.0.0.1").addContactPoint("127.0.0.2").withLoadBalancingPolicy(lbp).build();
            Session session = cluster.connect()) {
            {
                session.execute(withKeyspace("USE %s"));
                c.stream().forEach((i) -> i.runOnInstance(QueryProcessor::clearPreparedStatementsCache));
                lbp.setPrimary(firstContact);
                final PreparedStatement select = session.prepare(withKeyspace("SELECT * FROM %s.tbl"));
                session.execute(select.bind());
                c.stream().forEach((i) -> i.runOnInstance(QueryProcessor::clearPreparedStatementsCache));
                lbp.setPrimary(firstContact == 1 ? 2 : 1);
                session.execute(withKeyspace("USE %s2"));
                try {
                    session.execute(select.bind());
                } catch (DriverInternalError e) {
                    Assert.assertTrue(e.getCause().getMessage().contains("can't execute it on"));
                    continue;
                }
                fail("Should have thrown");
            }
        }
    }
}
Also used : LoadBalancingPolicy(com.datastax.driver.core.policies.LoadBalancingPolicy) MethodDelegation(net.bytebuddy.implementation.MethodDelegation) ByteBuddy(net.bytebuddy.ByteBuddy) ElementMatchers.takesArguments(net.bytebuddy.matcher.ElementMatchers.takesArguments) QueryProcessor(org.apache.cassandra.cql3.QueryProcessor) QueryHandler(org.apache.cassandra.cql3.QueryHandler) Iterators(com.google.common.collect.Iterators) NATIVE_PROTOCOL(org.apache.cassandra.distributed.api.Feature.NATIVE_PROTOCOL) PreparedStatement(com.datastax.driver.core.PreparedStatement) FixedValue(net.bytebuddy.implementation.FixedValue) Session(com.datastax.driver.core.Session) BiConsumer(java.util.function.BiConsumer) AssertUtils.fail(org.apache.cassandra.distributed.shared.AssertUtils.fail) InvalidRequestException(org.apache.cassandra.exceptions.InvalidRequestException) NETWORK(org.apache.cassandra.distributed.api.Feature.NETWORK) ResultMessage(org.apache.cassandra.transport.messages.ResultMessage) FBUtilities(org.apache.cassandra.utils.FBUtilities) Iterator(java.util.Iterator) ElementMatchers.named(net.bytebuddy.matcher.ElementMatchers.named) Collection(java.util.Collection) ClientState(org.apache.cassandra.service.ClientState) ICluster(org.apache.cassandra.distributed.api.ICluster) ClassLoadingStrategy(net.bytebuddy.dynamic.loading.ClassLoadingStrategy) DriverInternalError(com.datastax.driver.core.exceptions.DriverInternalError) List(java.util.List) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Cluster(com.datastax.driver.core.Cluster) Host(com.datastax.driver.core.Host) HostDistance(com.datastax.driver.core.HostDistance) Comparator(java.util.Comparator) Assert(org.junit.Assert) Collections(java.util.Collections) Statement(com.datastax.driver.core.Statement) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) GOSSIP(org.apache.cassandra.distributed.api.Feature.GOSSIP) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) DriverInternalError(com.datastax.driver.core.exceptions.DriverInternalError) ICluster(org.apache.cassandra.distributed.api.ICluster) Cluster(com.datastax.driver.core.Cluster) PreparedStatement(com.datastax.driver.core.PreparedStatement) Session(com.datastax.driver.core.Session)

Example 27 with NETWORK

use of org.apache.cassandra.distributed.api.Feature.NETWORK in project cassandra by apache.

the class ReprepareTestBase method testReprepare.

public void testReprepare(BiConsumer<ClassLoader, Integer> instanceInitializer, ReprepareTestConfiguration... configs) throws Throwable {
    try (ICluster<IInvokableInstance> c = init(builder().withNodes(2).withConfig(config -> config.with(GOSSIP, NETWORK, NATIVE_PROTOCOL)).withInstanceInitializer(instanceInitializer).start())) {
        ForceHostLoadBalancingPolicy lbp = new ForceHostLoadBalancingPolicy();
        c.schemaChange(withKeyspace("CREATE TABLE %s.tbl (pk int, ck int, v int, PRIMARY KEY (pk, ck));"));
        for (ReprepareTestConfiguration config : configs) {
            // 1 has old behaviour
            for (int firstContact : new int[] { 1, 2 }) {
                try (com.datastax.driver.core.Cluster cluster = com.datastax.driver.core.Cluster.builder().addContactPoint("127.0.0.1").addContactPoint("127.0.0.2").withLoadBalancingPolicy(lbp).build();
                    Session session = cluster.connect()) {
                    lbp.setPrimary(firstContact);
                    final PreparedStatement select = session.prepare(withKeyspace("SELECT * FROM %s.tbl"));
                    session.execute(select.bind());
                    c.stream().forEach((i) -> i.runOnInstance(QueryProcessor::clearPreparedStatementsCache));
                    lbp.setPrimary(firstContact == 1 ? 2 : 1);
                    if (config.withUse)
                        session.execute(withKeyspace("USE %s"));
                    // Re-preparing on the node
                    if (!config.skipBrokenBehaviours && firstContact == 1)
                        session.execute(select.bind());
                    c.stream().forEach((i) -> i.runOnInstance(QueryProcessor::clearPreparedStatementsCache));
                    lbp.setPrimary(firstContact);
                    // Re-preparing on the node with old behaviour will break no matter where the statement was initially prepared
                    if (!config.skipBrokenBehaviours)
                        session.execute(select.bind());
                    c.stream().forEach((i) -> i.runOnInstance(QueryProcessor::clearPreparedStatementsCache));
                }
            }
        }
    }
}
Also used : LoadBalancingPolicy(com.datastax.driver.core.policies.LoadBalancingPolicy) MethodDelegation(net.bytebuddy.implementation.MethodDelegation) ByteBuddy(net.bytebuddy.ByteBuddy) ElementMatchers.takesArguments(net.bytebuddy.matcher.ElementMatchers.takesArguments) QueryProcessor(org.apache.cassandra.cql3.QueryProcessor) QueryHandler(org.apache.cassandra.cql3.QueryHandler) Iterators(com.google.common.collect.Iterators) NATIVE_PROTOCOL(org.apache.cassandra.distributed.api.Feature.NATIVE_PROTOCOL) PreparedStatement(com.datastax.driver.core.PreparedStatement) FixedValue(net.bytebuddy.implementation.FixedValue) Session(com.datastax.driver.core.Session) BiConsumer(java.util.function.BiConsumer) AssertUtils.fail(org.apache.cassandra.distributed.shared.AssertUtils.fail) InvalidRequestException(org.apache.cassandra.exceptions.InvalidRequestException) NETWORK(org.apache.cassandra.distributed.api.Feature.NETWORK) ResultMessage(org.apache.cassandra.transport.messages.ResultMessage) FBUtilities(org.apache.cassandra.utils.FBUtilities) Iterator(java.util.Iterator) ElementMatchers.named(net.bytebuddy.matcher.ElementMatchers.named) Collection(java.util.Collection) ClientState(org.apache.cassandra.service.ClientState) ICluster(org.apache.cassandra.distributed.api.ICluster) ClassLoadingStrategy(net.bytebuddy.dynamic.loading.ClassLoadingStrategy) DriverInternalError(com.datastax.driver.core.exceptions.DriverInternalError) List(java.util.List) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Cluster(com.datastax.driver.core.Cluster) Host(com.datastax.driver.core.Host) HostDistance(com.datastax.driver.core.HostDistance) Comparator(java.util.Comparator) Assert(org.junit.Assert) Collections(java.util.Collections) Statement(com.datastax.driver.core.Statement) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) GOSSIP(org.apache.cassandra.distributed.api.Feature.GOSSIP) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Cluster(com.datastax.driver.core.Cluster) PreparedStatement(com.datastax.driver.core.PreparedStatement) Session(com.datastax.driver.core.Session)

Example 28 with NETWORK

use of org.apache.cassandra.distributed.api.Feature.NETWORK in project cassandra by apache.

the class PendingWritesTest method testPendingWrites.

@Test
public void testPendingWrites() throws Throwable {
    int originalNodeCount = 2;
    int expandedNodeCount = originalNodeCount + 1;
    try (Cluster cluster = builder().withNodes(originalNodeCount).withTokenSupplier(TokenSupplier.evenlyDistributedTokens(expandedNodeCount)).withNodeIdTopology(NetworkTopology.singleDcNetworkTopology(expandedNodeCount, "dc0", "rack0")).withConfig(config -> config.with(NETWORK, GOSSIP)).start()) {
        BootstrapTest.populate(cluster, 0, 100);
        IInstanceConfig config = cluster.newInstanceConfig();
        IInvokableInstance newInstance = cluster.bootstrap(config);
        withProperty("cassandra.join_ring", false, () -> newInstance.startup(cluster));
        cluster.forEach(statusToBootstrap(newInstance));
        cluster.run(bootstrap(false, Duration.ofSeconds(60), Duration.ofSeconds(60)), newInstance.config().num());
        cluster.get(1).acceptsOnInstance((InetSocketAddress ip) -> {
            Set<InetAddressAndPort> set = new HashSet<>();
            for (Map.Entry<Range<Token>, EndpointsForRange.Builder> e : StorageService.instance.getTokenMetadata().getPendingRanges(KEYSPACE)) {
                set.addAll(e.getValue().build().endpoints());
            }
            Assert.assertEquals(set.size(), 1);
            Assert.assertTrue(String.format("%s should contain %s", set, ip), set.contains(DistributedTestSnitch.toCassandraInetAddressAndPort(ip)));
        }).accept(cluster.get(3).broadcastAddress());
        BootstrapTest.populate(cluster, 100, 150);
        newInstance.nodetoolResult("join").asserts().success();
        cluster.run(disseminateGossipState(newInstance), 1, 2);
        cluster.run((instance) -> {
            instance.runOnInstance(() -> {
                PendingRangeCalculatorService.instance.update();
                PendingRangeCalculatorService.instance.blockUntilFinished();
            });
        }, 1, 2);
        cluster.get(1).acceptsOnInstance((InetSocketAddress ip) -> {
            Set<InetAddressAndPort> set = new HashSet<>();
            for (Map.Entry<Range<Token>, EndpointsForRange.Builder> e : StorageService.instance.getTokenMetadata().getPendingRanges(KEYSPACE)) set.addAll(e.getValue().build().endpoints());
            assert set.size() == 0 : set;
        }).accept(cluster.get(3).broadcastAddress());
        for (Map.Entry<Integer, Long> e : BootstrapTest.count(cluster).entrySet()) Assert.assertEquals("Node " + e.getKey() + " has incorrect row state", e.getValue().longValue(), 150L);
    }
}
Also used : InetAddressAndPort(org.apache.cassandra.locator.InetAddressAndPort) Range(org.apache.cassandra.dht.Range) PendingRangeCalculatorService(org.apache.cassandra.service.PendingRangeCalculatorService) HashSet(java.util.HashSet) TokenSupplier(org.apache.cassandra.distributed.api.TokenSupplier) DistributedTestSnitch(org.apache.cassandra.distributed.impl.DistributedTestSnitch) Token(org.apache.cassandra.dht.Token) Duration(java.time.Duration) Map(java.util.Map) TestBaseImpl(org.apache.cassandra.distributed.test.TestBaseImpl) IInstanceConfig(org.apache.cassandra.distributed.api.IInstanceConfig) GossipHelper.statusToBootstrap(org.apache.cassandra.distributed.action.GossipHelper.statusToBootstrap) NETWORK(org.apache.cassandra.distributed.api.Feature.NETWORK) EndpointsForRange(org.apache.cassandra.locator.EndpointsForRange) Set(java.util.Set) StorageService(org.apache.cassandra.service.StorageService) Test(org.junit.Test) InetSocketAddress(java.net.InetSocketAddress) GossipHelper.disseminateGossipState(org.apache.cassandra.distributed.action.GossipHelper.disseminateGossipState) GossipHelper.bootstrap(org.apache.cassandra.distributed.action.GossipHelper.bootstrap) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Cluster(org.apache.cassandra.distributed.Cluster) GossipHelper.withProperty(org.apache.cassandra.distributed.action.GossipHelper.withProperty) Assert(org.junit.Assert) NetworkTopology(org.apache.cassandra.distributed.shared.NetworkTopology) GOSSIP(org.apache.cassandra.distributed.api.Feature.GOSSIP) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) HashSet(java.util.HashSet) Set(java.util.Set) InetSocketAddress(java.net.InetSocketAddress) Cluster(org.apache.cassandra.distributed.Cluster) Token(org.apache.cassandra.dht.Token) IInstanceConfig(org.apache.cassandra.distributed.api.IInstanceConfig) EndpointsForRange(org.apache.cassandra.locator.EndpointsForRange) Map(java.util.Map) Test(org.junit.Test)

Example 29 with NETWORK

use of org.apache.cassandra.distributed.api.Feature.NETWORK in project cassandra by apache.

the class RepairOperationalTest method compactionBehindTest.

@Test
public void compactionBehindTest() throws IOException {
    try (Cluster cluster = init(Cluster.build(2).withConfig(config -> config.with(GOSSIP).with(NETWORK)).withInstanceInitializer(ByteBuddyHelper::install).start())) {
        cluster.schemaChange("create table " + KEYSPACE + ".tbl (id int primary key, x int)");
        for (int i = 0; i < 10; i++) cluster.coordinator(1).execute("insert into " + KEYSPACE + ".tbl (id, x) VALUES (?,?)", ConsistencyLevel.ALL, i, i);
        cluster.forEach(i -> i.flush(KEYSPACE));
        cluster.forEach(i -> i.nodetoolResult("repair", "--full").asserts().success());
        cluster.get(2).runOnInstance(() -> {
            ByteBuddyHelper.pendingCompactions = 1000;
            DatabaseDescriptor.setRepairPendingCompactionRejectThreshold(500);
        });
        // make sure repair gets rejected on both nodes if pendingCompactions > threshold:
        cluster.forEach(i -> i.nodetoolResult("repair", "--full").asserts().failure());
        cluster.get(2).runOnInstance(() -> ByteBuddyHelper.pendingCompactions = 499);
        cluster.forEach(i -> i.nodetoolResult("repair", "--full").asserts().success());
    }
}
Also used : CompactionManager(org.apache.cassandra.db.compaction.CompactionManager) MethodDelegation(net.bytebuddy.implementation.MethodDelegation) ElementMatchers.named(net.bytebuddy.matcher.ElementMatchers.named) ByteBuddy(net.bytebuddy.ByteBuddy) IOException(java.io.IOException) Test(org.junit.Test) ConsistencyLevel(org.apache.cassandra.distributed.api.ConsistencyLevel) ClassLoadingStrategy(net.bytebuddy.dynamic.loading.ClassLoadingStrategy) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Assertions(org.assertj.core.api.Assertions) Cluster(org.apache.cassandra.distributed.Cluster) DatabaseDescriptor(org.apache.cassandra.config.DatabaseDescriptor) NETWORK(org.apache.cassandra.distributed.api.Feature.NETWORK) GOSSIP(org.apache.cassandra.distributed.api.Feature.GOSSIP) Cluster(org.apache.cassandra.distributed.Cluster) Test(org.junit.Test)

Example 30 with NETWORK

use of org.apache.cassandra.distributed.api.Feature.NETWORK in project cassandra by apache.

the class NetstatsRepairStreamingTest method executeTest.

private void executeTest(boolean compressionEnabled) throws Exception {
    final ExecutorService executorService = Executors.newFixedThreadPool(1);
    try (final Cluster cluster = Cluster.build().withNodeIdTopology(NetworkTopology.singleDcNetworkTopology(2, "dc0", "rack0")).withConfig(config -> config.with(NETWORK, GOSSIP, NATIVE_PROTOCOL).set("stream_throughput_outbound", "122KiB/s").set("compaction_throughput", "1MiB/s").set("stream_entire_sstables", false)).start()) {
        final IInvokableInstance node1 = cluster.get(1);
        final IInvokableInstance node2 = cluster.get(2);
        createTable(cluster, 1, compressionEnabled);
        node1.nodetoolResult("disableautocompaction", "netstats_test").asserts().success();
        node2.nodetoolResult("disableautocompaction", "netstats_test").asserts().success();
        populateData(compressionEnabled);
        node1.flush("netstats_test");
        node2.flush("netstats_test");
        // change RF from 1 to 2 so we need to repair it, repairing will causes streaming shown in netstats
        changeReplicationFactor();
        final Future<NetstatResults> resultsFuture1 = executorService.submit(new NetstatsCallable(node1));
        node1.nodetoolResult("repair", "netstats_test").asserts().success();
        final NetstatResults results = resultsFuture1.get(1, MINUTES);
        results.assertSuccessful();
        NetstatsOutputParser.validate(NetstatsOutputParser.parse(results));
    }
}
Also used : Future(java.util.concurrent.Future) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) Test(org.junit.Test) Cluster(org.apache.cassandra.distributed.Cluster) MINUTES(java.util.concurrent.TimeUnit.MINUTES) NetworkTopology(org.apache.cassandra.distributed.shared.NetworkTopology) ExecutorService(java.util.concurrent.ExecutorService) Executors(java.util.concurrent.Executors) NETWORK(org.apache.cassandra.distributed.api.Feature.NETWORK) GOSSIP(org.apache.cassandra.distributed.api.Feature.GOSSIP) NATIVE_PROTOCOL(org.apache.cassandra.distributed.api.Feature.NATIVE_PROTOCOL) IInvokableInstance(org.apache.cassandra.distributed.api.IInvokableInstance) ExecutorService(java.util.concurrent.ExecutorService) Cluster(org.apache.cassandra.distributed.Cluster)

Aggregations

NETWORK (org.apache.cassandra.distributed.api.Feature.NETWORK)34 GOSSIP (org.apache.cassandra.distributed.api.Feature.GOSSIP)33 Test (org.junit.Test)32 Cluster (org.apache.cassandra.distributed.Cluster)27 Assert (org.junit.Assert)17 IInvokableInstance (org.apache.cassandra.distributed.api.IInvokableInstance)16 ConsistencyLevel (org.apache.cassandra.distributed.api.ConsistencyLevel)13 NATIVE_PROTOCOL (org.apache.cassandra.distributed.api.Feature.NATIVE_PROTOCOL)13 StorageService (org.apache.cassandra.service.StorageService)13 ByteBuddy (net.bytebuddy.ByteBuddy)12 ClassLoadingStrategy (net.bytebuddy.dynamic.loading.ClassLoadingStrategy)12 MethodDelegation (net.bytebuddy.implementation.MethodDelegation)12 ElementMatchers.named (net.bytebuddy.matcher.ElementMatchers.named)12 List (java.util.List)11 Map (java.util.Map)11 TimeUnit (java.util.concurrent.TimeUnit)11 ICluster (org.apache.cassandra.distributed.api.ICluster)10 NetworkTopology (org.apache.cassandra.distributed.shared.NetworkTopology)10 IOException (java.io.IOException)9 TokenSupplier (org.apache.cassandra.distributed.api.TokenSupplier)9