Search in sources :

Example 6 with EndpointsForToken

use of org.apache.cassandra.locator.EndpointsForToken in project cassandra by apache.

the class DigestResolverTest method transientResponseData.

@Test
public void transientResponseData() {
    SinglePartitionReadCommand command = SinglePartitionReadCommand.fullPartitionRead(cfm, nowInSec, dk);
    EndpointsForToken targetReplicas = EndpointsForToken.of(dk.getToken(), full(EP1), full(EP2), trans(EP3));
    DigestResolver<?, ?> resolver = new DigestResolver<>(command, plan(ConsistencyLevel.QUORUM, targetReplicas), 0);
    PartitionUpdate fullResponse = update(row(1000, 1, 1)).build();
    PartitionUpdate digestResponse = update(row(1000, 1, 1)).build();
    PartitionUpdate transientResponse = update(row(1000, 2, 2)).build();
    Assert.assertFalse(resolver.isDataPresent());
    Assert.assertFalse(resolver.hasTransientResponse());
    resolver.preprocess(response(command, EP1, iter(fullResponse), false));
    Assert.assertTrue(resolver.isDataPresent());
    resolver.preprocess(response(command, EP2, iter(digestResponse), true));
    resolver.preprocess(response(command, EP3, iter(transientResponse), false));
    Assert.assertTrue(resolver.hasTransientResponse());
    assertPartitionsEqual(filter(iter(dk, row(1000, 1, 1), row(1000, 2, 2))), resolver.getData());
}
Also used : EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) SinglePartitionReadCommand(org.apache.cassandra.db.SinglePartitionReadCommand) PartitionUpdate(org.apache.cassandra.db.partitions.PartitionUpdate) Test(org.junit.Test)

Example 7 with EndpointsForToken

use of org.apache.cassandra.locator.EndpointsForToken in project cassandra by apache.

the class DigestResolverTest method multiThreadedNoRepairNeededReadCallback.

/**
 * This test makes a time-boxed effort to reproduce the issue found in CASSANDRA-16807.
 */
@Test
public void multiThreadedNoRepairNeededReadCallback() {
    SinglePartitionReadCommand command = SinglePartitionReadCommand.fullPartitionRead(cfm, nowInSec, dk);
    EndpointsForToken targetReplicas = EndpointsForToken.of(dk.getToken(), full(EP1), full(EP2));
    PartitionUpdate response = update(row(1000, 4, 4), row(1000, 5, 5)).build();
    ReplicaPlan.SharedForTokenRead plan = plan(ConsistencyLevel.ONE, targetReplicas);
    ExecutorService pool = Executors.newFixedThreadPool(2);
    long endTime = System.nanoTime() + TimeUnit.MINUTES.toNanos(2);
    try {
        while (System.nanoTime() < endTime) {
            final long startNanos = System.nanoTime();
            final DigestResolver<EndpointsForToken, ReplicaPlan.ForTokenRead> resolver = new DigestResolver<>(command, plan, startNanos);
            final ReadCallback<EndpointsForToken, ReplicaPlan.ForTokenRead> callback = new ReadCallback<>(resolver, command, plan, startNanos);
            final CountDownLatch startlatch = new CountDownLatch(2);
            pool.execute(() -> {
                startlatch.countDown();
                waitForLatch(startlatch);
                callback.onResponse(response(command, EP1, iter(response), true));
            });
            pool.execute(() -> {
                startlatch.countDown();
                waitForLatch(startlatch);
                callback.onResponse(response(command, EP2, iter(response), true));
            });
            callback.awaitResults();
            Assert.assertTrue(resolver.isDataPresent());
            Assert.assertTrue(resolver.responsesMatch());
        }
    } finally {
        pool.shutdown();
    }
}
Also used : EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) ReplicaPlan(org.apache.cassandra.locator.ReplicaPlan) SinglePartitionReadCommand(org.apache.cassandra.db.SinglePartitionReadCommand) CountDownLatch(java.util.concurrent.CountDownLatch) ExecutorService(java.util.concurrent.ExecutorService) PartitionUpdate(org.apache.cassandra.db.partitions.PartitionUpdate) Test(org.junit.Test)

Example 8 with EndpointsForToken

use of org.apache.cassandra.locator.EndpointsForToken in project cassandra by apache.

the class DigestResolverTest method transientResponse.

/**
 * Transient responses shouldn't be classified as the single dataResponse
 */
@Test
public void transientResponse() {
    SinglePartitionReadCommand command = SinglePartitionReadCommand.fullPartitionRead(cfm, nowInSec, dk);
    EndpointsForToken targetReplicas = EndpointsForToken.of(dk.getToken(), full(EP1), trans(EP2));
    DigestResolver<?, ?> resolver = new DigestResolver<>(command, plan(ConsistencyLevel.QUORUM, targetReplicas), 0);
    PartitionUpdate response2 = update(row(1000, 5, 5)).build();
    Assert.assertFalse(resolver.isDataPresent());
    Assert.assertFalse(resolver.hasTransientResponse());
    resolver.preprocess(response(command, EP2, iter(response2), false));
    Assert.assertFalse(resolver.isDataPresent());
    Assert.assertTrue(resolver.hasTransientResponse());
}
Also used : EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) SinglePartitionReadCommand(org.apache.cassandra.db.SinglePartitionReadCommand) PartitionUpdate(org.apache.cassandra.db.partitions.PartitionUpdate) Test(org.junit.Test)

Example 9 with EndpointsForToken

use of org.apache.cassandra.locator.EndpointsForToken in project cassandra by apache.

the class AbstractReadRepairTest method repairPlan.

static ReplicaPlan.ForTokenWrite repairPlan(ReplicaPlan.ForRangeRead readPlan, EndpointsForRange liveAndDown) {
    Token token = readPlan.range().left.getToken();
    EndpointsForToken pending = EndpointsForToken.empty(token);
    return ReplicaPlans.forWrite(readPlan.keyspace(), ConsistencyLevel.TWO, liveAndDown.forToken(token), pending, replica -> true, ReplicaPlans.writeReadRepair(readPlan));
}
Also used : EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) Token(org.apache.cassandra.dht.Token)

Example 10 with EndpointsForToken

use of org.apache.cassandra.locator.EndpointsForToken in project cassandra by apache.

the class StorageProxy method findSuitableReplica.

/**
 * Find a suitable replica as leader for counter update.
 * For now, we pick a random replica in the local DC (or ask the snitch if
 * there is no replica alive in the local DC).
 * TODO: if we track the latency of the counter writes (which makes sense
 * contrarily to standard writes since there is a read involved), we could
 * trust the dynamic snitch entirely, which may be a better solution. It
 * is unclear we want to mix those latencies with read latencies, so this
 * may be a bit involved.
 */
private static Replica findSuitableReplica(String keyspaceName, DecoratedKey key, String localDataCenter, ConsistencyLevel cl) throws UnavailableException {
    Keyspace keyspace = Keyspace.open(keyspaceName);
    IEndpointSnitch snitch = DatabaseDescriptor.getEndpointSnitch();
    AbstractReplicationStrategy replicationStrategy = keyspace.getReplicationStrategy();
    EndpointsForToken replicas = replicationStrategy.getNaturalReplicasForToken(key);
    // CASSANDRA-13043: filter out those endpoints not accepting clients yet, maybe because still bootstrapping
    replicas = replicas.filter(replica -> StorageService.instance.isRpcReady(replica.endpoint()));
    // TODO have a way to compute the consistency level
    if (replicas.isEmpty())
        throw UnavailableException.create(cl, cl.blockFor(replicationStrategy), 0);
    List<Replica> localReplicas = new ArrayList<>(replicas.size());
    for (Replica replica : replicas) if (snitch.getDatacenter(replica).equals(localDataCenter))
        localReplicas.add(replica);
    if (localReplicas.isEmpty()) {
        // If the consistency required is local then we should not involve other DCs
        if (cl.isDatacenterLocal())
            throw UnavailableException.create(cl, cl.blockFor(replicationStrategy), 0);
        // No endpoint in local DC, pick the closest endpoint according to the snitch
        replicas = snitch.sortedByProximity(FBUtilities.getBroadcastAddressAndPort(), replicas);
        return replicas.get(0);
    }
    return localReplicas.get(ThreadLocalRandom.current().nextInt(localReplicas.size()));
}
Also used : Arrays(java.util.Arrays) StorageMetrics(org.apache.cassandra.metrics.StorageMetrics) Stage(org.apache.cassandra.concurrent.Stage) EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) ClientRequestsMetricsHolder.readMetrics(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.readMetrics) ReadResponse(org.apache.cassandra.db.ReadResponse) Global.nanoTime(org.apache.cassandra.utils.Clock.Global.nanoTime) ReadRepairMetrics(org.apache.cassandra.metrics.ReadRepairMetrics) UnfilteredPartitionIterator(org.apache.cassandra.db.partitions.UnfilteredPartitionIterator) Future(java.util.concurrent.Future) OverloadedException(org.apache.cassandra.exceptions.OverloadedException) Replicas(org.apache.cassandra.locator.Replicas) Mutation(org.apache.cassandra.db.Mutation) Pair(org.apache.cassandra.utils.Pair) Map(java.util.Map) HintsService(org.apache.cassandra.hints.HintsService) SinglePartitionReadCommand(org.apache.cassandra.db.SinglePartitionReadCommand) PartitionIterators(org.apache.cassandra.db.partitions.PartitionIterators) ReplicaPlans(org.apache.cassandra.locator.ReplicaPlans) ReplicaPlan(org.apache.cassandra.locator.ReplicaPlan) Keyspace(org.apache.cassandra.db.Keyspace) ReadAbortException(org.apache.cassandra.exceptions.ReadAbortException) PartitionRangeReadCommand(org.apache.cassandra.db.PartitionRangeReadCommand) FBUtilities(org.apache.cassandra.utils.FBUtilities) RejectException(org.apache.cassandra.db.RejectException) CasWriteTimeoutException(org.apache.cassandra.exceptions.CasWriteTimeoutException) Set(java.util.Set) WriteType(org.apache.cassandra.db.WriteType) Verb(org.apache.cassandra.net.Verb) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) UUIDGen(org.apache.cassandra.utils.UUIDGen) CassandraRelevantProperties(org.apache.cassandra.config.CassandraRelevantProperties) WriteFailureException(org.apache.cassandra.exceptions.WriteFailureException) ReplicaLayout(org.apache.cassandra.locator.ReplicaLayout) NoPayload.noPayload(org.apache.cassandra.net.NoPayload.noPayload) InetAddressAndPort(org.apache.cassandra.locator.InetAddressAndPort) Iterables(com.google.common.collect.Iterables) TableId(org.apache.cassandra.schema.TableId) PrepareVerbHandler.doPrepare(org.apache.cassandra.service.paxos.PrepareVerbHandler.doPrepare) Global.currentTimeMillis(org.apache.cassandra.utils.Clock.Global.currentTimeMillis) ReadCommand(org.apache.cassandra.db.ReadCommand) Message(org.apache.cassandra.net.Message) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) TruncateRequest(org.apache.cassandra.db.TruncateRequest) Schema(org.apache.cassandra.schema.Schema) IEndpointSnitch(org.apache.cassandra.locator.IEndpointSnitch) Token(org.apache.cassandra.dht.Token) CountDownLatch.newCountDownLatch(org.apache.cassandra.utils.concurrent.CountDownLatch.newCountDownLatch) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) BatchlogManager(org.apache.cassandra.batchlog.BatchlogManager) InvalidRequestException(org.apache.cassandra.exceptions.InvalidRequestException) RequestTimeoutException(org.apache.cassandra.exceptions.RequestTimeoutException) TriggerExecutor(org.apache.cassandra.triggers.TriggerExecutor) ConsistencyLevel(org.apache.cassandra.db.ConsistencyLevel) MessagingService(org.apache.cassandra.net.MessagingService) Uninterruptibles(com.google.common.util.concurrent.Uninterruptibles) ReadTimeoutException(org.apache.cassandra.exceptions.ReadTimeoutException) MessageParams(org.apache.cassandra.db.MessageParams) Hint(org.apache.cassandra.hints.Hint) FilteredPartition(org.apache.cassandra.db.partitions.FilteredPartition) RequestFailureReason(org.apache.cassandra.exceptions.RequestFailureReason) AtomicLong(java.util.concurrent.atomic.AtomicLong) ViewUtils(org.apache.cassandra.db.view.ViewUtils) CasWriteUnknownResultException(org.apache.cassandra.exceptions.CasWriteUnknownResultException) Preconditions(com.google.common.base.Preconditions) ClientRequestsMetricsHolder.casReadMetrics(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.casReadMetrics) ClientRequestsMetricsHolder.casWriteMetrics(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.casWriteMetrics) CounterMutation(org.apache.cassandra.db.CounterMutation) BatchlogCleanup(org.apache.cassandra.service.BatchlogResponseHandler.BatchlogCleanup) LoggerFactory(org.slf4j.LoggerFactory) TimeoutException(java.util.concurrent.TimeoutException) CountDownLatch(org.apache.cassandra.utils.concurrent.CountDownLatch) Message.out(org.apache.cassandra.net.Message.out) ByteBuffer(java.nio.ByteBuffer) Gossiper(org.apache.cassandra.gms.Gossiper) CASClientRequestMetrics(org.apache.cassandra.metrics.CASClientRequestMetrics) DecoratedKey(org.apache.cassandra.db.DecoratedKey) ClientRequestsMetricsHolder.readMetricsForLevel(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.readMetricsForLevel) ReadRepair(org.apache.cassandra.service.reads.repair.ReadRepair) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AbstractReadExecutor(org.apache.cassandra.service.reads.AbstractReadExecutor) Iterables.concat(com.google.common.collect.Iterables.concat) RowIterator(org.apache.cassandra.db.rows.RowIterator) DatabaseDescriptor(org.apache.cassandra.config.DatabaseDescriptor) Global.randomBallot(org.apache.cassandra.service.paxos.BallotGenerator.Global.randomBallot) UnavailableException(org.apache.cassandra.exceptions.UnavailableException) MessageFlag(org.apache.cassandra.net.MessageFlag) Collection(java.util.Collection) ClientRequestsMetricsHolder.writeMetrics(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.writeMetrics) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) WriteTimeoutException(org.apache.cassandra.exceptions.WriteTimeoutException) UUID(java.util.UUID) NoSpamLogger(org.apache.cassandra.utils.NoSpamLogger) RequestCallback(org.apache.cassandra.net.RequestCallback) Collectors(java.util.stream.Collectors) SERIAL(org.apache.cassandra.db.ConsistencyLevel.SERIAL) Objects(java.util.Objects) CacheLoader(com.google.common.cache.CacheLoader) UncheckedInterruptedException(org.apache.cassandra.utils.concurrent.UncheckedInterruptedException) List(java.util.List) ProposeVerbHandler.doPropose(org.apache.cassandra.service.paxos.ProposeVerbHandler.doPropose) ColumnFamilyStore(org.apache.cassandra.db.ColumnFamilyStore) ForwardingInfo(org.apache.cassandra.net.ForwardingInfo) TableMetadata(org.apache.cassandra.schema.TableMetadata) Optional(java.util.Optional) PartitionIterator(org.apache.cassandra.db.partitions.PartitionIterator) ClientRequestsMetricsHolder.writeMetricsForLevel(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.writeMetricsForLevel) SchemaConstants(org.apache.cassandra.schema.SchemaConstants) RangeCommands(org.apache.cassandra.service.reads.range.RangeCommands) org.apache.cassandra.service.paxos(org.apache.cassandra.service.paxos) ReadExecutionController(org.apache.cassandra.db.ReadExecutionController) PartitionDenylist(org.apache.cassandra.schema.PartitionDenylist) PartitionUpdate(org.apache.cassandra.db.partitions.PartitionUpdate) NANOSECONDS(java.util.concurrent.TimeUnit.NANOSECONDS) Config(org.apache.cassandra.config.Config) HashMap(java.util.HashMap) TombstoneOverwhelmingException(org.apache.cassandra.db.filter.TombstoneOverwhelmingException) RequestFailureException(org.apache.cassandra.exceptions.RequestFailureException) HashSet(java.util.HashSet) ClientRequestsMetricsHolder.viewWriteMetrics(org.apache.cassandra.metrics.ClientRequestsMetricsHolder.viewWriteMetrics) StringUtils.join(org.apache.commons.lang3.StringUtils.join) IsBootstrappingException(org.apache.cassandra.exceptions.IsBootstrappingException) Clock(org.apache.cassandra.utils.Clock) Global.nextBallotTimestampMicros(org.apache.cassandra.service.paxos.BallotGenerator.Global.nextBallotTimestampMicros) DenylistMetrics(org.apache.cassandra.metrics.DenylistMetrics) Logger(org.slf4j.Logger) ReadFailureException(org.apache.cassandra.exceptions.ReadFailureException) Tracing(org.apache.cassandra.tracing.Tracing) MonotonicClock(org.apache.cassandra.utils.MonotonicClock) Ints(com.google.common.primitives.Ints) Batch(org.apache.cassandra.batchlog.Batch) Replica(org.apache.cassandra.locator.Replica) TimeUnit(java.util.concurrent.TimeUnit) AbstractReplicationStrategy(org.apache.cassandra.locator.AbstractReplicationStrategy) ReadCallback(org.apache.cassandra.service.reads.ReadCallback) MBeanWrapper(org.apache.cassandra.utils.MBeanWrapper) IMutation(org.apache.cassandra.db.IMutation) EndpointsForToken(org.apache.cassandra.locator.EndpointsForToken) Keyspace(org.apache.cassandra.db.Keyspace) ArrayList(java.util.ArrayList) AbstractReplicationStrategy(org.apache.cassandra.locator.AbstractReplicationStrategy) Replica(org.apache.cassandra.locator.Replica) IEndpointSnitch(org.apache.cassandra.locator.IEndpointSnitch)

Aggregations

EndpointsForToken (org.apache.cassandra.locator.EndpointsForToken)21 Test (org.junit.Test)9 SinglePartitionReadCommand (org.apache.cassandra.db.SinglePartitionReadCommand)8 PartitionUpdate (org.apache.cassandra.db.partitions.PartitionUpdate)8 Token (org.apache.cassandra.dht.Token)8 Replica (org.apache.cassandra.locator.Replica)6 ReplicaPlan (org.apache.cassandra.locator.ReplicaPlan)5 Keyspace (org.apache.cassandra.db.Keyspace)4 InetAddressAndPort (org.apache.cassandra.locator.InetAddressAndPort)4 ReplicaLayout (org.apache.cassandra.locator.ReplicaLayout)4 UncheckedInterruptedException (org.apache.cassandra.utils.concurrent.UncheckedInterruptedException)4 List (java.util.List)3 UUID (java.util.UUID)3 Collectors (java.util.stream.Collectors)3 DatabaseDescriptor (org.apache.cassandra.config.DatabaseDescriptor)3 ColumnFamilyStore (org.apache.cassandra.db.ColumnFamilyStore)3 ConsistencyLevel (org.apache.cassandra.db.ConsistencyLevel)3 IMutation (org.apache.cassandra.db.IMutation)3 Mutation (org.apache.cassandra.db.Mutation)3 ReadFailureException (org.apache.cassandra.exceptions.ReadFailureException)3