Search in sources :

Example 1 with PreferedNodeFilter

use of org.apache.cassandra.repair.asymmetric.PreferedNodeFilter in project cassandra by apache.

the class RepairJob method createOptimisedSyncingSyncTasks.

static List<SyncTask> createOptimisedSyncingSyncTasks(RepairJobDesc desc, List<TreeResponse> trees, InetAddressAndPort local, Predicate<InetAddressAndPort> isTransient, Function<InetAddressAndPort, String> getDC, boolean isIncremental, PreviewKind previewKind) {
    long startedAt = currentTimeMillis();
    List<SyncTask> syncTasks = new ArrayList<>();
    // We need to difference all trees one against another
    DifferenceHolder diffHolder = new DifferenceHolder(trees);
    logger.trace("diffs = {}", diffHolder);
    PreferedNodeFilter preferSameDCFilter = (streaming, candidates) -> candidates.stream().filter(node -> getDC.apply(streaming).equals(getDC.apply(node))).collect(Collectors.toSet());
    ImmutableMap<InetAddressAndPort, HostDifferences> reducedDifferences = ReduceHelper.reduce(diffHolder, preferSameDCFilter);
    for (int i = 0; i < trees.size(); i++) {
        InetAddressAndPort address = trees.get(i).endpoint;
        // we don't stream to transient replicas
        if (isTransient.test(address))
            continue;
        HostDifferences streamsFor = reducedDifferences.get(address);
        if (streamsFor != null) {
            Preconditions.checkArgument(streamsFor.get(address).isEmpty(), "We should not fetch ranges from ourselves");
            for (InetAddressAndPort fetchFrom : streamsFor.hosts()) {
                List<Range<Token>> toFetch = new ArrayList<>(streamsFor.get(fetchFrom));
                assert !toFetch.isEmpty();
                logger.trace("{} is about to fetch {} from {}", address, toFetch, fetchFrom);
                SyncTask task;
                if (address.equals(local)) {
                    task = new LocalSyncTask(desc, address, fetchFrom, toFetch, isIncremental ? desc.parentSessionId : null, true, false, previewKind);
                } else {
                    task = new AsymmetricRemoteSyncTask(desc, address, fetchFrom, toFetch, previewKind);
                }
                syncTasks.add(task);
            }
        } else {
            logger.trace("Node {} has nothing to stream", address);
        }
    }
    logger.info("Created {} optimised sync tasks based on {} merkle tree responses for {} (took: {}ms)", syncTasks.size(), trees.size(), desc.parentSessionId, currentTimeMillis() - startedAt);
    logger.trace("Optimised sync tasks for {}: {}", desc.parentSessionId, syncTasks);
    return syncTasks;
}
Also used : InetAddressAndPort(org.apache.cassandra.locator.InetAddressAndPort) java.util(java.util) MerkleTrees(org.apache.cassandra.utils.MerkleTrees) LoggerFactory(org.slf4j.LoggerFactory) Global.currentTimeMillis(org.apache.cassandra.utils.Clock.Global.currentTimeMillis) Range(org.apache.cassandra.dht.Range) com.google.common.util.concurrent(com.google.common.util.concurrent) Function(java.util.function.Function) SystemDistributedKeyspace(org.apache.cassandra.schema.SystemDistributedKeyspace) PreferedNodeFilter(org.apache.cassandra.repair.asymmetric.PreferedNodeFilter) ActiveRepairService(org.apache.cassandra.service.ActiveRepairService) Token(org.apache.cassandra.dht.Token) DifferenceHolder(org.apache.cassandra.repair.asymmetric.DifferenceHolder) Pair(org.apache.cassandra.utils.Pair) DatabaseDescriptor(org.apache.cassandra.config.DatabaseDescriptor) Keyspace(org.apache.cassandra.db.Keyspace) ReduceHelper(org.apache.cassandra.repair.asymmetric.ReduceHelper) FutureCombiner(org.apache.cassandra.utils.concurrent.FutureCombiner) Logger(org.slf4j.Logger) FBUtilities(org.apache.cassandra.utils.FBUtilities) AsyncFuture(org.apache.cassandra.utils.concurrent.AsyncFuture) ImmutableMap(com.google.common.collect.ImmutableMap) Predicate(java.util.function.Predicate) Tracing(org.apache.cassandra.tracing.Tracing) Collectors(java.util.stream.Collectors) ExecutorPlus(org.apache.cassandra.concurrent.ExecutorPlus) PreviewKind(org.apache.cassandra.streaming.PreviewKind) ColumnFamilyStore(org.apache.cassandra.db.ColumnFamilyStore) Future(org.apache.cassandra.utils.concurrent.Future) ImmediateFuture(org.apache.cassandra.utils.concurrent.ImmediateFuture) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) HostDifferences(org.apache.cassandra.repair.asymmetric.HostDifferences) InetAddressAndPort(org.apache.cassandra.locator.InetAddressAndPort) Range(org.apache.cassandra.dht.Range) PreferedNodeFilter(org.apache.cassandra.repair.asymmetric.PreferedNodeFilter) HostDifferences(org.apache.cassandra.repair.asymmetric.HostDifferences) DifferenceHolder(org.apache.cassandra.repair.asymmetric.DifferenceHolder)

Aggregations

VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 Preconditions (com.google.common.base.Preconditions)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 com.google.common.util.concurrent (com.google.common.util.concurrent)1 java.util (java.util)1 Function (java.util.function.Function)1 Predicate (java.util.function.Predicate)1 Collectors (java.util.stream.Collectors)1 ExecutorPlus (org.apache.cassandra.concurrent.ExecutorPlus)1 DatabaseDescriptor (org.apache.cassandra.config.DatabaseDescriptor)1 ColumnFamilyStore (org.apache.cassandra.db.ColumnFamilyStore)1 Keyspace (org.apache.cassandra.db.Keyspace)1 Range (org.apache.cassandra.dht.Range)1 Token (org.apache.cassandra.dht.Token)1 InetAddressAndPort (org.apache.cassandra.locator.InetAddressAndPort)1 DifferenceHolder (org.apache.cassandra.repair.asymmetric.DifferenceHolder)1 HostDifferences (org.apache.cassandra.repair.asymmetric.HostDifferences)1 PreferedNodeFilter (org.apache.cassandra.repair.asymmetric.PreferedNodeFilter)1 ReduceHelper (org.apache.cassandra.repair.asymmetric.ReduceHelper)1 SystemDistributedKeyspace (org.apache.cassandra.schema.SystemDistributedKeyspace)1