Search in sources :

Example 1 with ShardCollectorProviderFactory

use of io.crate.execution.engine.collect.sources.ShardCollectorProviderFactory in project crate by crate.

the class RemoteCollectorFactory method createCollector.

/**
 * create a RemoteCollector
 * The RemoteCollector will collect data from another node using a wormhole as if it was collecting on this node.
 * <p>
 * This should only be used if a shard is not available on the current node due to a relocation
 */
public CompletableFuture<BatchIterator<Row>> createCollector(ShardId shardId, RoutedCollectPhase collectPhase, CollectTask collectTask, ShardCollectorProviderFactory shardCollectorProviderFactory) {
    ShardStateObserver shardStateObserver = new ShardStateObserver(clusterService);
    CompletableFuture<ShardRouting> shardBecameActive = shardStateObserver.waitForActiveShard(shardId);
    Runnable onClose = () -> {
    };
    Consumer<Throwable> kill = killReason -> {
        shardBecameActive.cancel(true);
        shardBecameActive.completeExceptionally(killReason);
    };
    return shardBecameActive.thenApply(activePrimaryRouting -> CollectingBatchIterator.newInstance(onClose, kill, () -> retrieveRows(activePrimaryRouting, collectPhase, collectTask, shardCollectorProviderFactory), true));
}
Also used : ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) ShardId(org.elasticsearch.index.shard.ShardId) Projections(io.crate.execution.dsl.projection.Projections) ClusterService(org.elasticsearch.cluster.service.ClusterService) Buckets(io.crate.data.Buckets) BatchIterator(io.crate.data.BatchIterator) CompletableFuture(java.util.concurrent.CompletableFuture) CollectingRowConsumer(io.crate.data.CollectingRowConsumer) Inject(org.elasticsearch.common.inject.Inject) ArrayList(java.util.ArrayList) Routing(io.crate.metadata.Routing) IntArrayList(com.carrotsearch.hppc.IntArrayList) RemoteCollector(io.crate.execution.engine.collect.collectors.RemoteCollector) Map(java.util.Map) ShardStateObserver(io.crate.execution.engine.collect.collectors.ShardStateObserver) ThreadPool(org.elasticsearch.threadpool.ThreadPool) DistributionInfo(io.crate.planner.distribution.DistributionInfo) IndicesService(org.elasticsearch.indices.IndicesService) Collector(java.util.stream.Collector) ShardCollectorProviderFactory(io.crate.execution.engine.collect.sources.ShardCollectorProviderFactory) Executor(java.util.concurrent.Executor) UUIDs(org.elasticsearch.common.UUIDs) RoutedCollectPhase(io.crate.execution.dsl.phases.RoutedCollectPhase) UUID(java.util.UUID) TransportActionProvider(io.crate.execution.TransportActionProvider) Collectors(java.util.stream.Collectors) Lists2(io.crate.common.collections.Lists2) CollectingBatchIterator(io.crate.data.CollectingBatchIterator) TasksService(io.crate.execution.jobs.TasksService) Consumer(java.util.function.Consumer) Exceptions(io.crate.exceptions.Exceptions) List(java.util.List) Row(io.crate.data.Row) Singleton(org.elasticsearch.common.inject.Singleton) ShardRouting(org.elasticsearch.cluster.routing.ShardRouting) ShardStateObserver(io.crate.execution.engine.collect.collectors.ShardStateObserver)

Aggregations

IntArrayList (com.carrotsearch.hppc.IntArrayList)1 Lists2 (io.crate.common.collections.Lists2)1 BatchIterator (io.crate.data.BatchIterator)1 Buckets (io.crate.data.Buckets)1 CollectingBatchIterator (io.crate.data.CollectingBatchIterator)1 CollectingRowConsumer (io.crate.data.CollectingRowConsumer)1 Row (io.crate.data.Row)1 Exceptions (io.crate.exceptions.Exceptions)1 TransportActionProvider (io.crate.execution.TransportActionProvider)1 RoutedCollectPhase (io.crate.execution.dsl.phases.RoutedCollectPhase)1 Projections (io.crate.execution.dsl.projection.Projections)1 RemoteCollector (io.crate.execution.engine.collect.collectors.RemoteCollector)1 ShardStateObserver (io.crate.execution.engine.collect.collectors.ShardStateObserver)1 ShardCollectorProviderFactory (io.crate.execution.engine.collect.sources.ShardCollectorProviderFactory)1 TasksService (io.crate.execution.jobs.TasksService)1 Routing (io.crate.metadata.Routing)1 DistributionInfo (io.crate.planner.distribution.DistributionInfo)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 Map (java.util.Map)1