Examples with JobExecutionContext io.crate.jobs.JobExecutionContext used on opensource projects

Example 1 with JobExecutionContext

use of io.crate.jobs.JobExecutionContext in project crate by crate.

the class TransportJobAction method nodeOperation.

@Override
public void nodeOperation(final JobRequest request, final ActionListener<JobResponse> actionListener) {
    JobExecutionContext.Builder contextBuilder = jobContextService.newBuilder(request.jobId(), request.coordinatorNodeId());
    SharedShardContexts sharedShardContexts = new SharedShardContexts(indicesService);
    List<CompletableFuture<Bucket>> directResponseFutures = contextPreparer.prepareOnRemote(request.nodeOperations(), contextBuilder, sharedShardContexts);
    try {
        JobExecutionContext context = jobContextService.createContext(contextBuilder);
        context.start();
    } catch (Throwable t) {
        actionListener.onFailure(t);
        return;
    }
    if (directResponseFutures.size() == 0) {
        actionListener.onResponse(new JobResponse());
    } else {
        CompletableFutures.allAsList(directResponseFutures).whenComplete((buckets, t) -> {
            if (t == null) {
                actionListener.onResponse(new JobResponse(buckets));
            } else {
                actionListener.onFailure(SQLExceptions.unwrap(t));
            }
        });
    }
}

Example 2 with JobExecutionContext

use of io.crate.jobs.JobExecutionContext in project crate by crate.

the class DocLevelCollectTest method collect.

private Bucket collect(RoutedCollectPhase collectNode) throws Throwable {
    ContextPreparer contextPreparer = internalCluster().getDataNodeInstance(ContextPreparer.class);
    JobContextService contextService = internalCluster().getDataNodeInstance(JobContextService.class);
    SharedShardContexts sharedShardContexts = new SharedShardContexts(internalCluster().getDataNodeInstance(IndicesService.class));
    JobExecutionContext.Builder builder = contextService.newBuilder(collectNode.jobId());
    NodeOperation nodeOperation = NodeOperation.withDownstream(collectNode, mock(ExecutionPhase.class), (byte) 0, "remoteNode");
    List<CompletableFuture<Bucket>> results = contextPreparer.prepareOnRemote(ImmutableList.of(nodeOperation), builder, sharedShardContexts);
    JobExecutionContext context = contextService.createContext(builder);
    context.start();
    return results.get(0).get(2, TimeUnit.SECONDS);
}

Example 3 with JobExecutionContext

use of io.crate.jobs.JobExecutionContext in project crate by crate.

the class TransportDistributedResultAction method nodeOperation.

private void nodeOperation(final DistributedResultRequest request, final ActionListener<DistributedResultResponse> listener, final int retry) {
    JobExecutionContext context = jobContextService.getContextOrNull(request.jobId());
    if (context == null) {
        retryOrFailureResponse(request, listener, retry);
        return;
    }
    DownstreamExecutionSubContext executionContext;
    try {
        executionContext = context.getSubContext(request.executionPhaseId());
    } catch (ClassCastException e) {
        listener.onFailure(new IllegalStateException(String.format(Locale.ENGLISH, "Found execution context for %d but it's not a downstream context", request.executionPhaseId()), e));
        return;
    } catch (Throwable t) {
        listener.onFailure(t);
        return;
    }
    PageBucketReceiver pageBucketReceiver = executionContext.getBucketReceiver(request.executionPhaseInputId());
    if (pageBucketReceiver == null) {
        listener.onFailure(new IllegalStateException(String.format(Locale.ENGLISH, "Couldn't find BucketReciever for input %d", request.executionPhaseInputId())));
        return;
    }
    Throwable throwable = request.throwable();
    if (throwable == null) {
        request.streamers(pageBucketReceiver.streamers());
        pageBucketReceiver.setBucket(request.bucketIdx(), request.rows(), request.isLast(), new SendResponsePageResultListener(listener));
    } else {
        if (request.isKilled()) {
            pageBucketReceiver.killed(request.bucketIdx(), throwable);
        } else {
            pageBucketReceiver.failure(request.bucketIdx(), throwable);
        }
        listener.onResponse(new DistributedResultResponse(false));
    }
}

Example 4 with JobExecutionContext

use of io.crate.jobs.JobExecutionContext in project crate by crate.

the class NodeFetchOperation method fetch.

public ListenableFuture<IntObjectMap<StreamBucket>> fetch(UUID jobId, int phaseId, @Nullable IntObjectMap<? extends IntContainer> docIdsToFetch, boolean closeContextOnFinish) {
    SettableFuture<IntObjectMap<StreamBucket>> resultFuture = SettableFuture.create();
    logStartAndSetupLogFinished(jobId, phaseId, resultFuture);
    if (docIdsToFetch == null) {
        if (closeContextOnFinish) {
            tryCloseContext(jobId, phaseId);
        }
        return Futures.immediateFuture(new IntObjectHashMap<>(0));
    }
    JobExecutionContext context = jobContextService.getContext(jobId);
    FetchContext fetchContext = context.getSubContext(phaseId);
    if (closeContextOnFinish) {
        Futures.addCallback(resultFuture, new CloseContextCallback(fetchContext));
    }
    try {
        doFetch(fetchContext, resultFuture, docIdsToFetch);
    } catch (Throwable t) {
        resultFuture.setException(t);
    }
    return resultFuture;
}

Example 5 with JobExecutionContext

use of io.crate.jobs.JobExecutionContext in project crate by crate.

the class ESDeleteTask method startContext.

private void startContext() throws Throwable {
    assert builder != null : "Context must be created first";
    JobExecutionContext ctx = jobContextService.createContext(builder);
    ctx.start();
}