Examples with DistResultRXTask io.crate.execution.jobs.DistResultRXTask used on opensource projects

Example 1 with DistResultRXTask

use of io.crate.execution.jobs.DistResultRXTask in project crate by crate.

the class RemoteCollector method createPageDownstreamContext.

private RootTask.Builder createPageDownstreamContext() {
    RootTask.Builder builder = tasksService.newBuilder(jobId, sessionSettings.userName(), localNode, Collections.emptySet());
    PassThroughPagingIterator<Integer, Row> pagingIterator;
    if (scrollRequired) {
        pagingIterator = PassThroughPagingIterator.repeatable();
    } else {
        pagingIterator = PassThroughPagingIterator.oneShot();
    }
    PageBucketReceiver pageBucketReceiver = new CumulativePageBucketReceiver(localNode, RECEIVER_PHASE_ID, executor, DataTypes.getStreamers(collectPhase.outputTypes()), consumer, pagingIterator, 1);
    builder.addTask(new DistResultRXTask(RECEIVER_PHASE_ID, "RemoteCollectPhase", pageBucketReceiver, ramAccounting, 1));
    return builder;
}

Example 2 with DistResultRXTask

use of io.crate.execution.jobs.DistResultRXTask in project crate by crate.

the class DistributingConsumerTest method testSendUsingDistributingConsumerAndReceiveWithDistResultRXTask.

@Test
public void testSendUsingDistributingConsumerAndReceiveWithDistResultRXTask() throws Exception {
    try {
        Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
        TestingRowConsumer collectingConsumer = new TestingRowConsumer();
        DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
        TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
        DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
        BatchSimulatingIterator<Row> batchSimulatingIterator = new BatchSimulatingIterator<>(TestingBatchIterators.range(0, 5), 2, 3, executorService);
        distributingConsumer.accept(batchSimulatingIterator, null);
        List<Object[]> result = collectingConsumer.getResult();
        assertThat(TestingHelpers.printedTable(new CollectionBucket(result)), is("0\n" + "1\n" + "2\n" + "3\n" + "4\n"));
        // pageSize=2 and 5 rows causes 3x pushResult
        verify(distributedResultAction, times(3)).pushResult(anyString(), any(), any());
    } finally {
        executorService.shutdown();
        executorService.awaitTermination(10, TimeUnit.SECONDS);
    }
}

Example 3 with DistResultRXTask

use of io.crate.execution.jobs.DistResultRXTask in project crate by crate.

the class DistributingConsumerTest method testFailureOnAllLoadedIsForwarded.

@Test
public void testFailureOnAllLoadedIsForwarded() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingRowConsumer collectingConsumer = new TestingRowConsumer();
    DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    distributingConsumer.accept(FailingBatchIterator.failOnAllLoaded(), null);
    expectedException.expect(InterruptedException.class);
    collectingConsumer.getResult();
}

Example 4 with DistResultRXTask

use of io.crate.execution.jobs.DistResultRXTask in project crate by crate.

the class DistributingConsumerTest method testDistributingConsumerForwardsFailure.

@Test
public void testDistributingConsumerForwardsFailure() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingRowConsumer collectingConsumer = new TestingRowConsumer();
    DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    distributingConsumer.accept(null, new CompletionException(new IllegalArgumentException("foobar")));
    expectedException.expect(IllegalArgumentException.class);
    expectedException.expectMessage("foobar");
    collectingConsumer.getResult();
}

Example 5 with DistResultRXTask

use of io.crate.execution.jobs.DistResultRXTask in project crate by crate.

the class DistributingConsumerTest method test_exception_on_loadNextBatch_is_forwarded.

@Test
public void test_exception_on_loadNextBatch_is_forwarded() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingRowConsumer collectingConsumer = new TestingRowConsumer();
    DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    BatchSimulatingIterator<Row> batchSimulatingIterator = new BatchSimulatingIterator<>(TestingBatchIterators.range(0, 5), 2, 3, executorService) {

        @Override
        public CompletionStage<?> loadNextBatch() {
            throw new CircuitBreakingException("data too large");
        }
    };
    distributingConsumer.accept(batchSimulatingIterator, null);
    expectedException.expect(CircuitBreakingException.class);
    collectingConsumer.getResult();
}