Examples with Streamer io.crate.Streamer used on opensource projects

Example 11 with Streamer

use of io.crate.Streamer in project crate by crate.

the class DistributingConsumerTest method testSendUsingDistributingConsumerAndReceiveWithDistResultRXTask.

@Test
public void testSendUsingDistributingConsumerAndReceiveWithDistResultRXTask() throws Exception {
    try {
        Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
        TestingRowConsumer collectingConsumer = new TestingRowConsumer();
        DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
        TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
        DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
        BatchSimulatingIterator<Row> batchSimulatingIterator = new BatchSimulatingIterator<>(TestingBatchIterators.range(0, 5), 2, 3, executorService);
        distributingConsumer.accept(batchSimulatingIterator, null);
        List<Object[]> result = collectingConsumer.getResult();
        assertThat(TestingHelpers.printedTable(new CollectionBucket(result)), is("0\n" + "1\n" + "2\n" + "3\n" + "4\n"));
        // pageSize=2 and 5 rows causes 3x pushResult
        verify(distributedResultAction, times(3)).pushResult(anyString(), any(), any());
    } finally {
        executorService.shutdown();
        executorService.awaitTermination(10, TimeUnit.SECONDS);
    }
}

Example 12 with Streamer

use of io.crate.Streamer in project crate by crate.

the class DistributingConsumerTest method testFailureOnAllLoadedIsForwarded.

@Test
public void testFailureOnAllLoadedIsForwarded() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingRowConsumer collectingConsumer = new TestingRowConsumer();
    DistResultRXTask distResultRXTask = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, distResultRXTask);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    distributingConsumer.accept(FailingBatchIterator.failOnAllLoaded(), null);
    expectedException.expect(InterruptedException.class);
    collectingConsumer.getResult();
}

Example 13 with Streamer

use of io.crate.Streamer in project crate by crate.

the class ContextPreparer method registerContextPhases.

private void registerContextPhases(Iterable<? extends NodeOperation> nodeOperations, PreparerContext preparerContext) {
    for (NodeOperation nodeOperation : nodeOperations) {
        // context for nodeOperations without dependencies can be built immediately (e.g. FetchPhase)
        if (nodeOperation.downstreamExecutionPhaseId() == NodeOperation.NO_DOWNSTREAM) {
            logger.trace("Building context for nodeOp without downstream: {}", nodeOperation);
            if (createContexts(nodeOperation.executionPhase(), preparerContext)) {
                preparerContext.opCtx.builtNodeOperations.set(nodeOperation.executionPhase().phaseId());
            }
        }
        if (ExecutionPhases.hasDirectResponseDownstream(nodeOperation.downstreamNodes())) {
            Streamer<?>[] streamers = StreamerVisitor.streamersFromOutputs(nodeOperation.executionPhase());
            SingleBucketBuilder bucketBuilder = new SingleBucketBuilder(streamers);
            preparerContext.directResponseFutures.add(bucketBuilder.completionFuture());
            preparerContext.registerBatchConsumer(nodeOperation.downstreamExecutionPhaseId(), bucketBuilder);
        }
    }
}

Example 14 with Streamer

use of io.crate.Streamer in project crate by crate.

the class DistributingConsumerTest method testSendUsingDistributingConsumerAndReceiveWithPageDownstreamContext.

@Test
public void testSendUsingDistributingConsumerAndReceiveWithPageDownstreamContext() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingBatchConsumer collectingConsumer = new TestingBatchConsumer();
    PageDownstreamContext pageDownstreamContext = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, pageDownstreamContext);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    distributingConsumer.accept(TestingBatchIterators.range(0, 5), null);
    List<Object[]> result = collectingConsumer.getResult();
    assertThat(TestingHelpers.printedTable(new CollectionBucket(result)), is("0\n" + "1\n" + "2\n" + "3\n" + "4\n"));
    // pageSize=2 and 5 rows causes 3x pushResult
    verify(distributedResultAction, times(3)).pushResult(anyString(), any(), any());
}

Example 15 with Streamer

use of io.crate.Streamer in project crate by crate.

the class DistributingConsumerTest method testDistributingConsumerForwardsFailure.

@Test
public void testDistributingConsumerForwardsFailure() throws Exception {
    Streamer<?>[] streamers = { DataTypes.INTEGER.streamer() };
    TestingBatchConsumer collectingConsumer = new TestingBatchConsumer();
    PageDownstreamContext pageDownstreamContext = createPageDownstreamContext(streamers, collectingConsumer);
    TransportDistributedResultAction distributedResultAction = createFakeTransport(streamers, pageDownstreamContext);
    DistributingConsumer distributingConsumer = createDistributingConsumer(streamers, distributedResultAction);
    distributingConsumer.accept(null, new CompletionException(new IllegalArgumentException("foobar")));
    expectedException.expect(IllegalArgumentException.class);
    expectedException.expectMessage("foobar");
    collectingConsumer.getResult();
}