Examples with TestingBatchConsumer io.crate.testing.TestingBatchConsumer used on opensource projects

Example 41 with TestingBatchConsumer

use of io.crate.testing.TestingBatchConsumer in project crate by crate.

the class FileReadingCollectorTest method testCollectFromDirectory.

@Test
public void testCollectFromDirectory() throws Throwable {
    TestingBatchConsumer projector = getObjects(Paths.get(tmpFile.getParentFile().toURI()).toUri().toString() + "*");
    assertCorrectResult(projector.getBucket());
}

Example 42 with TestingBatchConsumer

use of io.crate.testing.TestingBatchConsumer in project crate by crate.

the class FileReadingCollectorTest method testCollectWithOneSocketTimeout.

@Test
public void testCollectWithOneSocketTimeout() throws Throwable {
    S3ObjectInputStream inputStream = mock(S3ObjectInputStream.class);
    when(inputStream.read(new byte[anyInt()], anyInt(), anyByte())).thenAnswer(// first line: foo
    new WriteBufferAnswer(new byte[] { 102, 111, 111, 10 })).thenThrow(// exception causes retry
    new SocketTimeoutException()).thenAnswer(// first line again, because of retry
    new WriteBufferAnswer(new byte[] { 102, 111, 111, 10 })).thenAnswer(// second line: bar
    new WriteBufferAnswer(new byte[] { 98, 97, 114, 10 })).thenReturn(-1);
    TestingBatchConsumer consumer = getObjects(Collections.singletonList("s3://fakebucket/foo"), null, inputStream);
    Bucket rows = consumer.getBucket();
    assertThat(rows.size(), is(2));
    assertThat(TestingHelpers.printedTable(rows), is("foo\nbar\n"));
}

Example 43 with TestingBatchConsumer

use of io.crate.testing.TestingBatchConsumer in project crate by crate.

the class FileReadingCollectorTest method testCollectWithEmptyLine.

@Test
public void testCollectWithEmptyLine() throws Throwable {
    TestingBatchConsumer consumer = getObjects(Paths.get(tmpFileEmptyLine.toURI()).toUri().toString());
    assertCorrectResult(consumer.getBucket());
}

Example 44 with TestingBatchConsumer

use of io.crate.testing.TestingBatchConsumer in project crate by crate.

the class JobCollectContextTest method setUp.

@Before
public void setUp() throws Exception {
    localNodeId = "dummyLocalNodeId";
    collectPhase = Mockito.mock(RoutedCollectPhase.class);
    Routing routing = Mockito.mock(Routing.class);
    when(routing.containsShards(localNodeId)).thenReturn(true);
    when(collectPhase.routing()).thenReturn(routing);
    when(collectPhase.maxRowGranularity()).thenReturn(RowGranularity.DOC);
    jobCollectContext = new JobCollectContext(collectPhase, mock(MapSideDataCollectOperation.class), localNodeId, ramAccountingContext, new TestingBatchConsumer(), mock(SharedShardContexts.class));
}

Example 45 with TestingBatchConsumer

use of io.crate.testing.TestingBatchConsumer in project crate by crate.

the class FileReadingCollectorTest method testCollectFromUriWithGlob.

@Test
public void testCollectFromUriWithGlob() throws Throwable {
    TestingBatchConsumer projector = getObjects(Paths.get(tmpFile.getParentFile().toURI()).toUri().toString() + "file*.json");
    assertCorrectResult(projector.getBucket());
}