Examples with BlockingBuffer com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer used on opensource projects

Example 1 with BlockingBuffer

use of com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer in project data-prepper by opensearch-project.

the class HTTPSourceTest method getBuffer.

private BlockingBuffer<Record<Log>> getBuffer() {
    final HashMap<String, Object> integerHashMap = new HashMap<>();
    integerHashMap.put("buffer_size", 1);
    integerHashMap.put("batch_size", 1);
    final PluginSetting pluginSetting = new PluginSetting("blocking_buffer", integerHashMap) {

        {
            setPipelineName(TEST_PIPELINE_NAME);
        }
    };
    return new BlockingBuffer<>(pluginSetting);
}

Example 2 with BlockingBuffer

use of com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer in project data-prepper by opensearch-project.

the class FileSourceTests method getBuffer.

private BlockingBuffer<Record<Object>> getBuffer() {
    final HashMap<String, Object> integerHashMap = new HashMap<>();
    integerHashMap.put("buffer_size", 2);
    integerHashMap.put("batch_size", 2);
    final PluginSetting pluginSetting = new PluginSetting("blocking_buffer", integerHashMap);
    pluginSetting.setPipelineName(TEST_PIPELINE_NAME);
    return new BlockingBuffer<>(pluginSetting);
}

Example 3 with BlockingBuffer

use of com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer in project data-prepper by opensearch-project.

the class PipelineTests method testGetSource.

@Test
public void testGetSource() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertEquals(testSource, testPipeline.getSource());
}

Example 4 with BlockingBuffer

use of com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer in project data-prepper by opensearch-project.

the class PipelineTests method testExecuteFailingSource.

@Test
public void testExecuteFailingSource() {
    final Source<Record<String>> testSource = new TestSource(true);
    final TestSink testSink = new TestSink();
    try {
        final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
        testPipeline.execute();
    } catch (Exception ex) {
        assertThat("Incorrect exception message", ex.getMessage().contains("Source is expected to fail"));
        assertThat("Exception while starting the source should have pipeline.isStopRequested to false", !testPipeline.isStopRequested());
    }
}

Example 5 with BlockingBuffer

use of com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer in project data-prepper by opensearch-project.

the class PipelineTests method testGetSinks.

@Test
public void testGetSinks() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertEquals(1, testPipeline.getSinks().size());
    assertEquals(testSink, testPipeline.getSinks().iterator().next());
}