Search in sources :

Example 1 with TestSink

use of com.amazon.dataprepper.plugins.TestSink in project data-prepper by opensearch-project.

the class PipelineTests method testGetSource.

@Test
public void testGetSource() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertEquals(testSource, testPipeline.getSource());
}
Also used : TestSource(com.amazon.dataprepper.plugins.TestSource) BlockingBuffer(com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer) Record(com.amazon.dataprepper.model.record.Record) TestSink(com.amazon.dataprepper.plugins.TestSink) Test(org.junit.Test)

Example 2 with TestSink

use of com.amazon.dataprepper.plugins.TestSink in project data-prepper by opensearch-project.

the class PipelineTests method testExecuteFailingSource.

@Test
public void testExecuteFailingSource() {
    final Source<Record<String>> testSource = new TestSource(true);
    final TestSink testSink = new TestSink();
    try {
        final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
        testPipeline.execute();
    } catch (Exception ex) {
        assertThat("Incorrect exception message", ex.getMessage().contains("Source is expected to fail"));
        assertThat("Exception while starting the source should have pipeline.isStopRequested to false", !testPipeline.isStopRequested());
    }
}
Also used : TestSource(com.amazon.dataprepper.plugins.TestSource) BlockingBuffer(com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer) Record(com.amazon.dataprepper.model.record.Record) TestSink(com.amazon.dataprepper.plugins.TestSink) Test(org.junit.Test)

Example 3 with TestSink

use of com.amazon.dataprepper.plugins.TestSink in project data-prepper by opensearch-project.

the class PipelineTests method testGetSinks.

@Test
public void testGetSinks() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertEquals(1, testPipeline.getSinks().size());
    assertEquals(testSink, testPipeline.getSinks().iterator().next());
}
Also used : TestSource(com.amazon.dataprepper.plugins.TestSource) BlockingBuffer(com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer) Record(com.amazon.dataprepper.model.record.Record) TestSink(com.amazon.dataprepper.plugins.TestSink) Test(org.junit.Test)

Example 4 with TestSink

use of com.amazon.dataprepper.plugins.TestSink in project data-prepper by opensearch-project.

the class PipelineTests method testPipelineState.

@Test
public void testPipelineState() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.emptyList(), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertThat("Pipeline isStopRequested is expected to be false", testPipeline.isStopRequested(), is(false));
    assertThat("Pipeline is expected to have a default buffer", testPipeline.getBuffer(), notNullValue());
    assertTrue("Pipeline processors should be empty", testPipeline.getProcessorSets().isEmpty());
    testPipeline.execute();
    assertThat("Pipeline isStopRequested is expected to be false", testPipeline.isStopRequested(), is(false));
    testPipeline.shutdown();
    assertThat("Pipeline isStopRequested is expected to be true", testPipeline.isStopRequested(), is(true));
    assertThat("Sink shutdown should be called", testSink.isShutdown, is(true));
}
Also used : TestSource(com.amazon.dataprepper.plugins.TestSource) BlockingBuffer(com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer) Record(com.amazon.dataprepper.model.record.Record) TestSink(com.amazon.dataprepper.plugins.TestSink) Test(org.junit.Test)

Example 5 with TestSink

use of com.amazon.dataprepper.plugins.TestSink in project data-prepper by opensearch-project.

the class PipelineTests method testPipelineStateWithPrepper.

@Test
public void testPipelineStateWithPrepper() {
    final Source<Record<String>> testSource = new TestSource();
    final TestSink testSink = new TestSink();
    final TestProcessor testProcessor = new TestPrepper(new PluginSetting("test_processor", new HashMap<>()));
    final Pipeline testPipeline = new Pipeline(TEST_PIPELINE_NAME, testSource, new BlockingBuffer(TEST_PIPELINE_NAME), Collections.singletonList(Collections.singletonList(testProcessor)), Collections.singletonList(testSink), TEST_PROCESSOR_THREADS, TEST_READ_BATCH_TIMEOUT);
    assertThat("Pipeline isStopRequested is expected to be false", testPipeline.isStopRequested(), is(false));
    assertThat("Pipeline is expected to have a default buffer", testPipeline.getBuffer(), notNullValue());
    assertEquals("Pipeline processorSets size should be 1", 1, testPipeline.getProcessorSets().size());
    testPipeline.execute();
    assertThat("Pipeline isStopRequested is expected to be false", testPipeline.isStopRequested(), is(false));
    testPipeline.shutdown();
    assertThat("Pipeline isStopRequested is expected to be true", testPipeline.isStopRequested(), is(true));
    assertThat("Sink shutdown should be called", testSink.isShutdown, is(true));
    assertThat("Processor shutdown should be called", testProcessor.isShutdown, is(true));
}
Also used : TestSource(com.amazon.dataprepper.plugins.TestSource) HashMap(java.util.HashMap) BlockingBuffer(com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer) TestPrepper(com.amazon.dataprepper.pipeline.common.TestPrepper) TestProcessor(com.amazon.dataprepper.pipeline.common.TestProcessor) Record(com.amazon.dataprepper.model.record.Record) TestSink(com.amazon.dataprepper.plugins.TestSink) PluginSetting(com.amazon.dataprepper.model.configuration.PluginSetting) Test(org.junit.Test)

Aggregations

Record (com.amazon.dataprepper.model.record.Record)6 TestSink (com.amazon.dataprepper.plugins.TestSink)6 TestSource (com.amazon.dataprepper.plugins.TestSource)6 BlockingBuffer (com.amazon.dataprepper.plugins.buffer.blockingbuffer.BlockingBuffer)6 Test (org.junit.Test)6 PluginSetting (com.amazon.dataprepper.model.configuration.PluginSetting)2 TestProcessor (com.amazon.dataprepper.pipeline.common.TestProcessor)2 HashMap (java.util.HashMap)2 TestPrepper (com.amazon.dataprepper.pipeline.common.TestPrepper)1