Search in sources :

Example 1 with FixedFlowSource

use of org.talend.components.adapter.beam.example.FixedFlowSource in project components by Talend.

the class TCompBoundedSourceSinkAdapterTest method testSource.

@Test
public void testSource() {
    Pipeline pipeline = TestPipeline.create();
    FixedFlowProperties fixedFlowProperties = new FixedFlowProperties("fixedFlowProperties");
    fixedFlowProperties.init();
    fixedFlowProperties.data.setValue("a;b;c");
    fixedFlowProperties.rowDelimited.setValue(";");
    FixedFlowSource fixedFlowSource = new FixedFlowSource();
    fixedFlowSource.initialize(null, fixedFlowProperties);
    TCompBoundedSourceAdapter source = new TCompBoundedSourceAdapter(fixedFlowSource);
    PCollection<String> result = pipeline.apply(Read.from(source)).apply(ParDo.of(new DoFn<IndexedRecord, String>() {

        @DoFn.ProcessElement
        public void processElement(ProcessContext c) throws Exception {
            c.output(c.element().get(0).toString());
        }
    }));
    PAssert.that(result).containsInAnyOrder(Arrays.asList("a", "b", "c"));
    pipeline.run();
}
Also used : FixedFlowSource(org.talend.components.adapter.beam.example.FixedFlowSource) DoFn(org.apache.beam.sdk.transforms.DoFn) FixedFlowProperties(org.talend.components.adapter.beam.example.FixedFlowProperties) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Pipeline(org.apache.beam.sdk.Pipeline) Test(org.junit.Test)

Example 2 with FixedFlowSource

use of org.talend.components.adapter.beam.example.FixedFlowSource in project components by Talend.

the class TCompBoundedSourceSinkAdapterTest method testPipeline.

@Test
public void testPipeline() {
    Pipeline pipeline = TestPipeline.create();
    FixedFlowProperties fixedFlowProperties = new FixedFlowProperties("fixedFlowProperties");
    fixedFlowProperties.init();
    fixedFlowProperties.data.setValue("a;b;c");
    fixedFlowProperties.rowDelimited.setValue(";");
    AssertResultProperties assertResultProperties = new AssertResultProperties("assertResultProperties");
    assertResultProperties.init();
    assertResultProperties.data.setValue("b;c;a");
    assertResultProperties.rowDelimited.setValue(";");
    FixedFlowSource fixedFlowSource = new FixedFlowSource();
    fixedFlowSource.initialize(null, fixedFlowProperties);
    AssertResultSink assertResultSink = new AssertResultSink();
    assertResultSink.initialize(null, assertResultProperties);
    TCompBoundedSourceAdapter source = new TCompBoundedSourceAdapter(fixedFlowSource);
    TCompSinkAdapter sink = new TCompSinkAdapter(assertResultSink);
    pipeline.apply(Read.from(source)).apply(Write.to(sink));
    pipeline.run();
}
Also used : AssertResultProperties(org.talend.components.adapter.beam.example.AssertResultProperties) FixedFlowSource(org.talend.components.adapter.beam.example.FixedFlowSource) AssertResultSink(org.talend.components.adapter.beam.example.AssertResultSink) FixedFlowProperties(org.talend.components.adapter.beam.example.FixedFlowProperties) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Pipeline(org.apache.beam.sdk.Pipeline) Test(org.junit.Test)

Aggregations

Pipeline (org.apache.beam.sdk.Pipeline)2 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)2 Test (org.junit.Test)2 FixedFlowProperties (org.talend.components.adapter.beam.example.FixedFlowProperties)2 FixedFlowSource (org.talend.components.adapter.beam.example.FixedFlowSource)2 DoFn (org.apache.beam.sdk.transforms.DoFn)1 AssertResultProperties (org.talend.components.adapter.beam.example.AssertResultProperties)1 AssertResultSink (org.talend.components.adapter.beam.example.AssertResultSink)1