Search in sources :

Example 1 with AssertResultSink

use of org.talend.components.adapter.beam.example.AssertResultSink in project components by Talend.

the class TCompBoundedSourceSinkAdapterTest method testSink.

@Test
public void testSink() {
    Pipeline pipeline = TestPipeline.create();
    AssertResultProperties assertResultProperties = new AssertResultProperties("assertResultProperties");
    assertResultProperties.init();
    assertResultProperties.data.setValue("b;c;a");
    assertResultProperties.rowDelimited.setValue(";");
    AssertResultSink assertResultSink = new AssertResultSink();
    assertResultSink.initialize(null, assertResultProperties);
    TCompSinkAdapter sink = new TCompSinkAdapter(assertResultSink);
    final String schemaStr = assertResultProperties.schema.getValue().toString();
    pipeline.apply(Create.of("a", "b", "c")).apply(ParDo.of(new DoFn<String, IndexedRecord>() {

        @DoFn.ProcessElement
        public void processElement(ProcessContext c) throws Exception {
            IndexedRecord row = new GenericData.Record(new Schema.Parser().parse(schemaStr));
            row.put(0, c.element());
            c.output(row);
        }
    })).setCoder(LazyAvroCoder.of()).apply(Write.to(sink));
    pipeline.run();
}
Also used : AssertResultProperties(org.talend.components.adapter.beam.example.AssertResultProperties) AssertResultSink(org.talend.components.adapter.beam.example.AssertResultSink) DoFn(org.apache.beam.sdk.transforms.DoFn) IndexedRecord(org.apache.avro.generic.IndexedRecord) IndexedRecord(org.apache.avro.generic.IndexedRecord) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Pipeline(org.apache.beam.sdk.Pipeline) Test(org.junit.Test)

Example 2 with AssertResultSink

use of org.talend.components.adapter.beam.example.AssertResultSink in project components by Talend.

the class TCompBoundedSourceSinkAdapterTest method testPipeline.

@Test
public void testPipeline() {
    Pipeline pipeline = TestPipeline.create();
    FixedFlowProperties fixedFlowProperties = new FixedFlowProperties("fixedFlowProperties");
    fixedFlowProperties.init();
    fixedFlowProperties.data.setValue("a;b;c");
    fixedFlowProperties.rowDelimited.setValue(";");
    AssertResultProperties assertResultProperties = new AssertResultProperties("assertResultProperties");
    assertResultProperties.init();
    assertResultProperties.data.setValue("b;c;a");
    assertResultProperties.rowDelimited.setValue(";");
    FixedFlowSource fixedFlowSource = new FixedFlowSource();
    fixedFlowSource.initialize(null, fixedFlowProperties);
    AssertResultSink assertResultSink = new AssertResultSink();
    assertResultSink.initialize(null, assertResultProperties);
    TCompBoundedSourceAdapter source = new TCompBoundedSourceAdapter(fixedFlowSource);
    TCompSinkAdapter sink = new TCompSinkAdapter(assertResultSink);
    pipeline.apply(Read.from(source)).apply(Write.to(sink));
    pipeline.run();
}
Also used : AssertResultProperties(org.talend.components.adapter.beam.example.AssertResultProperties) FixedFlowSource(org.talend.components.adapter.beam.example.FixedFlowSource) AssertResultSink(org.talend.components.adapter.beam.example.AssertResultSink) FixedFlowProperties(org.talend.components.adapter.beam.example.FixedFlowProperties) TestPipeline(org.apache.beam.sdk.testing.TestPipeline) Pipeline(org.apache.beam.sdk.Pipeline) Test(org.junit.Test)

Aggregations

Pipeline (org.apache.beam.sdk.Pipeline)2 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)2 Test (org.junit.Test)2 AssertResultProperties (org.talend.components.adapter.beam.example.AssertResultProperties)2 AssertResultSink (org.talend.components.adapter.beam.example.AssertResultSink)2 IndexedRecord (org.apache.avro.generic.IndexedRecord)1 DoFn (org.apache.beam.sdk.transforms.DoFn)1 FixedFlowProperties (org.talend.components.adapter.beam.example.FixedFlowProperties)1 FixedFlowSource (org.talend.components.adapter.beam.example.FixedFlowSource)1