Examples with Twister2BoundedSource org.apache.beam.runners.twister2.translation.wrappers.Twister2BoundedSource used on opensource projects

Example 1 with Twister2BoundedSource

use of org.apache.beam.runners.twister2.translation.wrappers.Twister2BoundedSource in project twister2 by DSC-SPIDAL.

the class ReadSourceTranslatorBatch method translateNode.

@Override
public void translateNode(Read.Bounded<T> transform, Twister2BatchTranslationContext context) {
    BoundedSource<T> boundedSource = transform.getSource();
    Twister2BoundedSource<T> twister2BoundedSource = new Twister2BoundedSource<T>(boundedSource, context, context.getOptions());
    final TSetEnvironment tsetEnv = context.getEnvironment();
    // TODO: need to set paralliem value
    SourceTSet<WindowedValue<T>> sourceTSet = ((BatchEnvironment) tsetEnv).createSource(twister2BoundedSource, 1);
    PCollection<T> output = context.getOutput(transform);
    context.setOutputDataSet(output, sourceTSet);
}

Example 2 with Twister2BoundedSource

use of org.apache.beam.runners.twister2.translation.wrappers.Twister2BoundedSource in project beam by apache.

the class ReadSourceTranslatorBatch method translateNode.

@Override
public void translateNode(SplittableParDo.PrimitiveBoundedRead<T> transform, Twister2BatchTranslationContext context) {
    BoundedSource<T> boundedSource = transform.getSource();
    Twister2BoundedSource<T> twister2BoundedSource = new Twister2BoundedSource<T>(boundedSource, context, context.getOptions());
    final TSetEnvironment tsetEnv = context.getEnvironment();
    SourceTSet<WindowedValue<T>> sourceTSet = ((BatchTSetEnvironment) tsetEnv).createSource(twister2BoundedSource, context.getOptions().getParallelism());
    PCollection<T> output = context.getOutput(transform);
    context.setOutputDataSet(output, sourceTSet);
}