Examples with SplittableParDo org.apache.beam.runners.core.construction.SplittableParDo used on opensource projects

Example 1 with SplittableParDo

use of org.apache.beam.runners.core.construction.SplittableParDo in project beam by apache.

the class ParDoMultiOverrideFactory method getReplacementTransform.

@SuppressWarnings("unchecked")
private PTransform<PCollection<? extends InputT>, PCollectionTuple> getReplacementTransform(MultiOutput<InputT, OutputT> transform) {
    DoFn<InputT, OutputT> fn = transform.getFn();
    DoFnSignature signature = DoFnSignatures.getSignature(fn.getClass());
    if (signature.processElement().isSplittable()) {
        return new SplittableParDo(transform);
    } else if (signature.stateDeclarations().size() > 0 || signature.timerDeclarations().size() > 0) {
        // Based on the fact that the signature is stateful, DoFnSignatures ensures
        // that it is also keyed
        MultiOutput<KV<?, ?>, OutputT> keyedTransform = (MultiOutput<KV<?, ?>, OutputT>) transform;
        return new GbkThenStatefulParDo(keyedTransform);
    } else {
        return transform;
    }
}