Examples with AppliedPTransform org.apache.beam.sdk.runners.AppliedPTransform used on opensource projects

Example 6 with AppliedPTransform

use of org.apache.beam.sdk.runners.AppliedPTransform in project beam by apache.

the class PTransformTranslationTest method convert.

private RunnerApi.PTransform convert(ToAndFromProtoSpec spec, SdkComponents components) throws IOException {
    List<AppliedPTransform<?, ?, ?>> childTransforms = new ArrayList<>();
    for (ToAndFromProtoSpec child : spec.getChildren()) {
        childTransforms.add(child.getTransform());
        System.out.println("Converting child " + child);
        convert(child, components);
        // Sanity call
        components.getExistingPTransformId(child.getTransform());
    }
    PTransform convert = PTransformTranslation.toProto(spec.getTransform(), childTransforms, components);
    // Make sure the converted transform is registered. Convert it independently, but if this is a
    // child spec, the child must be in the components.
    components.registerPTransform(spec.getTransform(), childTransforms);
    return convert;
}

Example 7 with AppliedPTransform

use of org.apache.beam.sdk.runners.AppliedPTransform in project beam by apache.

the class PTransformMatchersTest method emptyFlattenWithNonFlatten.

@Test
public void emptyFlattenWithNonFlatten() {
    AppliedPTransform application = AppliedPTransform.<PCollection<Iterable<Object>>, PCollection<Object>, Flatten.Iterables<Object>>of("EmptyFlatten", Collections.<TupleTag<?>, PValue>emptyMap(), Collections.<TupleTag<?>, PValue>singletonMap(new TupleTag<Object>(), PCollection.createPrimitiveOutputInternal(p, WindowingStrategy.globalDefault(), IsBounded.BOUNDED)), Flatten.iterables(), /* This isn't actually possible to construct,
                                 * but for the sake of example */
    p);
    assertThat(PTransformMatchers.emptyFlatten().matches(application), is(false));
}

Example 8 with AppliedPTransform

use of org.apache.beam.sdk.runners.AppliedPTransform in project beam by apache.

the class PTransformMatchersTest method classEqualToDoesNotMatchSubclass.

@Test
public void classEqualToDoesNotMatchSubclass() {
    class MyPTransform extends PTransform<PCollection<KV<String, Integer>>, PCollection<Integer>> {

        @Override
        public PCollection<Integer> expand(PCollection<KV<String, Integer>> input) {
            return PCollection.createPrimitiveOutputInternal(input.getPipeline(), input.getWindowingStrategy(), input.isBounded());
        }
    }
    PTransformMatcher matcher = PTransformMatchers.classEqualTo(MyPTransform.class);
    MyPTransform subclass = new MyPTransform() {
    };
    assertThat(subclass.getClass(), not(Matchers.<Class<?>>equalTo(MyPTransform.class)));
    assertThat(subclass, instanceOf(MyPTransform.class));
    AppliedPTransform<?, ?, ?> application = getAppliedTransform(subclass);
    assertThat(matcher.matches(application), is(false));
}

Example 9 with AppliedPTransform

use of org.apache.beam.sdk.runners.AppliedPTransform in project beam by apache.

the class PTransformMatchersTest method flattenWithDuplicateInputsNonFlatten.

@Test
public void flattenWithDuplicateInputsNonFlatten() {
    AppliedPTransform application = AppliedPTransform.<PCollection<Iterable<Object>>, PCollection<Object>, Flatten.Iterables<Object>>of("EmptyFlatten", Collections.<TupleTag<?>, PValue>emptyMap(), Collections.<TupleTag<?>, PValue>singletonMap(new TupleTag<Object>(), PCollection.createPrimitiveOutputInternal(p, WindowingStrategy.globalDefault(), IsBounded.BOUNDED)), Flatten.iterables(), /* This isn't actually possible to construct,
                                 * but for the sake of example */
    p);
    assertThat(PTransformMatchers.flattenWithDuplicateInputs().matches(application), is(false));
}

Example 10 with AppliedPTransform

use of org.apache.beam.sdk.runners.AppliedPTransform in project beam by apache.

the class PTransformMatchers method splittableParDoSingle.

/**
   * A {@link PTransformMatcher} that matches a {@link ParDo.SingleOutput} containing a {@link DoFn}
   * that is splittable, as signified by {@link ProcessElementMethod#isSplittable()}.
   */
public static PTransformMatcher splittableParDoSingle() {
    return new PTransformMatcher() {

        @Override
        public boolean matches(AppliedPTransform<?, ?, ?> application) {
            PTransform<?, ?> transform = application.getTransform();
            if (transform instanceof ParDo.SingleOutput) {
                DoFn<?, ?> fn = ((ParDo.SingleOutput<?, ?>) transform).getFn();
                DoFnSignature signature = DoFnSignatures.signatureForDoFn(fn);
                return signature.processElement().isSplittable();
            }
            return false;
        }

        @Override
        public String toString() {
            return MoreObjects.toStringHelper("SplittableParDoSingleMatcher").toString();
        }
    };
}