Search in sources :

Example 6 with ReplacementOutput

use of org.apache.beam.sdk.runners.PTransformOverrideFactory.ReplacementOutput in project beam by apache.

the class ReplacementOutputsTest method taggedSucceeds.

@Test
public void taggedSucceeds() {
    PCollectionTuple original = PCollectionTuple.of(intsTag, ints).and(strsTag, strs).and(moreIntsTag, moreInts);
    Map<PCollection<?>, ReplacementOutput> replacements = ReplacementOutputs.tagged(PValues.expandOutput((POutput) original), PCollectionTuple.of(strsTag, replacementStrs).and(moreIntsTag, moreReplacementInts).and(intsTag, replacementInts));
    assertThat(replacements.keySet(), Matchers.containsInAnyOrder(replacementStrs, replacementInts, moreReplacementInts));
    ReplacementOutput intsReplacement = replacements.get(replacementInts);
    ReplacementOutput strsReplacement = replacements.get(replacementStrs);
    ReplacementOutput moreIntsReplacement = replacements.get(moreReplacementInts);
    assertThat(intsReplacement, equalTo(ReplacementOutput.of(TaggedPValue.of(intsTag, ints), TaggedPValue.of(intsTag, replacementInts))));
    assertThat(strsReplacement, equalTo(ReplacementOutput.of(TaggedPValue.of(strsTag, strs), TaggedPValue.of(strsTag, replacementStrs))));
    assertThat(moreIntsReplacement, equalTo(ReplacementOutput.of(TaggedPValue.of(moreIntsTag, moreInts), TaggedPValue.of(moreIntsTag, moreReplacementInts))));
}
Also used : PCollection(org.apache.beam.sdk.values.PCollection) ReplacementOutput(org.apache.beam.sdk.runners.PTransformOverrideFactory.ReplacementOutput) PCollectionTuple(org.apache.beam.sdk.values.PCollectionTuple) POutput(org.apache.beam.sdk.values.POutput) Test(org.junit.Test)

Aggregations

ReplacementOutput (org.apache.beam.sdk.runners.PTransformOverrideFactory.ReplacementOutput)6 PCollection (org.apache.beam.sdk.values.PCollection)5 TaggedPValue (org.apache.beam.sdk.values.TaggedPValue)4 Map (java.util.Map)3 TupleTag (org.apache.beam.sdk.values.TupleTag)3 Test (org.junit.Test)3 HashMap (java.util.HashMap)2 PValue (org.apache.beam.sdk.values.PValue)2 ImmutableMap (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMap)2 StorageObject (com.google.api.services.storage.model.StorageObject)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 HashSet (java.util.HashSet)1 StreamingShardedWriteFactory (org.apache.beam.runners.dataflow.DataflowRunner.StreamingShardedWriteFactory)1 Read (org.apache.beam.sdk.io.Read)1 WriteFiles (org.apache.beam.sdk.io.WriteFiles)1 WriteFilesResult (org.apache.beam.sdk.io.WriteFilesResult)1 Node (org.apache.beam.sdk.runners.TransformHierarchy.Node)1 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)1 Create (org.apache.beam.sdk.transforms.Create)1 PTransform (org.apache.beam.sdk.transforms.PTransform)1