Examples with PValue org.apache.beam.sdk.values.PValue used on opensource projects

Example 26 with PValue

use of org.apache.beam.sdk.values.PValue in project beam by apache.

the class DirectGraphVisitor method visitPrimitiveTransform.

@Override
public void visitPrimitiveTransform(TransformHierarchy.Node node) {
    AppliedPTransform<?, ?, ?> appliedTransform = getAppliedTransform(node);
    stepNames.put(appliedTransform, genStepName());
    if (node.getInputs().isEmpty()) {
        rootTransforms.add(appliedTransform);
    } else {
        for (PValue value : node.getInputs().values()) {
            primitiveConsumers.put(value, appliedTransform);
        }
        if (node.getTransform() instanceof ViewOverrideFactory.WriteView) {
            viewWriters.put(((ViewOverrideFactory.WriteView<?, ?>) node.getTransform()).getView(), node.toAppliedPTransform(getPipeline()));
        }
    }
}

Example 27 with PValue

use of org.apache.beam.sdk.values.PValue in project beam by apache.

the class WatermarkManager method getInputProcessingWatermarks.

private Collection<Watermark> getInputProcessingWatermarks(AppliedPTransform<?, ?, ?> transform) {
    ImmutableList.Builder<Watermark> inputWmsBuilder = ImmutableList.builder();
    Map<TupleTag<?>, PValue> inputs = transform.getInputs();
    if (inputs.isEmpty()) {
        inputWmsBuilder.add(THE_END_OF_TIME);
    }
    for (PValue pvalue : inputs.values()) {
        Watermark producerOutputWatermark = getValueWatermark(pvalue).synchronizedProcessingOutputWatermark;
        inputWmsBuilder.add(producerOutputWatermark);
    }
    return inputWmsBuilder.build();
}

Example 28 with PValue

use of org.apache.beam.sdk.values.PValue in project beam by apache.

the class WatermarkManager method getInputWatermarks.

private List<Watermark> getInputWatermarks(AppliedPTransform<?, ?, ?> transform) {
    ImmutableList.Builder<Watermark> inputWatermarksBuilder = ImmutableList.builder();
    Map<TupleTag<?>, PValue> inputs = transform.getInputs();
    if (inputs.isEmpty()) {
        inputWatermarksBuilder.add(THE_END_OF_TIME);
    }
    for (PValue pvalue : inputs.values()) {
        Watermark producerOutputWatermark = getValueWatermark(pvalue).outputWatermark;
        inputWatermarksBuilder.add(producerOutputWatermark);
    }
    List<Watermark> inputCollectionWatermarks = inputWatermarksBuilder.build();
    return inputCollectionWatermarks;
}