Search in sources :

Example 1 with ImmutableMultimap

use of org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMultimap in project beam by apache.

the class BatchSideInputHandlerFactory method forIterableSideInput.

@Override
public <V, W extends BoundedWindow> IterableSideInputHandler<V, W> forIterableSideInput(String transformId, String sideInputId, Coder<V> elementCoder, Coder<W> windowCoder) {
    PCollectionNode collectionNode = sideInputToCollection.get(SideInputId.newBuilder().setTransformId(transformId).setLocalName(sideInputId).build());
    checkArgument(collectionNode != null, "No side input for %s/%s", transformId, sideInputId);
    ImmutableMultimap.Builder<Object, V> windowToValuesBuilder = ImmutableMultimap.builder();
    List<WindowedValue<V>> broadcastVariable = sideInputGetter.getSideInput(collectionNode.getId());
    for (WindowedValue<V> windowedValue : broadcastVariable) {
        for (BoundedWindow boundedWindow : windowedValue.getWindows()) {
            @SuppressWarnings("unchecked") W window = (W) boundedWindow;
            windowToValuesBuilder.put(windowCoder.structuralValue(window), windowedValue.getValue());
        }
    }
    ImmutableMultimap<Object, V> windowToValues = windowToValuesBuilder.build();
    return new IterableSideInputHandler<V, W>() {

        @Override
        public Iterable<V> get(W window) {
            return windowToValues.get(windowCoder.structuralValue(window));
        }

        @Override
        public Coder<V> elementCoder() {
            return elementCoder;
        }
    };
}
Also used : PCollectionNode(org.apache.beam.runners.core.construction.graph.PipelineNode.PCollectionNode) IterableSideInputHandler(org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler) KV(org.apache.beam.sdk.values.KV) WindowedValue(org.apache.beam.sdk.util.WindowedValue) BoundedWindow(org.apache.beam.sdk.transforms.windowing.BoundedWindow) ImmutableMultimap(org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMultimap)

Aggregations

PCollectionNode (org.apache.beam.runners.core.construction.graph.PipelineNode.PCollectionNode)1 IterableSideInputHandler (org.apache.beam.runners.fnexecution.state.StateRequestHandlers.IterableSideInputHandler)1 BoundedWindow (org.apache.beam.sdk.transforms.windowing.BoundedWindow)1 WindowedValue (org.apache.beam.sdk.util.WindowedValue)1 KV (org.apache.beam.sdk.values.KV)1 ImmutableMultimap (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMultimap)1