Search in sources :

Example 6 with KeyedMultipleInputTransformation

use of org.apache.flink.streaming.api.transformations.KeyedMultipleInputTransformation in project flink by apache.

the class MultipleInputITCase method testKeyedState.

@Test
public void testKeyedState() throws Exception {
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    env.setParallelism(1);
    TestListResultSink<Long> resultSink = new TestListResultSink<>();
    DataStream<Long> source1 = env.fromElements(0L, 3L);
    DataStream<Long> source2 = env.fromElements(13L, 16L);
    DataStream<Long> source3 = env.fromElements(101L, 104L);
    KeyedMultipleInputTransformation<Long> transform = new KeyedMultipleInputTransformation<>("My Operator", new KeyedSumMultipleInputOperatorFactory(), BasicTypeInfo.LONG_TYPE_INFO, 1, BasicTypeInfo.LONG_TYPE_INFO);
    KeySelector<Long, Long> keySelector = (KeySelector<Long, Long>) value -> value % 3;
    env.addOperator(transform.addInput(source1.getTransformation(), keySelector).addInput(source2.getTransformation(), keySelector).addInput(source3.getTransformation(), keySelector));
    new MultipleConnectedStreams(env).transform(transform).addSink(resultSink);
    env.execute();
    List<Long> result = resultSink.getResult();
    Collections.sort(result);
    assertThat(result, contains(0L, 3L, 13L, 13L + 16L, 101L, 101L + 104L));
}
Also used : TestListResultSink(org.apache.flink.test.streaming.runtime.util.TestListResultSink) KeyedMultipleInputTransformation(org.apache.flink.streaming.api.transformations.KeyedMultipleInputTransformation) MultipleConnectedStreams(org.apache.flink.streaming.api.datastream.MultipleConnectedStreams) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) KeySelector(org.apache.flink.api.java.functions.KeySelector) Test(org.junit.Test)

Aggregations

KeyedMultipleInputTransformation (org.apache.flink.streaming.api.transformations.KeyedMultipleInputTransformation)6 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)5 Test (org.junit.Test)5 MultipleConnectedStreams (org.apache.flink.streaming.api.datastream.MultipleConnectedStreams)4 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)2 Arrays (java.util.Arrays)1 HashSet (java.util.HashSet)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Objects (java.util.Objects)1 Optional (java.util.Optional)1 Random (java.util.Random)1 Set (java.util.Set)1 Consumer (java.util.function.Consumer)1 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)1 RuntimeExecutionMode (org.apache.flink.api.common.RuntimeExecutionMode)1 Watermark (org.apache.flink.api.common.eventtime.Watermark)1 WatermarkGenerator (org.apache.flink.api.common.eventtime.WatermarkGenerator)1 WatermarkOutput (org.apache.flink.api.common.eventtime.WatermarkOutput)1 WatermarkStrategy (org.apache.flink.api.common.eventtime.WatermarkStrategy)1