Search in sources :

Example 21 with MultipleConnectedStreams

use of org.apache.flink.streaming.api.datastream.MultipleConnectedStreams in project flink-benchmarks by apache.

the class MultipleInputBenchmark method multiInputChainedIdleSource.

@Benchmark
@OperationsPerInvocation(RECORDS_PER_INVOCATION)
public void multiInputChainedIdleSource(FlinkEnvironmentContext context) throws Exception {
    final StreamExecutionEnvironment env = context.env;
    env.getConfig().enableObjectReuse();
    final DataStream<Long> source1 = env.fromSource(new NumberSequenceSource(1L, RECORDS_PER_INVOCATION), WatermarkStrategy.noWatermarks(), "source-1");
    final DataStreamSource<Integer> source2 = env.fromSource(new IdlingSource(1), WatermarkStrategy.noWatermarks(), "source-2");
    MultipleInputTransformation<Long> transform = new MultipleInputTransformation<>("custom operator", new MultiplyByTwoOperatorFactory(), BasicTypeInfo.LONG_TYPE_INFO, 1);
    transform.addInput(((DataStream<?>) source1).getTransformation());
    transform.addInput(((DataStream<?>) source2).getTransformation());
    transform.setChainingStrategy(ChainingStrategy.HEAD_WITH_SOURCES);
    env.addOperator(transform);
    new MultipleConnectedStreams(env).transform(transform).addSink(new SinkClosingIdlingSource()).setParallelism(1);
    context.execute();
}
Also used : MultiplyByTwoOperatorFactory(org.apache.flink.benchmark.operators.MultiplyByTwoOperatorFactory) MultipleConnectedStreams(org.apache.flink.streaming.api.datastream.MultipleConnectedStreams) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) NumberSequenceSource(org.apache.flink.api.connector.source.lib.NumberSequenceSource) MultipleInputTransformation(org.apache.flink.streaming.api.transformations.MultipleInputTransformation) Benchmark(org.openjdk.jmh.annotations.Benchmark) OperationsPerInvocation(org.openjdk.jmh.annotations.OperationsPerInvocation)

Aggregations

MultipleConnectedStreams (org.apache.flink.streaming.api.datastream.MultipleConnectedStreams)21 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)19 KeyedMultipleInputTransformation (org.apache.flink.streaming.api.transformations.KeyedMultipleInputTransformation)15 Test (org.junit.Test)12 MultipleInputTransformation (org.apache.flink.streaming.api.transformations.MultipleInputTransformation)9 TestListResultSink (org.apache.flink.test.streaming.runtime.util.TestListResultSink)6 KeySelector (org.apache.flink.api.java.functions.KeySelector)3 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)3 BatchExecutionStateBackend (org.apache.flink.streaming.api.operators.sorted.state.BatchExecutionStateBackend)3 UnionTransformation (org.apache.flink.streaming.api.transformations.UnionTransformation)3 MultiplyByTwoOperatorFactory (org.apache.flink.benchmark.operators.MultiplyByTwoOperatorFactory)2 NumberSequenceSource (org.apache.flink.api.connector.source.lib.NumberSequenceSource)1 Benchmark (org.openjdk.jmh.annotations.Benchmark)1 OperationsPerInvocation (org.openjdk.jmh.annotations.OperationsPerInvocation)1