Search in sources :

Example 6 with RocksDBStateBackend

use of org.apache.flink.contrib.streaming.state.RocksDBStateBackend in project flink by apache.

the class ManualWindowSpeedITCase method testTumblingIngestionTimeWindowsWithRocksDBBackendWithLateness.

@Test
public void testTumblingIngestionTimeWindowsWithRocksDBBackendWithLateness() throws Exception {
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);
    env.setParallelism(1);
    env.setStateBackend(new RocksDBStateBackend(new MemoryStateBackend()));
    env.addSource(new InfiniteTupleSource(10_000)).keyBy(0).timeWindow(Time.seconds(3)).allowedLateness(Time.seconds(1)).reduce(new ReduceFunction<Tuple2<String, Integer>>() {

        private static final long serialVersionUID = 1L;

        @Override
        public Tuple2<String, Integer> reduce(Tuple2<String, Integer> value1, Tuple2<String, Integer> value2) throws Exception {
            return Tuple2.of(value1.f0, value1.f1 + value2.f1);
        }
    }).filter(new FilterFunction<Tuple2<String, Integer>>() {

        private static final long serialVersionUID = 1L;

        @Override
        public boolean filter(Tuple2<String, Integer> value) throws Exception {
            return value.f0.startsWith("Tuple 0");
        }
    }).print();
    env.execute();
}
Also used : FilterFunction(org.apache.flink.api.common.functions.FilterFunction) RocksDBStateBackend(org.apache.flink.contrib.streaming.state.RocksDBStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) Tuple2(org.apache.flink.api.java.tuple.Tuple2) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment) Test(org.junit.Test)

Aggregations

RocksDBStateBackend (org.apache.flink.contrib.streaming.state.RocksDBStateBackend)6 MemoryStateBackend (org.apache.flink.runtime.state.memory.MemoryStateBackend)6 Test (org.junit.Test)6 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)5 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)5 FilterFunction (org.apache.flink.api.common.functions.FilterFunction)3 TypeHint (org.apache.flink.api.common.typeinfo.TypeHint)2 HashMap (java.util.HashMap)1 Map (java.util.Map)1 Event (org.apache.flink.cep.Event)1 SubEvent (org.apache.flink.cep.SubEvent)1 Watermark (org.apache.flink.streaming.api.watermark.Watermark)1 OperatorStateHandles (org.apache.flink.streaming.runtime.tasks.OperatorStateHandles)1 Ignore (org.junit.Ignore)1