Search in sources :

Example 36 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class InternalWindowFunctionTest method testInternalAggregateProcessWindowFunction.

@SuppressWarnings("unchecked")
@Test
public void testInternalAggregateProcessWindowFunction() throws Exception {
    AggregateProcessWindowFunctionMock mock = mock(AggregateProcessWindowFunctionMock.class);
    InternalAggregateProcessWindowFunction<Long, Set<Long>, Map<Long, Long>, String, Long, TimeWindow> windowFunction = new InternalAggregateProcessWindowFunction<>(new AggregateFunction<Long, Set<Long>, Map<Long, Long>>() {

        private static final long serialVersionUID = 1L;

        @Override
        public Set<Long> createAccumulator() {
            return new HashSet<>();
        }

        @Override
        public void add(Long value, Set<Long> accumulator) {
            accumulator.add(value);
        }

        @Override
        public Map<Long, Long> getResult(Set<Long> accumulator) {
            Map<Long, Long> result = new HashMap<>();
            for (Long in : accumulator) {
                result.put(in, in);
            }
            return result;
        }

        @Override
        public Set<Long> merge(Set<Long> a, Set<Long> b) {
            a.addAll(b);
            return a;
        }
    }, mock);
    // check setOutputType
    TypeInformation<String> stringType = BasicTypeInfo.STRING_TYPE_INFO;
    ExecutionConfig execConf = new ExecutionConfig();
    execConf.setParallelism(42);
    StreamingFunctionUtils.setOutputType(windowFunction, stringType, execConf);
    verify(mock).setOutputType(stringType, execConf);
    // check open
    Configuration config = new Configuration();
    windowFunction.open(config);
    verify(mock).open(config);
    // check setRuntimeContext
    RuntimeContext rCtx = mock(RuntimeContext.class);
    windowFunction.setRuntimeContext(rCtx);
    verify(mock).setRuntimeContext(rCtx);
    // check apply
    TimeWindow w = mock(TimeWindow.class);
    Collector<String> c = (Collector<String>) mock(Collector.class);
    List<Long> args = new LinkedList<>();
    args.add(23L);
    args.add(24L);
    windowFunction.apply(42L, w, args, c);
    verify(mock).process(eq(42L), (AggregateProcessWindowFunctionMock.Context) anyObject(), (Iterable) argThat(containsInAnyOrder(allOf(hasEntry(is(23L), is(23L)), hasEntry(is(24L), is(24L))))), eq(c));
    // check close
    windowFunction.close();
    verify(mock).close();
}
Also used : HashSet(java.util.HashSet) Set(java.util.Set) Configuration(org.apache.flink.configuration.Configuration) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) InternalAggregateProcessWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalAggregateProcessWindowFunction) TimeWindow(org.apache.flink.streaming.api.windowing.windows.TimeWindow) LinkedList(java.util.LinkedList) Collector(org.apache.flink.util.Collector) RuntimeContext(org.apache.flink.api.common.functions.RuntimeContext) HashMap(java.util.HashMap) Map(java.util.Map) Test(org.junit.Test)

Example 37 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class InternalWindowFunctionTest method testInternalIterableProcessWindowFunction.

@SuppressWarnings("unchecked")
@Test
public void testInternalIterableProcessWindowFunction() throws Exception {
    ProcessWindowFunctionMock mock = mock(ProcessWindowFunctionMock.class);
    InternalIterableProcessWindowFunction<Long, String, Long, TimeWindow> windowFunction = new InternalIterableProcessWindowFunction<>(mock);
    // check setOutputType
    TypeInformation<String> stringType = BasicTypeInfo.STRING_TYPE_INFO;
    ExecutionConfig execConf = new ExecutionConfig();
    execConf.setParallelism(42);
    StreamingFunctionUtils.setOutputType(windowFunction, stringType, execConf);
    verify(mock).setOutputType(stringType, execConf);
    // check open
    Configuration config = new Configuration();
    windowFunction.open(config);
    verify(mock).open(config);
    // check setRuntimeContext
    RuntimeContext rCtx = mock(RuntimeContext.class);
    windowFunction.setRuntimeContext(rCtx);
    verify(mock).setRuntimeContext(rCtx);
    // check apply
    TimeWindow w = mock(TimeWindow.class);
    Iterable<Long> i = (Iterable<Long>) mock(Iterable.class);
    Collector<String> c = (Collector<String>) mock(Collector.class);
    windowFunction.apply(42L, w, i, c);
    verify(mock).process(eq(42L), (ProcessWindowFunctionMock.Context) anyObject(), eq(i), eq(c));
    // check close
    windowFunction.close();
    verify(mock).close();
}
Also used : Configuration(org.apache.flink.configuration.Configuration) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) TimeWindow(org.apache.flink.streaming.api.windowing.windows.TimeWindow) Collector(org.apache.flink.util.Collector) InternalIterableProcessWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableProcessWindowFunction) RuntimeContext(org.apache.flink.api.common.functions.RuntimeContext) Test(org.junit.Test)

Example 38 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class InternalWindowFunctionTest method testInternalIterableProcessAllWindowFunction.

@SuppressWarnings("unchecked")
@Test
public void testInternalIterableProcessAllWindowFunction() throws Exception {
    ProcessAllWindowFunctionMock mock = mock(ProcessAllWindowFunctionMock.class);
    InternalIterableProcessAllWindowFunction<Long, String, TimeWindow> windowFunction = new InternalIterableProcessAllWindowFunction<>(mock);
    // check setOutputType
    TypeInformation<String> stringType = BasicTypeInfo.STRING_TYPE_INFO;
    ExecutionConfig execConf = new ExecutionConfig();
    execConf.setParallelism(42);
    StreamingFunctionUtils.setOutputType(windowFunction, stringType, execConf);
    verify(mock).setOutputType(stringType, execConf);
    // check open
    Configuration config = new Configuration();
    windowFunction.open(config);
    verify(mock).open(config);
    // check setRuntimeContext
    RuntimeContext rCtx = mock(RuntimeContext.class);
    windowFunction.setRuntimeContext(rCtx);
    verify(mock).setRuntimeContext(rCtx);
    // check apply
    TimeWindow w = mock(TimeWindow.class);
    Iterable<Long> i = (Iterable<Long>) mock(Iterable.class);
    Collector<String> c = (Collector<String>) mock(Collector.class);
    windowFunction.apply(((byte) 0), w, i, c);
    verify(mock).process((ProcessAllWindowFunctionMock.Context) anyObject(), eq(i), eq(c));
    // check close
    windowFunction.close();
    verify(mock).close();
}
Also used : Configuration(org.apache.flink.configuration.Configuration) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) TimeWindow(org.apache.flink.streaming.api.windowing.windows.TimeWindow) Collector(org.apache.flink.util.Collector) InternalIterableProcessAllWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableProcessAllWindowFunction) RuntimeContext(org.apache.flink.api.common.functions.RuntimeContext) Test(org.junit.Test)

Example 39 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class InternalWindowFunctionTest method testInternalSingleValueAllWindowFunction.

@SuppressWarnings("unchecked")
@Test
public void testInternalSingleValueAllWindowFunction() throws Exception {
    AllWindowFunctionMock mock = mock(AllWindowFunctionMock.class);
    InternalSingleValueAllWindowFunction<Long, String, TimeWindow> windowFunction = new InternalSingleValueAllWindowFunction<>(mock);
    // check setOutputType
    TypeInformation<String> stringType = BasicTypeInfo.STRING_TYPE_INFO;
    ExecutionConfig execConf = new ExecutionConfig();
    execConf.setParallelism(42);
    StreamingFunctionUtils.setOutputType(windowFunction, stringType, execConf);
    verify(mock).setOutputType(stringType, execConf);
    // check open
    Configuration config = new Configuration();
    windowFunction.open(config);
    verify(mock).open(config);
    // check setRuntimeContext
    RuntimeContext rCtx = mock(RuntimeContext.class);
    windowFunction.setRuntimeContext(rCtx);
    verify(mock).setRuntimeContext(rCtx);
    // check apply
    TimeWindow w = mock(TimeWindow.class);
    Collector<String> c = (Collector<String>) mock(Collector.class);
    windowFunction.apply(((byte) 0), w, 23L, c);
    verify(mock).apply(eq(w), (Iterable<Long>) argThat(IsIterableContainingInOrder.contains(23L)), eq(c));
    // check close
    windowFunction.close();
    verify(mock).close();
}
Also used : InternalSingleValueAllWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueAllWindowFunction) Configuration(org.apache.flink.configuration.Configuration) Collector(org.apache.flink.util.Collector) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) RuntimeContext(org.apache.flink.api.common.functions.RuntimeContext) TimeWindow(org.apache.flink.streaming.api.windowing.windows.TimeWindow) Test(org.junit.Test)

Example 40 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class InternalWindowFunctionTest method testInternalSingleValueProcessAllWindowFunction.

@SuppressWarnings("unchecked")
@Test
public void testInternalSingleValueProcessAllWindowFunction() throws Exception {
    ProcessAllWindowFunctionMock mock = mock(ProcessAllWindowFunctionMock.class);
    InternalSingleValueProcessAllWindowFunction<Long, String, TimeWindow> windowFunction = new InternalSingleValueProcessAllWindowFunction<>(mock);
    // check setOutputType
    TypeInformation<String> stringType = BasicTypeInfo.STRING_TYPE_INFO;
    ExecutionConfig execConf = new ExecutionConfig();
    execConf.setParallelism(42);
    StreamingFunctionUtils.setOutputType(windowFunction, stringType, execConf);
    verify(mock).setOutputType(stringType, execConf);
    // check open
    Configuration config = new Configuration();
    windowFunction.open(config);
    verify(mock).open(config);
    // check setRuntimeContext
    RuntimeContext rCtx = mock(RuntimeContext.class);
    windowFunction.setRuntimeContext(rCtx);
    verify(mock).setRuntimeContext(rCtx);
    // check apply
    TimeWindow w = mock(TimeWindow.class);
    Collector<String> c = (Collector<String>) mock(Collector.class);
    windowFunction.apply(((byte) 0), w, 23L, c);
    verify(mock).process((ProcessAllWindowFunctionMock.Context) anyObject(), (Iterable<Long>) argThat(IsIterableContainingInOrder.contains(23L)), eq(c));
    // check close
    windowFunction.close();
    verify(mock).close();
}
Also used : Configuration(org.apache.flink.configuration.Configuration) Collector(org.apache.flink.util.Collector) InternalSingleValueProcessAllWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueProcessAllWindowFunction) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) RuntimeContext(org.apache.flink.api.common.functions.RuntimeContext) TimeWindow(org.apache.flink.streaming.api.windowing.windows.TimeWindow) Test(org.junit.Test)

Aggregations

TimeWindow (org.apache.flink.streaming.api.windowing.windows.TimeWindow)158 Test (org.junit.Test)147 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)99 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)75 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)48 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)43 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)38 ListStateDescriptor (org.apache.flink.api.common.state.ListStateDescriptor)37 Watermark (org.apache.flink.streaming.api.watermark.Watermark)29 ConcurrentLinkedQueue (java.util.concurrent.ConcurrentLinkedQueue)28 TumblingEventTimeWindows (org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows)28 KeyedOneInputStreamOperatorTestHarness (org.apache.flink.streaming.util.KeyedOneInputStreamOperatorTestHarness)28 ReducingStateDescriptor (org.apache.flink.api.common.state.ReducingStateDescriptor)26 EventTimeTrigger (org.apache.flink.streaming.api.windowing.triggers.EventTimeTrigger)25 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)22 Configuration (org.apache.flink.configuration.Configuration)22 StreamRecord (org.apache.flink.streaming.runtime.streamrecord.StreamRecord)19 Tuple (org.apache.flink.api.java.tuple.Tuple)13 FoldingStateDescriptor (org.apache.flink.api.common.state.FoldingStateDescriptor)12 TumblingProcessingTimeWindows (org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows)12