Examples with TimeWindow org.apache.flink.streaming.api.windowing.windows.TimeWindow used on opensource projects

Example 96 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class SideOutputITCase method testKeyedWindowLateArrivingEvents.

@Test
public void testKeyedWindowLateArrivingEvents() throws Exception {
    TestListResultSink<String> resultSink = new TestListResultSink<>();
    TestListResultSink<Integer> lateResultSink = new TestListResultSink<>();
    StreamExecutionEnvironment see = StreamExecutionEnvironment.getExecutionEnvironment();
    see.setParallelism(3);
    see.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
    DataStream<Integer> dataStream = see.fromCollection(elements);
    OutputTag<Integer> lateDataTag = new OutputTag<Integer>("late") {
    };
    SingleOutputStreamOperator<String> windowOperator = dataStream.assignTimestampsAndWatermarks(new TestWatermarkAssigner()).keyBy(new TestKeySelector()).timeWindow(Time.milliseconds(1), Time.milliseconds(1)).allowedLateness(Time.milliseconds(2)).sideOutputLateData(lateDataTag).apply(new WindowFunction<Integer, String, Integer, TimeWindow>() {

        private static final long serialVersionUID = 1L;

        @Override
        public void apply(Integer key, TimeWindow window, Iterable<Integer> input, Collector<String> out) throws Exception {
            for (Integer val : input) {
                out.collect(String.valueOf(key) + "-" + String.valueOf(val));
            }
        }
    });
    windowOperator.addSink(resultSink);
    windowOperator.getSideOutput(lateDataTag).addSink(lateResultSink);
    see.execute();
    assertEquals(Arrays.asList("1-1", "2-2", "4-4", "5-5"), resultSink.getSortedResult());
    assertEquals(Collections.singletonList(3), lateResultSink.getSortedResult());
}

Example 97 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class SideOutputITCase method testAllWindowLateArrivingEvents.

/**
	 * Test window late arriving events stream
	 */
@Test
public void testAllWindowLateArrivingEvents() throws Exception {
    TestListResultSink<String> sideOutputResultSink = new TestListResultSink<>();
    StreamExecutionEnvironment see = StreamExecutionEnvironment.getExecutionEnvironment();
    see.setParallelism(1);
    see.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
    DataStream<Integer> dataStream = see.fromCollection(elements);
    OutputTag<Integer> lateDataTag = new OutputTag<Integer>("late") {
    };
    SingleOutputStreamOperator<Integer> windowOperator = dataStream.assignTimestampsAndWatermarks(new TestWatermarkAssigner()).timeWindowAll(Time.milliseconds(1), Time.milliseconds(1)).sideOutputLateData(lateDataTag).apply(new AllWindowFunction<Integer, Integer, TimeWindow>() {

        private static final long serialVersionUID = 1L;

        @Override
        public void apply(TimeWindow window, Iterable<Integer> values, Collector<Integer> out) throws Exception {
            for (Integer val : values) {
                out.collect(val);
            }
        }
    });
    windowOperator.getSideOutput(lateDataTag).flatMap(new FlatMapFunction<Integer, String>() {

        private static final long serialVersionUID = 1L;

        @Override
        public void flatMap(Integer value, Collector<String> out) throws Exception {
            out.collect("late-" + String.valueOf(value));
        }
    }).addSink(sideOutputResultSink);
    see.execute();
    assertEquals(sideOutputResultSink.getSortedResult(), Arrays.asList("late-3", "late-4"));
}

Example 98 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class AbstractAlignedProcessingTimeWindowOperator method computeWindow.

private void computeWindow(long timestamp) throws Exception {
    out.setAbsoluteTimestamp(timestamp);
    panes.truncatePanes(numPanesPerWindow);
    panes.evaluateWindow(out, new TimeWindow(timestamp - windowSize, timestamp), this);
}

Example 99 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class SlidingEventTimeWindows method assignWindows.

@Override
public Collection<TimeWindow> assignWindows(Object element, long timestamp, WindowAssignerContext context) {
    if (timestamp > Long.MIN_VALUE) {
        List<TimeWindow> windows = new ArrayList<>((int) (size / slide));
        long lastStart = TimeWindow.getWindowStartWithOffset(timestamp, offset, slide);
        for (long start = lastStart; start > timestamp - size; start -= slide) {
            windows.add(new TimeWindow(start, start + size));
        }
        return windows;
    } else {
        throw new RuntimeException("Record has Long.MIN_VALUE timestamp (= no timestamp marker). " + "Is the time characteristic set to 'ProcessingTime', or did you forget to call " + "'DataStream.assignTimestampsAndWatermarks(...)'?");
    }
}

Example 100 with TimeWindow

use of org.apache.flink.streaming.api.windowing.windows.TimeWindow in project flink by apache.

the class TumblingProcessingTimeWindows method assignWindows.

@Override
public Collection<TimeWindow> assignWindows(Object element, long timestamp, WindowAssignerContext context) {
    final long now = context.getCurrentProcessingTime();
    long start = TimeWindow.getWindowStartWithOffset(now, offset, size);
    return Collections.singletonList(new TimeWindow(start, start + size));
}