Examples with TimeCharacteristic org.apache.flink.streaming.api.TimeCharacteristic used on opensource projects

Example 1 with TimeCharacteristic

use of org.apache.flink.streaming.api.TimeCharacteristic in project flink by apache.

the class ContinuousFileReaderOperator method open.

@Override
public void open() throws Exception {
    super.open();
    checkState(this.reader == null, "The reader is already initialized.");
    checkState(this.serializer != null, "The serializer has not been set. " + "Probably the setOutputType() was not called. Please report it.");
    this.format.setRuntimeContext(getRuntimeContext());
    this.format.configure(new Configuration());
    this.checkpointLock = getContainingTask().getCheckpointLock();
    // set the reader context based on the time characteristic
    final TimeCharacteristic timeCharacteristic = getOperatorConfig().getTimeCharacteristic();
    final long watermarkInterval = getRuntimeContext().getExecutionConfig().getAutoWatermarkInterval();
    this.readerContext = StreamSourceContexts.getSourceContext(timeCharacteristic, getProcessingTimeService(), checkpointLock, getContainingTask().getStreamStatusMaintainer(), output, watermarkInterval, -1);
    // and initialize the split reading thread
    this.reader = new SplitReader<>(format, serializer, readerContext, checkpointLock, restoredReaderState);
    this.restoredReaderState = null;
    this.reader.start();
}

Example 2 with TimeCharacteristic

use of org.apache.flink.streaming.api.TimeCharacteristic in project flink by apache.

the class StreamSource method run.

public void run(final Object lockingObject, final StreamStatusMaintainer streamStatusMaintainer, final Output<StreamRecord<OUT>> collector) throws Exception {
    final TimeCharacteristic timeCharacteristic = getOperatorConfig().getTimeCharacteristic();
    LatencyMarksEmitter latencyEmitter = null;
    if (getExecutionConfig().isLatencyTrackingEnabled()) {
        latencyEmitter = new LatencyMarksEmitter<>(getProcessingTimeService(), collector, getExecutionConfig().getLatencyTrackingInterval(), getOperatorConfig().getVertexID(), getRuntimeContext().getIndexOfThisSubtask());
    }
    final long watermarkInterval = getRuntimeContext().getExecutionConfig().getAutoWatermarkInterval();
    this.ctx = StreamSourceContexts.getSourceContext(timeCharacteristic, getProcessingTimeService(), lockingObject, streamStatusMaintainer, collector, watermarkInterval, -1);
    try {
        userFunction.run(ctx);
        // a final watermark that indicates that we reached the end of event-time
        if (!isCanceledOrStopped()) {
            ctx.emitWatermark(Watermark.MAX_WATERMARK);
        }
    } finally {
        // make sure that the context is closed in any case
        ctx.close();
        if (latencyEmitter != null) {
            latencyEmitter.close();
        }
    }
}