Search in sources :

Example 1 with SavepointType

use of org.apache.flink.runtime.checkpoint.SavepointType in project flink by apache.

the class MultipleInputStreamTask method triggerStopWithSavepointAsync.

private CompletableFuture<Boolean> triggerStopWithSavepointAsync(CheckpointMetaData checkpointMetaData, CheckpointOptions checkpointOptions) {
    CompletableFuture<Void> sourcesStopped = new CompletableFuture<>();
    final StopMode stopMode = ((SavepointType) checkpointOptions.getCheckpointType()).shouldDrain() ? StopMode.DRAIN : StopMode.NO_DRAIN;
    mainMailboxExecutor.execute(() -> {
        setSynchronousSavepoint(checkpointMetaData.getCheckpointId());
        FutureUtils.forward(FutureUtils.waitForAll(operatorChain.getSourceTaskInputs().stream().map(s -> s.getOperator().stop(stopMode)).collect(Collectors.toList())), sourcesStopped);
    }, "stop chained Flip-27 source for stop-with-savepoint --drain");
    return sourcesStopped.thenCompose(ignore -> triggerSourcesCheckpointAsync(checkpointMetaData, checkpointOptions));
}
Also used : StreamConfig(org.apache.flink.streaming.api.graph.StreamConfig) CheckpointMetricsBuilder(org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder) CheckpointMetaData(org.apache.flink.runtime.checkpoint.CheckpointMetaData) CheckpointedInputGate(org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate) SavepointType(org.apache.flink.runtime.checkpoint.SavepointType) Watermark(org.apache.flink.streaming.api.watermark.Watermark) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Function(java.util.function.Function) ArrayList(java.util.ArrayList) CheckpointException(org.apache.flink.runtime.checkpoint.CheckpointException) StreamPartitioner(org.apache.flink.streaming.runtime.partitioner.StreamPartitioner) StreamRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) Output(org.apache.flink.streaming.api.operators.Output) StreamTaskSourceInput(org.apache.flink.streaming.runtime.io.StreamTaskSourceInput) InputConfig(org.apache.flink.streaming.api.graph.StreamConfig.InputConfig) InputChannelInfo(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo) Nullable(javax.annotation.Nullable) StreamEdge(org.apache.flink.streaming.api.graph.StreamEdge) StreamMultipleInputProcessorFactory(org.apache.flink.streaming.runtime.io.StreamMultipleInputProcessorFactory) IOException(java.io.IOException) CheckpointOptions(org.apache.flink.runtime.checkpoint.CheckpointOptions) CheckpointBarrierHandler(org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler) InputProcessorUtil(org.apache.flink.streaming.runtime.io.checkpointing.InputProcessorUtil) Collectors(java.util.stream.Collectors) StopMode(org.apache.flink.runtime.io.network.api.StopMode) MetricNames(org.apache.flink.runtime.metrics.MetricNames) MultipleInputStreamOperator(org.apache.flink.streaming.api.operators.MultipleInputStreamOperator) List(java.util.List) SnapshotType(org.apache.flink.runtime.checkpoint.SnapshotType) CheckpointBarrier(org.apache.flink.runtime.io.network.api.CheckpointBarrier) MinWatermarkGauge(org.apache.flink.streaming.runtime.metrics.MinWatermarkGauge) Optional(java.util.Optional) Internal(org.apache.flink.annotation.Internal) IndexedInputGate(org.apache.flink.runtime.io.network.partition.consumer.IndexedInputGate) Environment(org.apache.flink.runtime.execution.Environment) WatermarkGauge(org.apache.flink.streaming.runtime.metrics.WatermarkGauge) CompletableFuture(java.util.concurrent.CompletableFuture) StopMode(org.apache.flink.runtime.io.network.api.StopMode)

Aggregations

IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Optional (java.util.Optional)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 Function (java.util.function.Function)1 Collectors (java.util.stream.Collectors)1 Nullable (javax.annotation.Nullable)1 Internal (org.apache.flink.annotation.Internal)1 CheckpointException (org.apache.flink.runtime.checkpoint.CheckpointException)1 CheckpointMetaData (org.apache.flink.runtime.checkpoint.CheckpointMetaData)1 CheckpointMetricsBuilder (org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder)1 CheckpointOptions (org.apache.flink.runtime.checkpoint.CheckpointOptions)1 SavepointType (org.apache.flink.runtime.checkpoint.SavepointType)1 SnapshotType (org.apache.flink.runtime.checkpoint.SnapshotType)1 InputChannelInfo (org.apache.flink.runtime.checkpoint.channel.InputChannelInfo)1 Environment (org.apache.flink.runtime.execution.Environment)1 CheckpointBarrier (org.apache.flink.runtime.io.network.api.CheckpointBarrier)1 StopMode (org.apache.flink.runtime.io.network.api.StopMode)1