Examples with BootstrapTransformationWithID org.apache.flink.state.api.runtime.BootstrapTransformationWithID used on opensource projects

Example 1 with BootstrapTransformationWithID

use of org.apache.flink.state.api.runtime.BootstrapTransformationWithID in project flink by apache.

the class WritableSavepoint method write.

/**
 * Write out a new or updated savepoint.
 *
 * @param path The path to where the savepoint should be written.
 */
public final void write(String path) {
    final Path savepointPath = new Path(path);
    List<BootstrapTransformationWithID<?>> newOperatorTransformations = metadata.getNewOperators();
    DataSet<OperatorState> newOperatorStates = writeOperatorStates(newOperatorTransformations, configuration, savepointPath);
    List<OperatorState> existingOperators = metadata.getExistingOperators();
    DataSet<OperatorState> finalOperatorStates;
    if (existingOperators.isEmpty()) {
        finalOperatorStates = newOperatorStates;
    } else {
        DataSet<OperatorState> existingOperatorStates = newOperatorStates.getExecutionEnvironment().fromCollection(existingOperators).name("existingOperatorStates");
        existingOperatorStates.flatMap(new StatePathExtractor()).setParallelism(1).output(new FileCopyFunction(path));
        finalOperatorStates = newOperatorStates.union(existingOperatorStates);
    }
    finalOperatorStates.reduceGroup(new MergeOperatorStates(metadata.getMasterStates())).name("reduce(OperatorState)").output(new SavepointOutputFormat(savepointPath)).name(path);
}

Example 2 with BootstrapTransformationWithID

use of org.apache.flink.state.api.runtime.BootstrapTransformationWithID in project flink by apache.

the class SavepointMetadata method addOperator.

public void addOperator(String uid, BootstrapTransformation<?> transformation) {
    OperatorID id = OperatorIDGenerator.fromUid(uid);
    if (operatorStateIndex.containsKey(id)) {
        throw new IllegalArgumentException("The savepoint already contains uid " + uid + ". All uid's must be unique");
    }
    operatorStateIndex.put(id, OperatorStateSpec.newWithTransformation(new BootstrapTransformationWithID<>(id, transformation)));
}