Examples with SerializedInputSplit org.apache.flink.runtime.jobmaster.SerializedInputSplit used on opensource projects

Example 1 with SerializedInputSplit

use of org.apache.flink.runtime.jobmaster.SerializedInputSplit in project flink by apache.

the class RpcInputSplitProvider method getNextInputSplit.

@Override
public InputSplit getNextInputSplit(ClassLoader userCodeClassLoader) throws InputSplitProviderException {
    Preconditions.checkNotNull(userCodeClassLoader);
    CompletableFuture<SerializedInputSplit> futureInputSplit = jobMasterGateway.requestNextInputSplit(jobVertexID, executionAttemptID);
    try {
        SerializedInputSplit serializedInputSplit = futureInputSplit.get(timeout.getSize(), timeout.getUnit());
        if (serializedInputSplit.isEmpty()) {
            return null;
        } else {
            return InstantiationUtil.deserializeObject(serializedInputSplit.getInputSplitData(), userCodeClassLoader);
        }
    } catch (Exception e) {
        throw new InputSplitProviderException("Requesting the next input split failed.", e);
    }
}

Example 2 with SerializedInputSplit

use of org.apache.flink.runtime.jobmaster.SerializedInputSplit in project flink by apache.

the class ExecutionGraphHandler method requestNextInputSplit.

public SerializedInputSplit requestNextInputSplit(JobVertexID vertexID, ExecutionAttemptID executionAttempt) throws IOException {
    final Execution execution = executionGraph.getRegisteredExecutions().get(executionAttempt);
    if (execution == null) {
        // but TaskManager get some delay to aware of that situation
        if (log.isDebugEnabled()) {
            log.debug("Can not find Execution for attempt {}.", executionAttempt);
        }
        // but we should TaskManager be aware of this
        throw new IllegalArgumentException("Can not find Execution for attempt " + executionAttempt);
    }
    final ExecutionJobVertex vertex = executionGraph.getJobVertex(vertexID);
    if (vertex == null) {
        throw new IllegalArgumentException("Cannot find execution vertex for vertex ID " + vertexID);
    }
    if (vertex.getSplitAssigner() == null) {
        throw new IllegalStateException("No InputSplitAssigner for vertex ID " + vertexID);
    }
    final InputSplit nextInputSplit = execution.getNextInputSplit();
    if (nextInputSplit != null) {
        log.debug("Send next input split {}.", nextInputSplit);
    } else {
        log.debug("No more input splits available");
    }
    try {
        final byte[] serializedInputSplit = InstantiationUtil.serializeObject(nextInputSplit);
        return new SerializedInputSplit(serializedInputSplit);
    } catch (Exception ex) {
        IOException reason = new IOException("Could not serialize the next input split of class " + nextInputSplit.getClass() + ".", ex);
        vertex.fail(reason);
        throw reason;
    }
}