Search in sources :

Example 6 with SchedulingResultPartition

use of org.apache.flink.runtime.scheduler.strategy.SchedulingResultPartition in project flink by apache.

the class DefaultExecutionGraph method createResultPartitionId.

ResultPartitionID createResultPartitionId(final IntermediateResultPartitionID resultPartitionId) {
    final SchedulingResultPartition schedulingResultPartition = getSchedulingTopology().getResultPartition(resultPartitionId);
    final SchedulingExecutionVertex producer = schedulingResultPartition.getProducer();
    final ExecutionVertexID producerId = producer.getId();
    final JobVertexID jobVertexId = producerId.getJobVertexId();
    final ExecutionJobVertex jobVertex = getJobVertex(jobVertexId);
    checkNotNull(jobVertex, "Unknown job vertex %s", jobVertexId);
    final ExecutionVertex[] taskVertices = jobVertex.getTaskVertices();
    final int subtaskIndex = producerId.getSubtaskIndex();
    checkState(subtaskIndex < taskVertices.length, "Invalid subtask index %d for job vertex %s", subtaskIndex, jobVertexId);
    final ExecutionVertex taskVertex = taskVertices[subtaskIndex];
    final Execution execution = taskVertex.getCurrentExecutionAttempt();
    return new ResultPartitionID(resultPartitionId, execution.getAttemptId());
}
Also used : SchedulingExecutionVertex(org.apache.flink.runtime.scheduler.strategy.SchedulingExecutionVertex) SchedulingResultPartition(org.apache.flink.runtime.scheduler.strategy.SchedulingResultPartition) ExecutionVertexID(org.apache.flink.runtime.scheduler.strategy.ExecutionVertexID) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) ResultPartitionID(org.apache.flink.runtime.io.network.partition.ResultPartitionID) IntermediateResultPartitionID(org.apache.flink.runtime.jobgraph.IntermediateResultPartitionID) SchedulingExecutionVertex(org.apache.flink.runtime.scheduler.strategy.SchedulingExecutionVertex)

Example 7 with SchedulingResultPartition

use of org.apache.flink.runtime.scheduler.strategy.SchedulingResultPartition in project flink by apache.

the class SchedulingPipelinedRegionComputeUtil method getNonReconnectableConsumedResults.

private static Iterable<SchedulingResultPartition> getNonReconnectableConsumedResults(SchedulingExecutionVertex vertex, Function<IntermediateResultPartitionID, ? extends SchedulingResultPartition> resultPartitionRetriever) {
    List<SchedulingResultPartition> nonReconnectableConsumedResults = new ArrayList<>();
    for (ConsumedPartitionGroup consumedPartitionGroup : vertex.getConsumedPartitionGroups()) {
        for (IntermediateResultPartitionID partitionId : consumedPartitionGroup) {
            SchedulingResultPartition consumedResult = resultPartitionRetriever.apply(partitionId);
            if (consumedResult.getResultType().isReconnectable()) {
                // The result types of partitions in one ConsumedPartitionGroup are all the same
                break;
            }
            nonReconnectableConsumedResults.add(consumedResult);
        }
    }
    return nonReconnectableConsumedResults;
}
Also used : ConsumedPartitionGroup(org.apache.flink.runtime.scheduler.strategy.ConsumedPartitionGroup) SchedulingResultPartition(org.apache.flink.runtime.scheduler.strategy.SchedulingResultPartition) ArrayList(java.util.ArrayList) IntermediateResultPartitionID(org.apache.flink.runtime.jobgraph.IntermediateResultPartitionID)

Aggregations

SchedulingResultPartition (org.apache.flink.runtime.scheduler.strategy.SchedulingResultPartition)7 IntermediateResultPartitionID (org.apache.flink.runtime.jobgraph.IntermediateResultPartitionID)4 ConsumedPartitionGroup (org.apache.flink.runtime.scheduler.strategy.ConsumedPartitionGroup)4 ExecutionVertexID (org.apache.flink.runtime.scheduler.strategy.ExecutionVertexID)4 ArrayList (java.util.ArrayList)3 ConsumerVertexGroup (org.apache.flink.runtime.scheduler.strategy.ConsumerVertexGroup)3 SchedulingExecutionVertex (org.apache.flink.runtime.scheduler.strategy.SchedulingExecutionVertex)3 HashSet (java.util.HashSet)2 List (java.util.List)2 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)2 ArrayDeque (java.util.ArrayDeque)1 Collections (java.util.Collections)1 IdentityHashMap (java.util.IdentityHashMap)1 Map (java.util.Map)1 Set (java.util.Set)1 Supplier (java.util.function.Supplier)1 ExecutionState (org.apache.flink.runtime.execution.ExecutionState)1 ResultPartitionID (org.apache.flink.runtime.io.network.partition.ResultPartitionID)1 BLOCKING (org.apache.flink.runtime.io.network.partition.ResultPartitionType.BLOCKING)1 IntermediateDataSetID (org.apache.flink.runtime.jobgraph.IntermediateDataSetID)1