Examples with CheckpointedResultSubpartition org.apache.flink.runtime.io.network.partition.CheckpointedResultSubpartition used on opensource projects

Example 1 with CheckpointedResultSubpartition

use of org.apache.flink.runtime.io.network.partition.CheckpointedResultSubpartition in project flink by apache.

the class ResultSubpartitionRecoveredStateHandler method recover.

@Override
public void recover(ResultSubpartitionInfo subpartitionInfo, int oldSubtaskIndex, BufferWithContext<BufferBuilder> bufferWithContext) throws IOException {
    try (BufferBuilder bufferBuilder = bufferWithContext.context) {
        try (BufferConsumer bufferConsumer = bufferBuilder.createBufferConsumerFromBeginning()) {
            bufferBuilder.finish();
            if (bufferConsumer.isDataAvailable()) {
                final List<CheckpointedResultSubpartition> channels = getMappedChannels(subpartitionInfo);
                for (final CheckpointedResultSubpartition channel : channels) {
                    // channel selector is created from the downstream's point of view: the
                    // subtask of downstream = subpartition index of recovered buffer
                    final SubtaskConnectionDescriptor channelSelector = new SubtaskConnectionDescriptor(subpartitionInfo.getSubPartitionIdx(), oldSubtaskIndex);
                    channel.addRecovered(EventSerializer.toBufferConsumer(channelSelector, false));
                    channel.addRecovered(bufferConsumer.copy());
                }
            }
        }
    }
}

Example 2 with CheckpointedResultSubpartition

use of org.apache.flink.runtime.io.network.partition.CheckpointedResultSubpartition in project flink by apache.

the class ResultSubpartitionRecoveredStateHandler method getBuffer.

@Override
public BufferWithContext<BufferBuilder> getBuffer(ResultSubpartitionInfo subpartitionInfo) throws IOException, InterruptedException {
    // request the buffer from any mapped subpartition as they all will receive the same buffer
    final List<CheckpointedResultSubpartition> channels = getMappedChannels(subpartitionInfo);
    BufferBuilder bufferBuilder = channels.get(0).requestBufferBuilderBlocking();
    return new BufferWithContext<>(wrap(bufferBuilder), bufferBuilder);
}

Example 3 with CheckpointedResultSubpartition

use of org.apache.flink.runtime.io.network.partition.CheckpointedResultSubpartition in project flink by apache.

the class ResultSubpartitionRecoveredStateHandler method calculateMapping.

private List<CheckpointedResultSubpartition> calculateMapping(ResultSubpartitionInfo info) {
    final RescaleMappings oldToNewMapping = oldToNewMappings.computeIfAbsent(info.getPartitionIdx(), idx -> channelMapping.getChannelMapping(idx).invert());
    final List<CheckpointedResultSubpartition> subpartitions = Arrays.stream(oldToNewMapping.getMappedIndexes(info.getSubPartitionIdx())).mapToObj(newIndexes -> getSubpartition(info.getPartitionIdx(), newIndexes)).collect(Collectors.toList());
    if (subpartitions.isEmpty()) {
        throw new IllegalStateException("Recovered a buffer from old " + info + " that has no mapping in " + channelMapping.getChannelMapping(info.getPartitionIdx()));
    }
    return subpartitions;
}