Examples with BufferBuilder org.apache.flink.runtime.io.network.buffer.BufferBuilder used on opensource projects

Example 26 with BufferBuilder

use of org.apache.flink.runtime.io.network.buffer.BufferBuilder in project flink-mirror by flink-ci.

the class BufferWritingResultPartition method appendBroadcastDataForRecordContinuation.

private BufferBuilder appendBroadcastDataForRecordContinuation(final ByteBuffer remainingRecordBytes) throws IOException {
    final BufferBuilder buffer = requestNewBroadcastBufferBuilder();
    // !! Be aware, in case of partialRecordBytes != 0, partial length and data has to
    // `appendAndCommit` first
    // before consumer is created. Otherwise it would be confused with the case the buffer
    // starting
    // with a complete record.
    // !! The next two lines can not change order.
    final int partialRecordBytes = buffer.appendAndCommit(remainingRecordBytes);
    createBroadcastBufferConsumers(buffer, partialRecordBytes, partialRecordBytes);
    return buffer;
}

Example 27 with BufferBuilder

use of org.apache.flink.runtime.io.network.buffer.BufferBuilder in project flink-mirror by flink-ci.

the class BufferWritingResultPartition method requestNewBroadcastBufferBuilder.

private BufferBuilder requestNewBroadcastBufferBuilder() throws IOException {
    checkInProduceState();
    ensureBroadcastMode();
    final BufferBuilder bufferBuilder = requestNewBufferBuilderFromPool(0);
    broadcastBufferBuilder = bufferBuilder;
    return bufferBuilder;
}

Example 28 with BufferBuilder

use of org.apache.flink.runtime.io.network.buffer.BufferBuilder in project flink-mirror by flink-ci.

the class HashBasedDataBuffer method release.

@Override
public void release() {
    if (isReleased) {
        return;
    }
    isReleased = true;
    for (int channel = 0; channel < builders.length; ++channel) {
        BufferBuilder builder = builders[channel];
        if (builder != null) {
            builder.close();
            builders[channel] = null;
        }
    }
    for (ArrayDeque<BufferConsumer> buffer : buffers) {
        BufferConsumer consumer = buffer.poll();
        while (consumer != null) {
            consumer.close();
            consumer = buffer.poll();
        }
    }
}

Example 29 with BufferBuilder

use of org.apache.flink.runtime.io.network.buffer.BufferBuilder in project flink-mirror by flink-ci.

the class SpanningRecordSerializationTest method appendLeftOverBytes.

private static Buffer appendLeftOverBytes(Buffer buffer, byte[] leftOverBytes) {
    try (BufferBuilder bufferBuilder = new BufferBuilder(MemorySegmentFactory.allocateUnpooledSegment(buffer.readableBytes() + leftOverBytes.length), FreeingBufferRecycler.INSTANCE)) {
        try (BufferConsumer bufferConsumer = bufferBuilder.createBufferConsumer()) {
            bufferBuilder.append(buffer.getNioBufferReadable());
            bufferBuilder.appendAndCommit(ByteBuffer.wrap(leftOverBytes));
            return bufferConsumer.build();
        }
    }
}

Example 30 with BufferBuilder

use of org.apache.flink.runtime.io.network.buffer.BufferBuilder in project flink-mirror by flink-ci.

the class ResultSubpartitionRecoveredStateHandler method getBuffer.

@Override
public BufferWithContext<BufferBuilder> getBuffer(ResultSubpartitionInfo subpartitionInfo) throws IOException, InterruptedException {
    // request the buffer from any mapped subpartition as they all will receive the same buffer
    final List<CheckpointedResultSubpartition> channels = getMappedChannels(subpartitionInfo);
    BufferBuilder bufferBuilder = channels.get(0).requestBufferBuilderBlocking();
    return new BufferWithContext<>(wrap(bufferBuilder), bufferBuilder);
}