Search in sources :

Example 6 with ByteBuf

use of org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf in project incubator-ratis by apache.

the class NettyDataStreamUtils method decodeDataStreamRequestHeader.

static DataStreamRequestHeader decodeDataStreamRequestHeader(ByteBuf buf) {
    if (DataStreamPacketHeader.getSizeOfHeaderBodyLen() > buf.readableBytes()) {
        return null;
    }
    long headerBodyBufLen = buf.readLong();
    if (headerBodyBufLen > buf.readableBytes()) {
        buf.resetReaderIndex();
        return null;
    }
    int headerBufLen = buf.readInt();
    if (headerBufLen > buf.readableBytes()) {
        buf.resetReaderIndex();
        return null;
    }
    try {
        ByteBuf headerBuf = buf.slice(buf.readerIndex(), headerBufLen);
        DataStreamRequestHeaderProto header = DataStreamRequestHeaderProto.parseFrom(headerBuf.nioBuffer());
        final DataStreamPacketHeaderProto h = header.getPacketHeader();
        if (h.getDataLength() + headerBufLen <= buf.readableBytes()) {
            buf.readerIndex(buf.readerIndex() + headerBufLen);
            WriteOption[] options = new WriteOption[h.getOptionsCount()];
            for (int i = 0; i < options.length; i++) {
                options[i] = StandardWriteOption.values()[h.getOptions(i).ordinal()];
            }
            return new DataStreamRequestHeader(ClientId.valueOf(h.getClientId()), h.getType(), h.getStreamId(), h.getStreamOffset(), h.getDataLength(), options);
        } else {
            buf.resetReaderIndex();
            return null;
        }
    } catch (InvalidProtocolBufferException e) {
        LOG.error("Fail to decode request header:", e);
        buf.resetReaderIndex();
        return null;
    }
}
Also used : DataStreamPacketHeaderProto(org.apache.ratis.proto.RaftProtos.DataStreamPacketHeaderProto) DataStreamRequestHeaderProto(org.apache.ratis.proto.RaftProtos.DataStreamRequestHeaderProto) InvalidProtocolBufferException(org.apache.ratis.thirdparty.com.google.protobuf.InvalidProtocolBufferException) DataStreamRequestHeader(org.apache.ratis.protocol.DataStreamRequestHeader) DataStreamRequestByteBuf(org.apache.ratis.netty.server.DataStreamRequestByteBuf) ByteBuf(org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf) StandardWriteOption(org.apache.ratis.io.StandardWriteOption) WriteOption(org.apache.ratis.io.WriteOption)

Example 7 with ByteBuf

use of org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf in project incubator-ratis by apache.

the class NettyDataStreamUtils method encodeDataStreamReplyByteBuffer.

static void encodeDataStreamReplyByteBuffer(DataStreamReplyByteBuffer reply, Consumer<ByteBuf> out, ByteBufAllocator allocator) {
    ByteBuffer headerBuf = getDataStreamReplyHeaderProtoByteBuf(reply);
    final ByteBuf headerLenBuf = allocator.directBuffer(DataStreamPacketHeader.getSizeOfHeaderLen());
    headerLenBuf.writeInt(headerBuf.remaining());
    out.accept(headerLenBuf);
    out.accept(Unpooled.wrappedBuffer(headerBuf));
    out.accept(Unpooled.wrappedBuffer(reply.slice()));
}
Also used : DataStreamRequestByteBuf(org.apache.ratis.netty.server.DataStreamRequestByteBuf) ByteBuf(org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf) DataStreamRequestByteBuffer(org.apache.ratis.datastream.impl.DataStreamRequestByteBuffer) ByteBuffer(java.nio.ByteBuffer) DataStreamReplyByteBuffer(org.apache.ratis.datastream.impl.DataStreamReplyByteBuffer)

Example 8 with ByteBuf

use of org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf in project incubator-ratis by apache.

the class NettyDataStreamUtils method encodeDataStreamRequestHeader.

static void encodeDataStreamRequestHeader(DataStreamRequest request, Consumer<Object> out, ByteBufAllocator allocator) {
    final ByteBuffer headerBuf = getDataStreamRequestHeaderProtoByteBuffer(request);
    final ByteBuf headerBodyLenBuf = allocator.directBuffer(DataStreamPacketHeader.getSizeOfHeaderBodyLen());
    headerBodyLenBuf.writeLong(headerBuf.remaining() + request.getDataLength());
    out.accept(headerBodyLenBuf);
    final ByteBuf headerLenBuf = allocator.directBuffer(DataStreamPacketHeader.getSizeOfHeaderLen());
    headerLenBuf.writeInt(headerBuf.remaining());
    out.accept(headerLenBuf);
    out.accept(Unpooled.wrappedBuffer(headerBuf));
}
Also used : DataStreamRequestByteBuf(org.apache.ratis.netty.server.DataStreamRequestByteBuf) ByteBuf(org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf) DataStreamRequestByteBuffer(org.apache.ratis.datastream.impl.DataStreamRequestByteBuffer) ByteBuffer(java.nio.ByteBuffer) DataStreamReplyByteBuffer(org.apache.ratis.datastream.impl.DataStreamReplyByteBuffer)

Example 9 with ByteBuf

use of org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf in project incubator-ratis by apache.

the class RequestDecoderComposite method decode.

@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
    if (msg.readableBytes() >= 8) {
        int id = msg.readInt();
        int buflen = msg.readInt();
        if (msg.readableBytes() >= buflen) {
            RequestDataComposite req = new RequestDataComposite();
            req.setDataId(id);
            // System.out.printf("msg id and buflen %d and %d bytes\n", id, buflen, msg.readableBytes());
            try {
                ByteBuf bf = msg.slice(msg.readerIndex(), buflen);
                bf.retain();
                req.setBuff(bf);
            } catch (Exception e) {
                System.out.println(e);
            }
            msg.readerIndex(msg.readerIndex() + buflen);
            msg.markReaderIndex();
            out.add(req);
        } else {
            msg.resetReaderIndex();
            return;
        }
    } else {
        return;
    }
}
Also used : RequestDataComposite(org.apache.ratis.experiments.nettyzerocopy.objects.RequestDataComposite) ByteBuf(org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf)

Aggregations

ByteBuf (org.apache.ratis.thirdparty.io.netty.buffer.ByteBuf)9 DataStreamRequestByteBuf (org.apache.ratis.netty.server.DataStreamRequestByteBuf)4 ByteBuffer (java.nio.ByteBuffer)3 DataStreamReplyByteBuffer (org.apache.ratis.datastream.impl.DataStreamReplyByteBuffer)3 CompletableFuture (java.util.concurrent.CompletableFuture)2 DataStreamRequestByteBuffer (org.apache.ratis.datastream.impl.DataStreamRequestByteBuffer)2 StandardWriteOption (org.apache.ratis.io.StandardWriteOption)2 WriteOption (org.apache.ratis.io.WriteOption)2 DataStreamPacketHeaderProto (org.apache.ratis.proto.RaftProtos.DataStreamPacketHeaderProto)2 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)1 IOException (java.io.IOException)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1 List (java.util.List)1 Optional (java.util.Optional)1 Set (java.util.Set)1 CompletionException (java.util.concurrent.CompletionException)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 ConcurrentMap (java.util.concurrent.ConcurrentMap)1 Executor (java.util.concurrent.Executor)1