Search in sources :

Example 1 with PerSessionSerializationService

use of org.apache.ignite.internal.network.serialization.PerSessionSerializationService in project ignite-3 by apache.

the class InboundDecoderTest method testPartialReadWithReuseBuffer.

/**
 * Tests that an {@link InboundDecoder} can handle a {@link ByteBuf} where reader index is not {@code 0} at the start of the {@link
 * InboundDecoder#decode}.
 *
 * @throws Exception If failed.
 */
@Test
public void testPartialReadWithReuseBuffer() throws Exception {
    ChannelHandlerContext ctx = Mockito.mock(ChannelHandlerContext.class);
    var channel = new EmbeddedChannel();
    Mockito.doReturn(channel).when(ctx).channel();
    var serializationService = new SerializationService(registry, mock(UserObjectSerializationContext.class));
    var perSessionSerializationService = new PerSessionSerializationService(serializationService);
    final var decoder = new InboundDecoder(perSessionSerializationService);
    final var list = new ArrayList<>();
    var writer = new DirectMessageWriter(perSessionSerializationService, ConnectionManager.DIRECT_PROTOCOL_VERSION);
    var msg = new TestMessagesFactory().testMessage().msg("abcdefghijklmn").build();
    MessageSerializer<NetworkMessage> serializer = registry.createSerializer(msg.groupType(), msg.messageType());
    ByteBuffer nioBuffer = ByteBuffer.allocate(10_000);
    writer.setBuffer(nioBuffer);
    // Write message to the ByteBuffer.
    boolean fullyWritten = serializer.writeMessage(msg, writer);
    assertTrue(fullyWritten);
    nioBuffer.flip();
    ByteBuf buffer = allocator.buffer();
    // Write first 3 bytes of a message.
    for (int i = 0; i < 3; i++) {
        buffer.writeByte(nioBuffer.get());
    }
    decoder.decode(ctx, buffer, list);
    // At this point a header and a first byte of a message have been decoded.
    assertEquals(0, list.size());
    // Write next 3 bytes of a message.
    for (int i = 0; i < 3; i++) {
        buffer.writeByte(nioBuffer.get());
    }
    // Reader index of a buffer is not zero and it must be handled correctly by the InboundDecoder.
    decoder.decode(ctx, buffer, list);
    // Check if reader index has been tracked correctly.
    assertEquals(6, buffer.readerIndex());
    assertEquals(0, list.size());
    buffer.writeBytes(nioBuffer);
    decoder.decode(ctx, buffer, list);
    assertEquals(1, list.size());
    TestMessage actualMessage = (TestMessage) list.get(0);
    assertEquals(msg, actualMessage);
}
Also used : ArrayList(java.util.ArrayList) EmbeddedChannel(io.netty.channel.embedded.EmbeddedChannel) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) SerializationService(org.apache.ignite.internal.network.serialization.SerializationService) TestMessagesFactory(org.apache.ignite.network.TestMessagesFactory) ChannelHandlerContext(io.netty.channel.ChannelHandlerContext) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) ByteBuf(io.netty.buffer.ByteBuf) ByteBuffer(java.nio.ByteBuffer) DirectMessageWriter(org.apache.ignite.internal.network.direct.DirectMessageWriter) TestMessage(org.apache.ignite.network.TestMessage) NetworkMessage(org.apache.ignite.network.NetworkMessage) UserObjectSerializationContext(org.apache.ignite.internal.network.serialization.UserObjectSerializationContext) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 2 with PerSessionSerializationService

use of org.apache.ignite.internal.network.serialization.PerSessionSerializationService in project ignite-3 by apache.

the class NettyServer method start.

/**
 * Starts the server.
 *
 * @return Future that resolves when the server is successfully started.
 */
public CompletableFuture<Void> start() {
    synchronized (startStopLock) {
        if (stopped) {
            throw new IgniteInternalException("Attempted to start an already stopped server");
        }
        if (serverStartFuture != null) {
            throw new IgniteInternalException("Attempted to start an already started server");
        }
        ServerBootstrap bootstrap = bootstrapFactory.createServerBootstrap();
        bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {

            /**
             * {@inheritDoc}
             */
            @Override
            public void initChannel(SocketChannel ch) {
                var sessionSerializationService = new PerSessionSerializationService(serializationService);
                // Get handshake manager for the new channel.
                HandshakeManager manager = handshakeManager.get();
                ch.pipeline().addLast(/*
                                     * Decoder that uses the MessageReader
                                     * to read chunked data.
                                     */
                new InboundDecoder(sessionSerializationService), // Handshake handler.
                new HandshakeHandler(manager, (consistentId) -> new MessageHandler(messageListener, consistentId, sessionSerializationService)), /*
                                     * Encoder that uses the MessageWriter
                                     * to write chunked data.
                                     */
                new ChunkedWriteHandler(), // Converts NetworkMessage to a ChunkedNetworkMessageInput
                new OutboundEncoder(sessionSerializationService), new IoExceptionSuppressingHandler());
                manager.handshakeFuture().thenAccept(newConnectionListener);
            }
        });
        int port = configuration.port();
        int portRange = configuration.portRange();
        var bindFuture = new CompletableFuture<Channel>();
        tryBind(bootstrap, port, port + portRange, port, bindFuture);
        serverStartFuture = bindFuture.handle((channel, err) -> {
            synchronized (startStopLock) {
                if (channel != null) {
                    serverCloseFuture = NettyUtils.toCompletableFuture(channel.closeFuture());
                }
                this.channel = (ServerChannel) channel;
                if (err != null || stopped) {
                    Throwable stopErr = err != null ? err : new CancellationException("Server was stopped");
                    return CompletableFuture.<Void>failedFuture(stopErr);
                } else {
                    return CompletableFuture.<Void>completedFuture(null);
                }
            }
        }).thenCompose(Function.identity());
        return serverStartFuture;
    }
}
Also used : SocketChannel(io.netty.channel.socket.SocketChannel) IgniteInternalException(org.apache.ignite.lang.IgniteInternalException) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) ServerBootstrap(io.netty.bootstrap.ServerBootstrap) CompletableFuture(java.util.concurrent.CompletableFuture) ChunkedWriteHandler(io.netty.handler.stream.ChunkedWriteHandler) CancellationException(java.util.concurrent.CancellationException) HandshakeManager(org.apache.ignite.internal.network.handshake.HandshakeManager)

Example 3 with PerSessionSerializationService

use of org.apache.ignite.internal.network.serialization.PerSessionSerializationService in project ignite-3 by apache.

the class InboundDecoderTest method testPartialHeader.

/**
 * Tests that an {@link InboundDecoder} doesn't hang if it encounters a byte buffer with only partially written header.
 *
 * @throws Exception If failed.
 */
@Test
public void testPartialHeader() throws Exception {
    var serializationService = new SerializationService(registry, mock(UserObjectSerializationContext.class));
    var perSessionSerializationService = new PerSessionSerializationService(serializationService);
    var channel = new EmbeddedChannel(new InboundDecoder(perSessionSerializationService));
    ByteBuf buffer = allocator.buffer();
    buffer.writeByte(1);
    CompletableFuture.runAsync(() -> {
        channel.writeInbound(buffer);
        // In case if a header was written partially, readInbound() should not loop forever, as
        // there might not be new data anymore (example: remote host gone offline).
        channel.readInbound();
    }).get(3, TimeUnit.SECONDS);
}
Also used : PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) SerializationService(org.apache.ignite.internal.network.serialization.SerializationService) EmbeddedChannel(io.netty.channel.embedded.EmbeddedChannel) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) ByteBuf(io.netty.buffer.ByteBuf) UserObjectSerializationContext(org.apache.ignite.internal.network.serialization.UserObjectSerializationContext) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 4 with PerSessionSerializationService

use of org.apache.ignite.internal.network.serialization.PerSessionSerializationService in project ignite-3 by apache.

the class InboundDecoderTest method sendAndReceive.

/**
 * Serializes and then deserializes the given message.
 */
private <T extends NetworkMessage> T sendAndReceive(T msg) {
    var serializationService = new SerializationService(registry, mock(UserObjectSerializationContext.class));
    var perSessionSerializationService = new PerSessionSerializationService(serializationService);
    var channel = new EmbeddedChannel(new InboundDecoder(perSessionSerializationService));
    var writer = new DirectMessageWriter(perSessionSerializationService, ConnectionManager.DIRECT_PROTOCOL_VERSION);
    MessageSerializer<NetworkMessage> serializer = registry.createSerializer(msg.groupType(), msg.messageType());
    ByteBuffer buf = ByteBuffer.allocate(10_000);
    T received;
    do {
        buf.clear();
        writer.setBuffer(buf);
        serializer.writeMessage(msg, writer);
        buf.flip();
        ByteBuf buffer = allocator.buffer(buf.limit());
        buffer.writeBytes(buf);
        channel.writeInbound(buffer);
    } while ((received = channel.readInbound()) == null);
    return received;
}
Also used : DirectMessageWriter(org.apache.ignite.internal.network.direct.DirectMessageWriter) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) SerializationService(org.apache.ignite.internal.network.serialization.SerializationService) EmbeddedChannel(io.netty.channel.embedded.EmbeddedChannel) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) NetworkMessage(org.apache.ignite.network.NetworkMessage) ByteBuf(io.netty.buffer.ByteBuf) UserObjectSerializationContext(org.apache.ignite.internal.network.serialization.UserObjectSerializationContext) ByteBuffer(java.nio.ByteBuffer)

Example 5 with PerSessionSerializationService

use of org.apache.ignite.internal.network.serialization.PerSessionSerializationService in project ignite-3 by apache.

the class NettyClient method start.

/**
 * Start client.
 *
 * @param bootstrapTemplate Template client bootstrap.
 * @return Future that resolves when client channel is opened.
 */
public CompletableFuture<NettySender> start(Bootstrap bootstrapTemplate) {
    synchronized (startStopLock) {
        if (stopped) {
            throw new IgniteInternalException("Attempted to start an already stopped NettyClient");
        }
        if (clientFuture != null) {
            throw new IgniteInternalException("Attempted to start an already started NettyClient");
        }
        Bootstrap bootstrap = bootstrapTemplate.clone();
        bootstrap.handler(new ChannelInitializer<SocketChannel>() {

            /**
             * {@inheritDoc}
             */
            @Override
            public void initChannel(SocketChannel ch) {
                var sessionSerializationService = new PerSessionSerializationService(serializationService);
                ch.pipeline().addLast(new InboundDecoder(sessionSerializationService), new HandshakeHandler(handshakeManager, (consistentId) -> new MessageHandler(messageListener, consistentId, sessionSerializationService)), new ChunkedWriteHandler(), new OutboundEncoder(sessionSerializationService), new IoExceptionSuppressingHandler());
            }
        });
        clientFuture = NettyUtils.toChannelCompletableFuture(bootstrap.connect(address)).handle((channel, throwable) -> {
            synchronized (startStopLock) {
                this.channel = channel;
                if (throwable != null) {
                    channelFuture.completeExceptionally(throwable);
                } else {
                    channelFuture.complete(null);
                }
                if (stopped) {
                    return CompletableFuture.<NettySender>failedFuture(new CancellationException("Client was stopped"));
                } else if (throwable != null) {
                    return CompletableFuture.<NettySender>failedFuture(throwable);
                } else {
                    return handshakeManager.handshakeFuture();
                }
            }
        }).thenCompose(Function.identity());
        return clientFuture;
    }
}
Also used : SocketChannel(io.netty.channel.socket.SocketChannel) IgniteInternalException(org.apache.ignite.lang.IgniteInternalException) PerSessionSerializationService(org.apache.ignite.internal.network.serialization.PerSessionSerializationService) ChunkedWriteHandler(io.netty.handler.stream.ChunkedWriteHandler) CancellationException(java.util.concurrent.CancellationException) Bootstrap(io.netty.bootstrap.Bootstrap)

Aggregations

PerSessionSerializationService (org.apache.ignite.internal.network.serialization.PerSessionSerializationService)5 ByteBuf (io.netty.buffer.ByteBuf)3 EmbeddedChannel (io.netty.channel.embedded.EmbeddedChannel)3 SerializationService (org.apache.ignite.internal.network.serialization.SerializationService)3 UserObjectSerializationContext (org.apache.ignite.internal.network.serialization.UserObjectSerializationContext)3 SocketChannel (io.netty.channel.socket.SocketChannel)2 ChunkedWriteHandler (io.netty.handler.stream.ChunkedWriteHandler)2 ByteBuffer (java.nio.ByteBuffer)2 CancellationException (java.util.concurrent.CancellationException)2 DirectMessageWriter (org.apache.ignite.internal.network.direct.DirectMessageWriter)2 IgniteInternalException (org.apache.ignite.lang.IgniteInternalException)2 NetworkMessage (org.apache.ignite.network.NetworkMessage)2 Test (org.junit.jupiter.api.Test)2 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)2 Bootstrap (io.netty.bootstrap.Bootstrap)1 ServerBootstrap (io.netty.bootstrap.ServerBootstrap)1 ChannelHandlerContext (io.netty.channel.ChannelHandlerContext)1 ArrayList (java.util.ArrayList)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 HandshakeManager (org.apache.ignite.internal.network.handshake.HandshakeManager)1