Search in sources :

Example 21 with Future

use of org.apache.flink.shaded.netty4.io.netty.util.concurrent.Future in project redisson by redisson.

the class RedisClient method resolveAddr.

public CompletableFuture<InetSocketAddress> resolveAddr() {
    if (resolvedAddrFuture.get() != null) {
        return resolvedAddrFuture.get();
    }
    CompletableFuture<InetSocketAddress> promise = new CompletableFuture<>();
    if (!resolvedAddrFuture.compareAndSet(null, promise)) {
        return resolvedAddrFuture.get();
    }
    byte[] addr = NetUtil.createByteArrayFromIpAddressString(uri.getHost());
    if (addr != null) {
        try {
            resolvedAddr = new InetSocketAddress(InetAddress.getByAddress(uri.getHost(), addr), uri.getPort());
        } catch (UnknownHostException e) {
        // skip
        }
        promise.complete(resolvedAddr);
        return promise;
    }
    AddressResolver<InetSocketAddress> resolver = (AddressResolver<InetSocketAddress>) bootstrap.config().resolver().getResolver(bootstrap.config().group().next());
    Future<InetSocketAddress> resolveFuture = resolver.resolve(InetSocketAddress.createUnresolved(uri.getHost(), uri.getPort()));
    resolveFuture.addListener((FutureListener<InetSocketAddress>) future -> {
        if (!future.isSuccess()) {
            promise.completeExceptionally(future.cause());
            return;
        }
        InetSocketAddress resolved = future.getNow();
        byte[] addr1 = resolved.getAddress().getAddress();
        resolvedAddr = new InetSocketAddress(InetAddress.getByAddress(uri.getHost(), addr1), resolved.getPort());
        promise.complete(resolvedAddr);
    });
    return promise;
}
Also used : DefaultChannelGroup(io.netty.channel.group.DefaultChannelGroup) RedisURI(org.redisson.misc.RedisURI) AtomicReference(java.util.concurrent.atomic.AtomicReference) EpollDatagramChannel(io.netty.channel.epoll.EpollDatagramChannel) EpollSocketChannel(io.netty.channel.epoll.EpollSocketChannel) RFuture(org.redisson.api.RFuture) InetAddress(java.net.InetAddress) CompletableFutureWrapper(org.redisson.misc.CompletableFutureWrapper) io.netty.channel(io.netty.channel) RedisChannelInitializer(org.redisson.client.handler.RedisChannelInitializer) Type(org.redisson.client.handler.RedisChannelInitializer.Type) AddressResolver(io.netty.resolver.AddressResolver) ChannelGroup(io.netty.channel.group.ChannelGroup) FutureListener(io.netty.util.concurrent.FutureListener) java.util.concurrent(java.util.concurrent) NetUtil(io.netty.util.NetUtil) DnsServerAddressStreamProviders(io.netty.resolver.dns.DnsServerAddressStreamProviders) InetSocketAddress(java.net.InetSocketAddress) UnknownHostException(java.net.UnknownHostException) NioEventLoopGroup(io.netty.channel.nio.NioEventLoopGroup) Bootstrap(io.netty.bootstrap.Bootstrap) NioDatagramChannel(io.netty.channel.socket.nio.NioDatagramChannel) HashedWheelTimer(io.netty.util.HashedWheelTimer) Timer(io.netty.util.Timer) ChannelGroupFuture(io.netty.channel.group.ChannelGroupFuture) Future(io.netty.util.concurrent.Future) DnsAddressResolverGroup(io.netty.resolver.dns.DnsAddressResolverGroup) AddressResolver(io.netty.resolver.AddressResolver) UnknownHostException(java.net.UnknownHostException) InetSocketAddress(java.net.InetSocketAddress)

Example 22 with Future

use of org.apache.flink.shaded.netty4.io.netty.util.concurrent.Future in project redisson by redisson.

the class SentinelConnectionManager method performSentinelDNSCheck.

private void performSentinelDNSCheck(FutureListener<List<InetSocketAddress>> commonListener) {
    for (RedisURI host : sentinelHosts) {
        Future<List<InetSocketAddress>> allNodes = sentinelResolver.resolveAll(InetSocketAddress.createUnresolved(host.getHost(), host.getPort()));
        allNodes.addListener((FutureListener<List<InetSocketAddress>>) future -> {
            if (!future.isSuccess()) {
                log.error("Unable to resolve " + host.getHost(), future.cause());
                return;
            }
            future.getNow().stream().map(addr -> toURI(addr)).filter(uri -> !sentinels.containsKey(uri)).forEach(uri -> registerSentinel(uri, getConfig(), host.getHost()));
        });
        if (commonListener != null) {
            allNodes.addListener(commonListener);
        }
    }
}
Also used : AddressResolver(io.netty.resolver.AddressResolver) java.util(java.util) NodeType(org.redisson.api.NodeType) org.redisson.config(org.redisson.config) Logger(org.slf4j.Logger) FutureListener(io.netty.util.concurrent.FutureListener) StringCodec(org.redisson.client.codec.StringCodec) java.util.concurrent(java.util.concurrent) ScheduledFuture(io.netty.util.concurrent.ScheduledFuture) org.redisson.client(org.redisson.client) LoggerFactory(org.slf4j.LoggerFactory) NetUtil(io.netty.util.NetUtil) RedisURI(org.redisson.misc.RedisURI) StringUtil(io.netty.util.internal.StringUtil) InetSocketAddress(java.net.InetSocketAddress) AtomicReference(java.util.concurrent.atomic.AtomicReference) Collectors(java.util.stream.Collectors) RedisCommands(org.redisson.client.protocol.RedisCommands) RFuture(org.redisson.api.RFuture) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) RedisStrictCommand(org.redisson.client.protocol.RedisStrictCommand) FreezeReason(org.redisson.connection.ClientConnectionsEntry.FreezeReason) Future(io.netty.util.concurrent.Future) RedisURI(org.redisson.misc.RedisURI)

Example 23 with Future

use of org.apache.flink.shaded.netty4.io.netty.util.concurrent.Future in project flink by apache.

the class RestClient method submitRequest.

private <P extends ResponseBody> CompletableFuture<P> submitRequest(String targetAddress, int targetPort, Request httpRequest, JavaType responseType) {
    final ChannelFuture connectFuture = bootstrap.connect(targetAddress, targetPort);
    final CompletableFuture<Channel> channelFuture = new CompletableFuture<>();
    connectFuture.addListener((ChannelFuture future) -> {
        if (future.isSuccess()) {
            channelFuture.complete(future.channel());
        } else {
            channelFuture.completeExceptionally(future.cause());
        }
    });
    return channelFuture.thenComposeAsync(channel -> {
        ClientHandler handler = channel.pipeline().get(ClientHandler.class);
        CompletableFuture<JsonResponse> future;
        boolean success = false;
        try {
            if (handler == null) {
                throw new IOException("Netty pipeline was not properly initialized.");
            } else {
                httpRequest.writeTo(channel);
                future = handler.getJsonFuture();
                success = true;
            }
        } catch (IOException e) {
            future = FutureUtils.completedExceptionally(new ConnectionException("Could not write request.", e));
        } finally {
            if (!success) {
                channel.close();
            }
        }
        return future;
    }, executor).thenComposeAsync((JsonResponse rawResponse) -> parseResponse(rawResponse, responseType), executor);
}
Also used : ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) SimpleChannelInboundHandler(org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler) MessageParameters(org.apache.flink.runtime.rest.messages.MessageParameters) FullHttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse) ChunkedWriteHandler(org.apache.flink.shaded.netty4.io.netty.handler.stream.ChunkedWriteHandler) OutboundChannelHandlerFactory(org.apache.flink.runtime.io.network.netty.OutboundChannelHandlerFactory) MemoryAttribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.MemoryAttribute) LoggerFactory(org.slf4j.LoggerFactory) ExceptionUtils(org.apache.flink.util.ExceptionUtils) JsonParser(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser) JsonNode(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonNode) NetUtils(org.apache.flink.util.NetUtils) EmptyMessageParameters(org.apache.flink.runtime.rest.messages.EmptyMessageParameters) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) HttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse) RequestBody(org.apache.flink.runtime.rest.messages.RequestBody) HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) RestClientException(org.apache.flink.runtime.rest.util.RestClientException) Path(java.nio.file.Path) ChannelHandlerContext(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) Attribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.Attribute) ExecutorThreadFactory(org.apache.flink.util.concurrent.ExecutorThreadFactory) RestConstants(org.apache.flink.runtime.rest.util.RestConstants) Collection(java.util.Collection) NioEventLoopGroup(org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoopGroup) HttpVersion(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion) HttpObjectAggregator(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpObjectAggregator) IdleStateHandler(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateHandler) ServiceLoader(java.util.ServiceLoader) JavaType(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JavaType) Preconditions(org.apache.flink.util.Preconditions) ByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf) Collectors(java.util.stream.Collectors) List(java.util.List) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) HttpClientCodec(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpClientCodec) DefaultHttpDataFactory(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.DefaultHttpDataFactory) Optional(java.util.Optional) IdleStateEvent(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateEvent) Time(org.apache.flink.api.common.time.Time) RestMapperUtils(org.apache.flink.runtime.rest.util.RestMapperUtils) Bootstrap(org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap) ChannelInitializer(org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer) ConfigurationException(org.apache.flink.util.ConfigurationException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SSLHandlerFactory(org.apache.flink.runtime.io.network.netty.SSLHandlerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) ChannelHandler(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler) MessageHeaders(org.apache.flink.runtime.rest.messages.MessageHeaders) HttpResponseStatus(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus) ArrayList(java.util.ArrayList) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) ByteBufInputStream(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBufInputStream) HttpHeaders(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaders) REQUEST_ENTITY_TOO_LARGE(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus.REQUEST_ENTITY_TOO_LARGE) ObjectMapper(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper) ConfigConstants(org.apache.flink.configuration.ConfigConstants) RestOptions(org.apache.flink.configuration.RestOptions) RestAPIVersion(org.apache.flink.runtime.rest.versioning.RestAPIVersion) ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) ChannelOption(org.apache.flink.shaded.netty4.io.netty.channel.ChannelOption) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) AutoCloseableAsync(org.apache.flink.util.AutoCloseableAsync) Files(java.nio.file.Files) Executor(java.util.concurrent.Executor) StringWriter(java.io.StringWriter) Configuration(org.apache.flink.configuration.Configuration) Unpooled(org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled) IOException(java.io.IOException) JsonProcessingException(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException) File(java.io.File) VisibleForTesting(org.apache.flink.annotation.VisibleForTesting) TimeUnit(java.util.concurrent.TimeUnit) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody) TooLongFrameException(org.apache.flink.shaded.netty4.io.netty.handler.codec.TooLongFrameException) HttpPostRequestEncoder(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.HttpPostRequestEncoder) HttpMethod(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod) Comparator(java.util.Comparator) Collections(java.util.Collections) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) InputStream(java.io.InputStream) CompletableFuture(java.util.concurrent.CompletableFuture) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) IOException(java.io.IOException)

Example 24 with Future

use of org.apache.flink.shaded.netty4.io.netty.util.concurrent.Future in project flink by apache.

the class PartitionRequestClientFactory method connect.

private NettyPartitionRequestClient connect(ConnectionID connectionId) throws RemoteTransportException, InterruptedException {
    try {
        // It's important to use `sync` here because it waits for this future until it is
        // done, and rethrows the cause of the failure if this future failed. `await` only
        // waits for this future to be completed, without throwing the error.
        Channel channel = nettyClient.connect(connectionId.getAddress()).sync().channel();
        NetworkClientHandler clientHandler = channel.pipeline().get(NetworkClientHandler.class);
        return new NettyPartitionRequestClient(channel, clientHandler, connectionId, this);
    } catch (InterruptedException e) {
        throw e;
    } catch (Exception e) {
        throw new RemoteTransportException("Connecting to remote task manager '" + connectionId.getAddress() + "' has failed. This might indicate that the remote task " + "manager has been lost.", connectionId.getAddress(), e);
    }
}
Also used : RemoteTransportException(org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException) NetworkClientHandler(org.apache.flink.runtime.io.network.NetworkClientHandler) RemoteInputChannel(org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) RemoteTransportException(org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException)

Example 25 with Future

use of org.apache.flink.shaded.netty4.io.netty.util.concurrent.Future in project flink by apache.

the class ClientTest method testServerClosesChannel.

/**
 * Tests that a server channel close, closes the connection and removes it from the established
 * connections.
 */
@Test
public void testServerClosesChannel() throws Exception {
    AtomicKvStateRequestStats stats = new AtomicKvStateRequestStats();
    final MessageSerializer<KvStateInternalRequest, KvStateResponse> serializer = new MessageSerializer<>(new KvStateInternalRequest.KvStateInternalRequestDeserializer(), new KvStateResponse.KvStateResponseDeserializer());
    Client<KvStateInternalRequest, KvStateResponse> client = null;
    Channel serverChannel = null;
    try {
        client = new Client<>("Test Client", 1, serializer, stats);
        final LinkedBlockingQueue<ByteBuf> received = new LinkedBlockingQueue<>();
        final AtomicReference<Channel> channel = new AtomicReference<>();
        serverChannel = createServerChannel(new ChannelDataCollectingHandler(channel, received));
        InetSocketAddress serverAddress = getKvStateServerAddress(serverChannel);
        // Requests
        KvStateInternalRequest request = new KvStateInternalRequest(new KvStateID(), new byte[0]);
        Future<KvStateResponse> future = client.sendRequest(serverAddress, request);
        received.take();
        assertEquals(1, stats.getNumConnections());
        channel.get().close().await();
        try {
            future.get();
            fail("Did not throw expected server failure");
        } catch (ExecutionException e) {
            if (!(e.getCause() instanceof ClosedChannelException)) {
                fail("Did not throw expected Exception");
            }
        // Expected
        }
        assertEquals(0L, stats.getNumConnections());
        // Counts can take some time to propagate
        while (stats.getNumSuccessful() != 0L || stats.getNumFailed() != 1L) {
            Thread.sleep(100L);
        }
        assertEquals(1L, stats.getNumRequests());
        assertEquals(0L, stats.getNumSuccessful());
        assertEquals(1L, stats.getNumFailed());
    } finally {
        if (client != null) {
            try {
                client.shutdown().get();
            } catch (Exception e) {
                e.printStackTrace();
            }
            Assert.assertTrue(client.isEventGroupShutdown());
        }
        if (serverChannel != null) {
            serverChannel.close();
        }
        assertEquals("Channel leak", 0L, stats.getNumConnections());
    }
}
Also used : ClosedChannelException(java.nio.channels.ClosedChannelException) MessageSerializer(org.apache.flink.queryablestate.network.messages.MessageSerializer) InetSocketAddress(java.net.InetSocketAddress) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) NioServerSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioServerSocketChannel) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) AtomicReference(java.util.concurrent.atomic.AtomicReference) ByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) ConnectException(java.net.ConnectException) ClosedChannelException(java.nio.channels.ClosedChannelException) UnknownHostException(java.net.UnknownHostException) ExecutionException(java.util.concurrent.ExecutionException) KvStateInternalRequest(org.apache.flink.queryablestate.messages.KvStateInternalRequest) KvStateResponse(org.apache.flink.queryablestate.messages.KvStateResponse) KvStateID(org.apache.flink.queryablestate.KvStateID) ExecutionException(java.util.concurrent.ExecutionException) AtomicKvStateRequestStats(org.apache.flink.queryablestate.network.stats.AtomicKvStateRequestStats) Test(org.junit.Test)

Aggregations

Future (io.netty.util.concurrent.Future)177 Channel (io.netty.channel.Channel)61 ChannelFuture (io.netty.channel.ChannelFuture)58 InetSocketAddress (java.net.InetSocketAddress)45 ArrayList (java.util.ArrayList)45 IOException (java.io.IOException)44 GenericFutureListener (io.netty.util.concurrent.GenericFutureListener)42 CompletableFuture (java.util.concurrent.CompletableFuture)40 NioEventLoopGroup (io.netty.channel.nio.NioEventLoopGroup)35 ChannelHandlerContext (io.netty.channel.ChannelHandlerContext)34 List (java.util.List)34 ChannelFutureListener (io.netty.channel.ChannelFutureListener)31 EventLoopGroup (io.netty.channel.EventLoopGroup)30 NioSocketChannel (io.netty.channel.socket.nio.NioSocketChannel)30 FutureListener (io.netty.util.concurrent.FutureListener)28 Logger (org.slf4j.Logger)28 LoggerFactory (org.slf4j.LoggerFactory)28 TimeUnit (java.util.concurrent.TimeUnit)27 Bootstrap (io.netty.bootstrap.Bootstrap)25 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)25