Search in sources :

Example 6 with TransportClient

use of com.linkedin.r2.transport.common.bridge.client.TransportClient in project rest.li by linkedin.

the class TestQueryTunnel method setUp.

@BeforeClass
protected void setUp() throws Exception {
    Map<String, String> clientProperties = new HashMap<String, String>();
    clientProperties.put(HttpClientFactory.HTTP_QUERY_POST_THRESHOLD, String.valueOf(QUERY_TUNNEL_THRESHOLD));
    clientProperties.put(HttpClientFactory.HTTP_PROTOCOL_VERSION, _httpProtocolVersion);
    _clientFactory = new HttpClientFactory();
    final TransportClient transportClient = _clientFactory.getClient(clientProperties);
    _client = new TransportClientAdapter(transportClient, _clientROS);
    final RestRequestHandler restHandler = new CheckQueryTunnelHandler();
    final StreamRequestHandler streamHandler = new StreamRequestHandlerAdapter(restHandler);
    TransportDispatcher dispatcher = new TransportDispatcher() {

        @Override
        public void handleRestRequest(RestRequest req, Map<String, String> wireAttrs, RequestContext requestContext, TransportCallback<RestResponse> callback) {
            restHandler.handleRequest(req, requestContext, new TransportCallbackAdapter<RestResponse>(callback));
        }

        @Override
        public void handleStreamRequest(StreamRequest req, Map<String, String> wireAttrs, RequestContext requestContext, TransportCallback<StreamResponse> callback) {
            streamHandler.handleRequest(req, requestContext, new TransportCallbackAdapter<StreamResponse>(callback));
        }
    };
    _server = new HttpServerFactory(_servletType).createH2cServer(_port, dispatcher, _serverROS);
    _server.start();
}
Also used : HttpServerFactory(com.linkedin.r2.transport.http.server.HttpServerFactory) TransportCallback(com.linkedin.r2.transport.common.bridge.common.TransportCallback) TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) HashMap(java.util.HashMap) RestResponse(com.linkedin.r2.message.rest.RestResponse) StreamResponse(com.linkedin.r2.message.stream.StreamResponse) TransportDispatcher(com.linkedin.r2.transport.common.bridge.server.TransportDispatcher) RestRequestHandler(com.linkedin.r2.transport.common.RestRequestHandler) StreamRequest(com.linkedin.r2.message.stream.StreamRequest) StreamRequestHandlerAdapter(com.linkedin.r2.transport.common.StreamRequestHandlerAdapter) StreamRequestHandler(com.linkedin.r2.transport.common.StreamRequestHandler) RestRequest(com.linkedin.r2.message.rest.RestRequest) TransportClientAdapter(com.linkedin.r2.transport.common.bridge.client.TransportClientAdapter) RequestContext(com.linkedin.r2.message.RequestContext) HttpClientFactory(com.linkedin.r2.transport.http.client.HttpClientFactory) HashMap(java.util.HashMap) Map(java.util.Map) BeforeClass(org.testng.annotations.BeforeClass)

Example 7 with TransportClient

use of com.linkedin.r2.transport.common.bridge.client.TransportClient in project rest.li by linkedin.

the class TestDisruptor method testStreamLatencyDisrupt.

@Test
public void testStreamLatencyDisrupt() throws Exception {
    final Map<String, String> properties = new HashMap<>();
    final TransportClientFactory factory = new HttpClientFactory.Builder().build();
    final TransportClient client = factory.getClient(properties);
    final RequestContext requestContext = new RequestContext();
    requestContext.putLocalAttr(DISRUPT_CONTEXT_KEY, DisruptContexts.delay(REQUEST_LATENCY));
    final CountDownLatch latch = new CountDownLatch(1);
    final AtomicBoolean success = new AtomicBoolean(false);
    client.streamRequest(new StreamRequestBuilder(new URI(REQUEST_URI)).build(EntityStreams.emptyStream()), requestContext, new HashMap<>(), response -> {
        success.set(!response.hasError() && response.getResponse() != null);
        latch.countDown();
    });
    Assert.assertTrue(latch.await(TEST_TIMEOUT, TimeUnit.MILLISECONDS), "Test execution timeout");
    Assert.assertTrue(success.get(), "Unexpected transport response");
}
Also used : TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) HashMap(java.util.HashMap) CountDownLatch(java.util.concurrent.CountDownLatch) StreamRequestBuilder(com.linkedin.r2.message.stream.StreamRequestBuilder) URI(java.net.URI) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) RequestContext(com.linkedin.r2.message.RequestContext) TransportClientFactory(com.linkedin.r2.transport.common.TransportClientFactory) HttpClientFactory(com.linkedin.r2.transport.http.client.HttpClientFactory) Test(org.testng.annotations.Test)

Example 8 with TransportClient

use of com.linkedin.r2.transport.common.bridge.client.TransportClient in project rest.li by linkedin.

the class Bootstrap method createHttpClient.

public static Client createHttpClient(FilterChain filters, boolean restOverStream) {
    HashMap<String, String> properties = new HashMap<>();
    properties.put(HttpClientFactory.HTTP_PROTOCOL_VERSION, HttpProtocolVersion.HTTP_1_1.name());
    final TransportClient client = new HttpClientFactory.Builder().setFilterChain(filters).build().getClient(properties);
    return new TransportClientAdapter(client, restOverStream);
}
Also used : TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) HashMap(java.util.HashMap) TransportClientAdapter(com.linkedin.r2.transport.common.bridge.client.TransportClientAdapter) HttpClientFactory(com.linkedin.r2.transport.http.client.HttpClientFactory)

Example 9 with TransportClient

use of com.linkedin.r2.transport.common.bridge.client.TransportClient in project rest.li by linkedin.

the class SimpleLoadBalancer method getClient.

/**
 * Given a Request, returns a TransportClient that can handle requests for the Request.
 * The callback is given a client that can be called to retrieve data for the URN.
 *
 * @param request
 *          A request whose URI is a URL of the format "d2://&gt;servicename&lt;/optional/path".
 * @param requestContext context for this request
 * @throws ServiceUnavailableException
 *           If the load balancer can't figure out how to reach a service for the given
 *           URN, an ServiceUnavailableException will be thrown.
 */
@Override
public void getClient(Request request, RequestContext requestContext, Callback<TransportClient> clientCallback) {
    URI uri = request.getURI();
    debug(_log, "get client for uri: ", uri);
    if (!D2_SCHEME_NAME.equalsIgnoreCase(uri.getScheme())) {
        throw new IllegalArgumentException("Unsupported scheme in URI " + uri);
    }
    // get the service for this uri
    String extractedServiceName = LoadBalancerUtil.getServiceNameFromUri(uri);
    listenToServiceAndCluster(extractedServiceName, Callbacks.handle(service -> {
        String serviceName = service.getServiceName();
        String clusterName = service.getClusterName();
        try {
            ClusterProperties cluster = getClusterProperties(serviceName, clusterName);
            // Check if we want to override the service URL and bypass choosing among the existing
            // tracker clients. This is useful when the service we want is not announcing itself to
            // the cluster, ie a private service for a set of clients. This mechanism is deprecated;
            // use host override list instead.
            @SuppressWarnings("deprecation") URI targetService = LoadBalancerUtil.TargetHints.getRequestContextTargetService(requestContext);
            // Checks if we have a host override list provided in the request context. If present,
            // get the override URI available override for the current cluster and service names.
            HostOverrideList overrides = (HostOverrideList) requestContext.getLocalAttr(HOST_OVERRIDE_LIST);
            URI override = overrides == null ? null : overrides.getOverride(clusterName, serviceName);
            if (targetService == null && override == null) {
                LoadBalancerStateItem<UriProperties> uriItem = getUriItem(serviceName, clusterName, cluster);
                UriProperties uris = uriItem.getProperty();
                List<LoadBalancerState.SchemeStrategyPair> orderedStrategies = _state.getStrategiesForService(serviceName, service.getPrioritizedSchemes());
                TrackerClient trackerClient = chooseTrackerClient(request, requestContext, serviceName, clusterName, cluster, uriItem, uris, orderedStrategies, service);
                String clusterAndServiceUriString = trackerClient.getUri() + service.getPath();
                _serviceAvailableStats.inc();
                clientCallback.onSuccess(new RewriteLoadBalancerClient(serviceName, URI.create(clusterAndServiceUriString), trackerClient));
            } else {
                URI target = override == null ? targetService : URI.create(override + service.getPath());
                if (targetService != null && override != null) {
                    _log.warn("Both TargetHints and HostOverrideList are found. HostOverList will take precedence %s.", target);
                }
                if (_log.isDebugEnabled()) {
                    _log.debug("Rewrite URI as specified in the TargetHints/HostOverrideList {} for cluster {} and service {}.", target, clusterName, serviceName);
                }
                TransportClient transportClient = _state.getClient(serviceName, target.getScheme());
                if (transportClient == null) {
                    throw new ServiceUnavailableException(serviceName, String.format("PEGA_1001. Cannot find transportClient for service %s and scheme %s with URI specified in" + "TargetHints/HostOverrideList %s", serviceName, target.getScheme(), target));
                }
                clientCallback.onSuccess(new RewriteLoadBalancerClient(serviceName, target, transportClient));
            }
        } catch (ServiceUnavailableException e) {
            clientCallback.onError(e);
        }
    }, clientCallback));
}
Also used : LoadBalancerStateListenerCallback(com.linkedin.d2.balancer.LoadBalancerState.LoadBalancerStateListenerCallback) NullStateListenerCallback(com.linkedin.d2.balancer.LoadBalancerState.NullStateListenerCallback) LoadBalancerStrategy(com.linkedin.d2.balancer.strategies.LoadBalancerStrategy) LoggerFactory(org.slf4j.LoggerFactory) TimeoutException(java.util.concurrent.TimeoutException) Random(java.util.Random) KeysAndHosts(com.linkedin.d2.balancer.util.KeysAndHosts) Request(com.linkedin.r2.message.Request) LoadBalancerStateItem(com.linkedin.d2.balancer.LoadBalancerStateItem) TransportClientFactory(com.linkedin.r2.transport.common.TransportClientFactory) Map(java.util.Map) URI(java.net.URI) HashFunction(com.linkedin.d2.balancer.util.hashing.HashFunction) ClusterProperties(com.linkedin.d2.balancer.properties.ClusterProperties) Collection(java.util.Collection) LogUtil.warn(com.linkedin.d2.discovery.util.LogUtil.warn) UriProperties(com.linkedin.d2.balancer.properties.UriProperties) Set(java.util.Set) Collectors(java.util.stream.Collectors) TimeoutCallback(com.linkedin.r2.transport.http.client.TimeoutCallback) RewriteLoadBalancerClient(com.linkedin.d2.balancer.clients.RewriteLoadBalancerClient) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) None(com.linkedin.common.util.None) LoadBalancer(com.linkedin.d2.balancer.LoadBalancer) LoadBalancerState(com.linkedin.d2.balancer.LoadBalancerState) PropertyEventShutdownCallback(com.linkedin.d2.discovery.event.PropertyEventThread.PropertyEventShutdownCallback) PartitionData(com.linkedin.d2.balancer.properties.PartitionData) ServiceProperties(com.linkedin.d2.balancer.properties.ServiceProperties) PartitionInfoProvider(com.linkedin.d2.balancer.util.partitions.PartitionInfoProvider) Callbacks(com.linkedin.common.callback.Callbacks) DarkClusterConfigMap(com.linkedin.d2.DarkClusterConfigMap) PartitionAccessException(com.linkedin.d2.balancer.util.partitions.PartitionAccessException) LogUtil.debug(com.linkedin.d2.discovery.util.LogUtil.debug) FutureCallback(com.linkedin.common.callback.FutureCallback) PartitionAccessor(com.linkedin.d2.balancer.util.partitions.PartitionAccessor) Callback(com.linkedin.common.callback.Callback) TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) HashMap(java.util.HashMap) LoadBalancerClusterListener(com.linkedin.d2.balancer.LoadBalancerClusterListener) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) HashRingProvider(com.linkedin.d2.balancer.util.hashing.HashRingProvider) ClusterInfoProvider(com.linkedin.d2.balancer.util.ClusterInfoProvider) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Ring(com.linkedin.d2.balancer.util.hashing.Ring) HostOverrideList(com.linkedin.d2.balancer.util.HostOverrideList) ClientFactoryProvider(com.linkedin.d2.balancer.util.ClientFactoryProvider) SubsettingState(com.linkedin.d2.balancer.subsetting.SubsettingState) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) Stats(com.linkedin.d2.discovery.util.Stats) HostToKeyMapper(com.linkedin.d2.balancer.util.HostToKeyMapper) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) WarmUpService(com.linkedin.d2.balancer.WarmUpService) RequestContext(com.linkedin.r2.message.RequestContext) TreeMap(java.util.TreeMap) LogUtil.info(com.linkedin.d2.discovery.util.LogUtil.info) KeyMapper(com.linkedin.d2.balancer.KeyMapper) MapKeyResult(com.linkedin.d2.balancer.util.MapKeyResult) ServiceUnavailableException(com.linkedin.d2.balancer.ServiceUnavailableException) Collections(java.util.Collections) TrackerClient(com.linkedin.d2.balancer.clients.TrackerClient) LoadBalancerUtil(com.linkedin.d2.balancer.util.LoadBalancerUtil) TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) RewriteLoadBalancerClient(com.linkedin.d2.balancer.clients.RewriteLoadBalancerClient) ServiceUnavailableException(com.linkedin.d2.balancer.ServiceUnavailableException) URI(java.net.URI) LoadBalancerState(com.linkedin.d2.balancer.LoadBalancerState) TrackerClient(com.linkedin.d2.balancer.clients.TrackerClient) UriProperties(com.linkedin.d2.balancer.properties.UriProperties) HostOverrideList(com.linkedin.d2.balancer.util.HostOverrideList) ClusterProperties(com.linkedin.d2.balancer.properties.ClusterProperties) LoadBalancerStateItem(com.linkedin.d2.balancer.LoadBalancerStateItem) List(java.util.List) ArrayList(java.util.ArrayList) HostOverrideList(com.linkedin.d2.balancer.util.HostOverrideList)

Example 10 with TransportClient

use of com.linkedin.r2.transport.common.bridge.client.TransportClient in project rest.li by linkedin.

the class SimpleLoadBalancerState method shutdown.

@Override
public void shutdown(final PropertyEventShutdownCallback shutdown) {
    trace(_log, "shutdown");
    // shutdown all three registries, all tracker clients, and the event thread
    _executor.execute(new PropertyEvent("shutdown load balancer state") {

        @Override
        public void innerRun() {
            // Need to shutdown loadBalancerStrategies before the transportClients are shutdown
            for (Map<String, LoadBalancerStrategy> strategyEntry : _serviceStrategies.values()) {
                strategyEntry.values().forEach(LoadBalancerStrategy::shutdown);
            }
            // put all tracker clients into a single set for convenience
            Set<TransportClient> transportClients = new HashSet<>();
            for (Map<String, TransportClient> clientsByScheme : _serviceClients.values()) {
                transportClients.addAll(clientsByScheme.values());
            }
            Callback<None> trackerCallback = Callbacks.countDown(Callbacks.<None>adaptSimple(new SimpleCallback() {

                @Override
                public void onDone() {
                    shutdown.done();
                }
            }), transportClients.size());
            info(_log, "shutting down cluster clients");
            for (TransportClient transportClient : transportClients) {
                transportClient.shutdown(trackerCallback);
            }
            // so it is needed to notify all the listeners
            for (SimpleLoadBalancerStateListener listener : _listeners) {
                // Notify the strategy removal
                for (Map.Entry<String, Map<String, LoadBalancerStrategy>> serviceStrategy : _serviceStrategies.entrySet()) {
                    for (Map.Entry<String, LoadBalancerStrategy> strategyEntry : serviceStrategy.getValue().entrySet()) {
                        listener.onStrategyRemoved(serviceStrategy.getKey(), strategyEntry.getKey(), strategyEntry.getValue());
                    }
                    // Also notify the client removal
                    Map<URI, TrackerClient> trackerClients = _trackerClients.get(serviceStrategy.getKey());
                    if (trackerClients != null) {
                        for (TrackerClient client : trackerClients.values()) {
                            listener.onClientRemoved(serviceStrategy.getKey(), client);
                        }
                    }
                }
            }
            // When SimpleLoadBalancerStateis shutdown, all the cluster listener also need to be notified.
            for (LoadBalancerClusterListener clusterListener : _clusterListeners) {
                for (String clusterName : _clusterInfo.keySet()) {
                    clusterListener.onClusterRemoved(clusterName);
                }
            }
        }
    });
}
Also used : Set(java.util.Set) HashSet(java.util.HashSet) TransportClient(com.linkedin.r2.transport.common.bridge.client.TransportClient) LoadBalancerClusterListener(com.linkedin.d2.balancer.LoadBalancerClusterListener) PropertyEvent(com.linkedin.d2.discovery.event.PropertyEventThread.PropertyEvent) RelativeLoadBalancerStrategy(com.linkedin.d2.balancer.strategies.relative.RelativeLoadBalancerStrategy) LoadBalancerStrategy(com.linkedin.d2.balancer.strategies.LoadBalancerStrategy) PropertyEventShutdownCallback(com.linkedin.d2.discovery.event.PropertyEventThread.PropertyEventShutdownCallback) Callback(com.linkedin.common.callback.Callback) SimpleCallback(com.linkedin.common.callback.SimpleCallback) TrackerClient(com.linkedin.d2.balancer.clients.TrackerClient) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) None(com.linkedin.common.util.None) SimpleCallback(com.linkedin.common.callback.SimpleCallback)

Aggregations

TransportClient (com.linkedin.r2.transport.common.bridge.client.TransportClient)46 HashMap (java.util.HashMap)28 RequestContext (com.linkedin.r2.message.RequestContext)22 URI (java.net.URI)21 Test (org.testng.annotations.Test)17 HttpClientFactory (com.linkedin.r2.transport.http.client.HttpClientFactory)15 TransportClientFactory (com.linkedin.r2.transport.common.TransportClientFactory)13 TrackerClient (com.linkedin.d2.balancer.clients.TrackerClient)10 TransportClientAdapter (com.linkedin.r2.transport.common.bridge.client.TransportClientAdapter)10 Map (java.util.Map)10 CountDownLatch (java.util.concurrent.CountDownLatch)10 Callback (com.linkedin.common.callback.Callback)9 ServiceProperties (com.linkedin.d2.balancer.properties.ServiceProperties)8 RestRequest (com.linkedin.r2.message.rest.RestRequest)8 UriProperties (com.linkedin.d2.balancer.properties.UriProperties)7 LoadBalancerStrategy (com.linkedin.d2.balancer.strategies.LoadBalancerStrategy)7 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)7 None (com.linkedin.common.util.None)6 ClusterProperties (com.linkedin.d2.balancer.properties.ClusterProperties)6 RestRequestBuilder (com.linkedin.r2.message.rest.RestRequestBuilder)6