Examples with D2Client com.linkedin.d2.balancer.D2Client used on opensource projects

Example 1 with D2Client

use of com.linkedin.d2.balancer.D2Client in project rest.li by linkedin.

the class TestRouteLookupClient method testRouteLookupClientCallback.

@Test
public void testRouteLookupClientCallback() throws InterruptedException, ExecutionException, TimeoutException {
    RouteLookup routeLookup = new SimpleTestRouteLookup();
    final D2Client d2Client = new D2ClientBuilder().setZkHosts("localhost:2121").build();
    d2Client.start(new FutureCallback<None>());
    RouteLookupClient routeLookupClient = new RouteLookupClient(d2Client, routeLookup, "WestCoast");
    RestRequest dummyRestRequest = new RestRequestBuilder(URI.create("d2://simple_uri")).build();
    FutureCallback<RestResponse> futureCallback = new FutureCallback<RestResponse>();
    routeLookupClient.restRequest(dummyRestRequest, futureCallback, "5555");
    try {
        RestResponse response = futureCallback.get(10, TimeUnit.SECONDS);
        Assert.fail("Unexpected success, request should have thrown a ServiceUnavailableException");
    } catch (Exception e) {
        String message = e.getMessage();
        if (!message.contains("_serviceName=simple_uriWestCoast5555Foo")) {
            Assert.fail("request was not rewritten to point at the d2 service simple_uriWestCoast5555Foo");
        }
    }
}

Example 2 with D2Client

use of com.linkedin.d2.balancer.D2Client in project rest.li by linkedin.

the class DegraderLoadBalancerStrategyV3 method checkQuarantineState.

/**
   * checkQuarantineState decides if the D2Quarantine can be enabled or not, by health
   * checking all the trackerClients once. It enables quarantine only if at least one of the
   * clients return success for the checking.
   *
   * The reasons for this checking include:
   *
   * . The default method "OPTIONS" is not always enabled by the service
   * . The user can config any path/method for checking. We do a sanity checking to
   *   make sure the configuration is correct, and service/host responds in time.
   *   Otherwise the host can be kept in quarantine forever if we blindly enable it.
   *
   * This check actually can warm up the R2 connection pool by making a connection to
   * each trackerClient. However since the check happens before any real requests are sent,
   * it generally takes much longer time to get the results, due to different warming up
   * requirements. Therefore the checking will be retried in next update if current check
   * fails.

   *
   * This function is supposed to be protected by the update lock.
   *
   * @param clients
   * @param config
   */
private void checkQuarantineState(List<TrackerClientUpdater> clients, DegraderLoadBalancerStrategyConfig config) {
    Callback<None> healthCheckCallback = new Callback<None>() {

        @Override
        public void onError(Throwable e) {
            // Do nothing as the quarantine is disabled by default
            _rateLimitedLogger.error("Error to enable quarantine. Health checking failed for service {}: ", _state._serviceName, e);
        }

        @Override
        public void onSuccess(None result) {
            if (_state._enableQuarantine.compareAndSet(false, true)) {
                _log.info("Quarantine is enabled for service {}", _state._serviceName);
            }
        }
    };
    // Ideally we would like to healthchecking all the service hosts (ie all TrackerClients) because
    // this can help to warm up the R2 connections to the service hosts, thus speed up the initial access
    // speed when d2client starts to access those hosts. However this can expose/expedite the problem that
    // the d2client host needs too many connections or file handles to all the hosts, when the downstream
    // services have large amount of hosts. Before that problem is addressed, we limit the number of hosts
    // for pre-healthchecking to a small number
    clients.stream().limit(MAX_HOSTS_TO_CHECK_QUARANTINE).forEach(client -> {
        try {
            HealthCheck healthCheckClient = _state.getHealthCheckClient(client);
            if (healthCheckClient == null) {
                healthCheckClient = new HealthCheckClientBuilder().setHealthCheckOperations(config.getHealthCheckOperations()).setHealthCheckPath(config.getHealthCheckPath()).setServicePath(config.getServicePath()).setClock(config.getClock()).setLatency(config.getQuarantineLatency()).setMethod(config.getHealthCheckMethod()).setClient(client.getTrackerClient()).build();
                _state.putHealthCheckClient(client, healthCheckClient);
            }
            healthCheckClient.checkHealth(healthCheckCallback);
        } catch (URISyntaxException e) {
            _log.error("Error to build healthCheckClient ", e);
        }
    });
    // also remove the entries that the corresponding trackerClientUpdaters do not exist anymore
    for (TrackerClientUpdater client : _state._healthCheckMap.keySet()) {
        if (!clients.contains(client)) {
            _state._healthCheckMap.remove(client);
        }
    }
}

Example 3 with D2Client

use of com.linkedin.d2.balancer.D2Client in project rest.li by linkedin.

the class BigDataClientExample method main.

public static void main(String[] args) throws Exception {
    stats.put("normalClient", new AtomicInteger());
    stats.put("overrideClient", new AtomicInteger());
    //get client configuration
    JSONObject json = parseConfig();
    String zkConnectString = (String) json.get("zkConnectString");
    Long zkSessionTimeout = (Long) json.get("zkSessionTimeout");
    String zkBasePath = (String) json.get("zkBasePath");
    Long zkStartupTimeout = (Long) json.get("zkStartupTimeout");
    Long zkLoadBalancerNotificationTimeout = (Long) json.get("zkLoadBalancerNotificationTimeout");
    String zkFlagFile = (String) json.get("zkFlagFile");
    String fsBasePath = (String) json.get("fsBasePath");
    final Map<String, Long> trafficProportion = (Map<String, Long>) json.get("trafficProportion");
    Map<String, Map<String, Object>> clientServicesConfig = (Map<String, Map<String, Object>>) json.get("clientServicesConfig");
    final Long clientShutdownTimeout = (Long) json.get("clientShutdownTimeout");
    final Long clientStartTimeout = (Long) json.get("clientStartTimeout");
    System.out.println("Finished parsing client config");
    D2ClientBuilder builder = new D2ClientBuilder().setZkHosts(zkConnectString).setZkSessionTimeout(zkSessionTimeout, TimeUnit.MILLISECONDS).setZkStartupTimeout(zkStartupTimeout, TimeUnit.MILLISECONDS).setLbWaitTimeout(zkLoadBalancerNotificationTimeout, TimeUnit.MILLISECONDS).setFlagFile(zkFlagFile).setBasePath(zkBasePath).setFsBasePath(fsBasePath);
    final D2Client normalD2Client = builder.build();
    final D2Client overrideD2Client = builder.setClientServicesConfig(clientServicesConfig).build();
    System.out.println("Finished creating d2 clients, starting d2 clients...");
    ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
    final CountDownLatch latch = new CountDownLatch(2);
    //start both d2 clients
    startClients(normalD2Client, overrideD2Client, executorService, clientStartTimeout, new Callback<None>() {

        @Override
        public void onError(Throwable e) {
            System.exit(1);
        }

        @Override
        public void onSuccess(None result) {
            latch.countDown();
        }
    });
    latch.await();
    System.out.println("D2 clients are sending traffic to 'compute' service.");
    System.out.println("NormalClient will fail because the timeout is 5 seconds.");
    System.out.println("The server responds to our request in 10 seconds.");
    System.out.println("OverrideClient will succeed because we override the timeout to 10 seconds");
    ScheduledFuture normalTask = executorService.scheduleAtFixedRate(new Runnable() {

        @Override
        public void run() {
            try {
                sendTraffic(trafficProportion, normalD2Client, "Normal d2 client");
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }, 0, 1000, TimeUnit.MILLISECONDS);
    ScheduledFuture overrideTask = executorService.scheduleAtFixedRate(new Runnable() {

        @Override
        public void run() {
            try {
                sendTraffic(trafficProportion, overrideD2Client, "Override d2 client");
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }, 0, 1000, TimeUnit.MILLISECONDS);
    System.out.println("Press enter to shutdown");
    System.out.println("===========================================================\n\n");
    System.in.read();
    normalTask.cancel(false);
    overrideTask.cancel(false);
    System.out.println("Shutting down... please wait 15 seconds.");
    shutdown(normalD2Client, overrideD2Client, executorService, clientShutdownTimeout);
}

Example 4 with D2Client

use of com.linkedin.d2.balancer.D2Client in project rest.li by linkedin.

the class ExampleD2Client method main.

public static void main(String[] args) throws IOException, ParseException, InterruptedException {
    //get client configuration
    JSONObject json = parseConfig();
    String zkConnectString = (String) json.get("zkConnectString");
    Long zkSessionTimeout = (Long) json.get("zkSessionTimeout");
    String zkBasePath = (String) json.get("zkBasePath");
    Long zkStartupTimeout = (Long) json.get("zkStartupTimeout");
    Long zkLoadBalancerNotificationTimeout = (Long) json.get("zkLoadBalancerNotificationTimeout");
    String zkFlagFile = (String) json.get("zkFlagFile");
    String fsBasePath = (String) json.get("fsBasePath");
    final Map<String, Long> trafficProportion = (Map<String, Long>) json.get("trafficProportion");
    final Long clientShutdownTimeout = (Long) json.get("clientShutdownTimeout");
    final Long clientStartTimeout = (Long) json.get("clientStartTimeout");
    Long rate = (Long) json.get("rateMillisecond");
    System.out.println("Finished parsing client config");
    //create d2 client
    final D2Client d2Client = new D2ClientBuilder().setZkHosts(zkConnectString).setZkSessionTimeout(zkSessionTimeout, TimeUnit.MILLISECONDS).setZkStartupTimeout(zkStartupTimeout, TimeUnit.MILLISECONDS).setLbWaitTimeout(zkLoadBalancerNotificationTimeout, TimeUnit.MILLISECONDS).setFlagFile(zkFlagFile).setBasePath(zkBasePath).setFsBasePath(fsBasePath).build();
    System.out.println("Finished creating d2 client, starting d2 client...");
    ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();
    final CountDownLatch latch = new CountDownLatch(1);
    //start d2 client by connecting to zookeeper
    startClient(d2Client, executorService, clientStartTimeout, new Callback<None>() {

        @Override
        public void onError(Throwable e) {
            System.exit(1);
        }

        @Override
        public void onSuccess(None result) {
            latch.countDown();
        }
    });
    latch.await();
    System.out.println("D2 client is sending traffic");
    ScheduledFuture task = executorService.scheduleAtFixedRate(new Runnable() {

        @Override
        public void run() {
            try {
                sendTraffic(trafficProportion, d2Client);
            } catch (URISyntaxException e) {
                e.printStackTrace();
            }
        }
    }, 0, rate, TimeUnit.MILLISECONDS);
    System.out.println("Press enter to stop D2 client...");
    System.in.read();
    task.cancel(false);
    System.out.println("Shutting down...");
    shutdown(d2Client, executorService, clientShutdownTimeout);
}

Example 5 with D2Client

use of com.linkedin.d2.balancer.D2Client in project rest.li by linkedin.

the class RestLiExampleD2Client method main.

/**
   * This is a stand-alone app to demo the use of client-side Pegasus API. To run in,
   * com.linkedin.restli.example.RestLiExamplesServer has to be running.
   *
   * The only argument is the path to the resource on the photo server, e.g. /album/1
   */
public static void main(String[] args) throws Exception {
    final D2Client d2Client = new D2ClientBuilder().build();
    d2Client.start(new FutureCallback<None>());
    final RestClient restClient = new RestClient(d2Client, "d2://");
    final RestLiExampleBasicClient photoClient = new RestLiExampleBasicClient(restClient);
    photoClient.sendRequest(args[0], new PrintWriter(System.out));
    photoClient.shutdown();
}