Examples with D2Monitor com.linkedin.d2.balancer.event.D2Monitor used on opensource projects

Search in sources :

Example 1 with D2Monitor

use of com.linkedin.d2.balancer.event.D2Monitor in project rest.li by linkedin.

the class SimpleLoadBalancerDelayTest method loadBalancerD2MonitorWithQuarantineTest.

@Test(groups = { "small", "back-end" })
public void loadBalancerD2MonitorWithQuarantineTest() throws Exception {
    String uri1 = "test.qa1.com:1234";
    String uri2 = "test.qa2.com:2345";
    List<String> uris = Arrays.asList(uri1, uri2);
    URI uriU1 = new URI(uri1);
    URI uriU2 = new URI(uri2);
    // Construct the delay patterns: for each URI there is a list of delays for each interval
    Map<String, List<Long>> delayMaps = new HashMap<>();
    delayMaps.put("test.qa1.com:1234", Arrays.asList(80l, 30l, 30l, 30l, 30l, 80l, 30l, 30l, 30l, 30l, 30l, 80l, 60l, 80l, 80l, 60l, 80l, 80l, 60l, 80l, 80l));
    delayMaps.put("test.qa2.com:2345", Arrays.asList(80l, 80l, 30l, 30l, 30l, 80l, 30l, 30l, 3060l, 4080l, 3050l, 3080l, 4080l, 4080l, 4080l, 4080l, 60l, 80l, 60l, 80l, 80l));
    LoadBalancerSimulator.TimedValueGenerator<String, Long> delayGenerator = new DelayValueGenerator<>(delayMaps, DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    Map<String, Object> strategyProperties = DegraderLoadBalancerTest.lbDefaultConfig();
    // setting the event emitting interval to 40s
    strategyProperties.put(PropertyKeys.HTTP_LB_HIGH_EVENT_EMITTING_INTERVAL, "40000");
    strategyProperties.put(PropertyKeys.HTTP_LB_QUARANTINE_MAX_PERCENT, 0.05);
    strategyProperties.put(PropertyKeys.HTTP_LB_HASH_CONFIG, HASH_CONFIG_MAP);
    // Construct the QPS generator
    LoadBalancerSimulator.QPSGenerator qpsGenerator = new ConstantQPSGenerator(2000);
    // Create the simulator
    LoadBalancerSimulator loadBalancerSimulator = LoadBalancerSimulationBuilder.build("cluster-1", "foo", uris, strategyProperties, null, null, delayGenerator, qpsGenerator);
    URI expectedUri1 = LoadBalancerSimulationBuilder.getExpectedUri("test.qa1.com:1234", "foo");
    // Start the simulation
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    printStates(loadBalancerSimulator);
    // the points for uri1 should be 100
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2), 100);
    List<D2Monitor> d2Monitors = _d2MonitorMap.get("foo");
    // For healthy state, there is no emission yet.
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 2);
    d2Monitors = _d2MonitorMap.get("foo");
    // For healthy state, there is no emission yet.
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    printStates(loadBalancerSimulator);
    // wait for 3 intervals due to call dropping involved
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 6);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors != null);
    // the first emitting
    assertFalse(d2Monitors.isEmpty());
    D2Monitor d2Monitor = d2Monitors.get(0);
    assertTrue(d2Monitor.getClusterStats().getClusterCallCount() > 0);
    assertTrue(d2Monitor.getClusterStats().getClusterDropLevel() < 0.00001);
    assertEquals(d2Monitor.getIntervalMs(), 40000);
    printStates(loadBalancerSimulator);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2), 100);
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 8);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertTrue(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2) < 10);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors != null);
    // the first emitting
    assertFalse(d2Monitors.isEmpty());
    d2Monitor = d2Monitors.get(0);
    assertTrue(d2Monitor.getClusterStats().getClusterCallCount() > 0);
    List<D2Monitor.UriInfo> uriList = d2Monitor.getUriList();
    assertFalse(uriList.isEmpty());
    D2Monitor.UriInfo goodUri = uriList.get(0).getCurrentAvgLatency() < uriList.get(0).getCurrentAvgLatency() ? uriList.get(0) : uriList.get(1);
    D2Monitor.UriInfo badUri = uriList.get(0).getCurrentAvgLatency() >= uriList.get(0).getCurrentAvgLatency() ? uriList.get(0) : uriList.get(1);
    assertTrue(goodUri.getCurrentAvgLatency() <= 80);
    assertTrue(goodUri.getCurrentCallCount() > 1900);
    assertTrue(badUri.getCurrentCallCount() == 0);
    assertTrue(badUri.getQuarantineDuration() > 0);
    assertEquals(d2Monitor.getIntervalMs(), 40000);
    printStates(loadBalancerSimulator);
    // Done. Shutdown the simulation
    loadBalancerSimulator.shutdown();
}
Also used : HashMap(java.util.HashMap) URI(java.net.URI) ArrayList(java.util.ArrayList) List(java.util.List) D2Monitor(com.linkedin.d2.balancer.event.D2Monitor) Test(org.testng.annotations.Test) DegraderLoadBalancerTest(com.linkedin.d2.balancer.strategies.degrader.DegraderLoadBalancerTest)

Example 2 with D2Monitor

use of com.linkedin.d2.balancer.event.D2Monitor in project rest.li by linkedin.

the class SimpleLoadBalancerDelayTest method loadBalancerD2MonitorTest.

/**
 * Simple test to verify d2Monitor emitting
 * @throws Exception
 */
@Test(groups = { "small", "back-end" })
public void loadBalancerD2MonitorTest() throws Exception {
    String uri1 = "test.qa1.com:1234";
    String uri2 = "test.qa2.com:2345";
    List<String> uris = Arrays.asList(uri1, uri2);
    URI uriU1 = new URI(uri1);
    URI uriU2 = new URI(uri2);
    // Construct the delay patterns: for each URI there is a list of delays for each interval
    Map<String, List<Long>> delayMaps = new HashMap<>();
    delayMaps.put("test.qa1.com:1234", Arrays.asList(80l, 30l, 30l, 30l, 30l, 80l, 30l, 30l, 30l, 30l, 30l, 80l, 60l, 80l, 80l, 60l, 80l, 80l, 60l, 80l, 80l));
    delayMaps.put("test.qa2.com:2345", Arrays.asList(80l, 80l, 30l, 30l, 30l, 80l, 30l, 30l, 3060l, 4080l, 3050l, 3080l, 80l, 80l, 60l, 80l, 60l, 80l, 60l, 80l, 80l));
    LoadBalancerSimulator.TimedValueGenerator<String, Long> delayGenerator = new DelayValueGenerator<>(delayMaps, DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    Map<String, Object> strategyProperties = DegraderLoadBalancerTest.lbDefaultConfig();
    // setting the event emitting interval to 40s
    strategyProperties.put(PropertyKeys.HTTP_LB_HIGH_EVENT_EMITTING_INTERVAL, "40000");
    strategyProperties.put(PropertyKeys.HTTP_LB_HASH_CONFIG, HASH_CONFIG_MAP);
    // Construct the QPS generator
    LoadBalancerSimulator.QPSGenerator qpsGenerator = new ConstantQPSGenerator(2000);
    // Create the simulator
    LoadBalancerSimulator loadBalancerSimulator = LoadBalancerSimulationBuilder.build("cluster-1", "foo", uris, strategyProperties, null, null, delayGenerator, qpsGenerator);
    URI expectedUri1 = LoadBalancerSimulationBuilder.getExpectedUri("test.qa1.com:1234", "foo");
    // Start the simulation
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    printStates(loadBalancerSimulator);
    // the points for uri1 should be 100
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2), 100);
    List<D2Monitor> d2Monitors = _d2MonitorMap.get("foo");
    // For healthy state, there is no emission yet.
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 2);
    d2Monitors = _d2MonitorMap.get("foo");
    // For healthy state, there is no emission yet.
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    printStates(loadBalancerSimulator);
    // wait for 3 intervals due to call dropping involved
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 6);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors != null);
    // the first emitting
    assertFalse(d2Monitors.isEmpty());
    D2Monitor d2Monitor = d2Monitors.get(0);
    assertTrue(d2Monitor.getClusterStats().getClusterCallCount() > 0);
    assertTrue(d2Monitor.getClusterStats().getClusterDropLevel() < 0.00001);
    List<D2Monitor.UriInfo> uriList = d2Monitor.getUriList();
    assertFalse(uriList.isEmpty());
    assertTrue(uriList.get(0).getCurrentAvgLatency() - 50 < 0.0001);
    assertTrue(uriList.get(0).getCurrentCallCount() > 900);
    assertTrue(uriList.get(1).getCurrentAvgLatency() - 30 < 0.0001);
    assertTrue(uriList.get(1).getCurrentCallCount() > 900);
    assertEquals(d2Monitor.getIntervalMs(), 40000);
    printStates(loadBalancerSimulator);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2), 100);
    // continue the simulation
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertTrue(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2) < 100);
    d2Monitors = _d2MonitorMap.get("foo");
    // There's degrading, but no emitting yet
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    printStates(loadBalancerSimulator);
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertTrue(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2) < 100);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    printStates(loadBalancerSimulator);
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 2);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertTrue(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2) < 10);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors == null || d2Monitors.isEmpty());
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 5);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors != null);
    assertFalse(d2Monitors.isEmpty());
    loadBalancerSimulator.runWait(DegraderLoadBalancerStrategyConfig.DEFAULT_UPDATE_INTERVAL_MS * 8);
    // uri1 should fully recovered by now
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri1), 100);
    assertEquals(loadBalancerSimulator.getPoint("foo", DefaultPartitionAccessor.DEFAULT_PARTITION_ID, uri2), 100);
    d2Monitors = _d2MonitorMap.get("foo");
    assertTrue(d2Monitors != null);
    assertFalse(_d2MonitorMap.get("foo").isEmpty());
    printStates(loadBalancerSimulator);
    // Done. Shutdown the simulation
    loadBalancerSimulator.shutdown();
}
Also used : HashMap(java.util.HashMap) URI(java.net.URI) ArrayList(java.util.ArrayList) List(java.util.List) D2Monitor(com.linkedin.d2.balancer.event.D2Monitor) Test(org.testng.annotations.Test) DegraderLoadBalancerTest(com.linkedin.d2.balancer.strategies.degrader.DegraderLoadBalancerTest)

Aggregations

D2Monitor (com.linkedin.d2.balancer.event.D2Monitor)2 DegraderLoadBalancerTest (com.linkedin.d2.balancer.strategies.degrader.DegraderLoadBalancerTest)2 URI (java.net.URI)2 ArrayList (java.util.ArrayList)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Test (org.testng.annotations.Test)2
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial