Search in sources :

Example 26 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class SchedulerUtils method setSchedulerLocation.

/**
 * Set the location of scheduler for other processes to discover
 *
 * @param runtime the runtime configuration
 * @param schedulerEndpoint the endpoint that scheduler listens for receives requests
 * @param scheduler the IScheduler to provide more info
 */
public static boolean setSchedulerLocation(Config runtime, String schedulerEndpoint, IScheduler scheduler) {
    // Set scheduler location to host:port by default. Overwrite scheduler location if behind DNS.
    Scheduler.SchedulerLocation.Builder builder = Scheduler.SchedulerLocation.newBuilder().setTopologyName(Runtime.topologyName(runtime)).setHttpEndpoint(schedulerEndpoint);
    // Set the job link in SchedulerLocation if any
    List<String> jobLinks = scheduler.getJobLinks();
    // Check whether IScheduler provides valid job link
    if (jobLinks != null) {
        builder.addAllJobPageLink(jobLinks);
    }
    Scheduler.SchedulerLocation location = builder.build();
    LOG.log(Level.INFO, "Setting Scheduler locations: {0}", location);
    SchedulerStateManagerAdaptor statemgr = Runtime.schedulerStateManagerAdaptor(runtime);
    Boolean result = statemgr.setSchedulerLocation(location, Runtime.topologyName(runtime));
    if (result == null || !result) {
        LOG.severe("Failed to set Scheduler location");
        return false;
    }
    return true;
}
Also used : Scheduler(org.apache.heron.proto.scheduler.Scheduler) IScheduler(org.apache.heron.spi.scheduler.IScheduler) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor)

Example 27 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class HealthManager method createStateMgrAdaptor.

@VisibleForTesting
SchedulerStateManagerAdaptor createStateMgrAdaptor() throws ReflectiveOperationException {
    String stateMgrClass = Context.stateManagerClass(config);
    IStateManager stateMgr = ReflectionUtils.newInstance(stateMgrClass);
    stateMgr.initialize(config);
    return new SchedulerStateManagerAdaptor(stateMgr, 5000);
}
Also used : IStateManager(org.apache.heron.spi.statemgr.IStateManager) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 28 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class PackingPlanProviderTest method refreshesPackingPlanOnUpdate.

@Test
public void refreshesPackingPlanOnUpdate() {
    PackingPlans.PackingPlan proto = PackingTestUtils.testProtoPackingPlan(topologyName, new RoundRobinPacking());
    SchedulerStateManagerAdaptor adaptor = mock(SchedulerStateManagerAdaptor.class);
    when(adaptor.getPackingPlan(topologyName)).thenReturn(proto);
    PackingPlanProvider provider = new PackingPlanProvider(adaptor, eventManager, topologyName);
    PackingPlan packing = provider.get();
    Assert.assertEquals(1, packing.getContainers().size());
    provider.onEvent(new TopologyUpdate(null, null));
    provider.get();
    verify(adaptor, times(2)).getPackingPlan(topologyName);
}
Also used : PackingPlans(org.apache.heron.proto.system.PackingPlans) RoundRobinPacking(org.apache.heron.packing.roundrobin.RoundRobinPacking) PackingPlan(org.apache.heron.spi.packing.PackingPlan) TopologyUpdate(org.apache.heron.healthmgr.common.HealthManagerEvents.TopologyUpdate) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) Test(org.junit.Test)

Example 29 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class PackingPlanProviderTest method providesBoltInstanceNames.

@Test
public void providesBoltInstanceNames() {
    PackingPlans.PackingPlan proto = PackingTestUtils.testProtoPackingPlan(topologyName, new RoundRobinPacking());
    SchedulerStateManagerAdaptor adaptor = mock(SchedulerStateManagerAdaptor.class);
    when(adaptor.getPackingPlan(topologyName)).thenReturn(proto);
    PackingPlanProvider packing = new PackingPlanProvider(adaptor, eventManager, topologyName);
    String[] boltNames = packing.getBoltInstanceNames("testBolt");
    assertEquals(3, boltNames.length);
    HashSet<String> expectedBoltNames = new HashSet<>();
    expectedBoltNames.add("container_1_testBolt_3");
    expectedBoltNames.add("container_1_testBolt_4");
    expectedBoltNames.add("container_1_testBolt_5");
    for (String name : boltNames) {
        expectedBoltNames.remove(name);
    }
    assertEquals(0, expectedBoltNames.size());
}
Also used : PackingPlans(org.apache.heron.proto.system.PackingPlans) RoundRobinPacking(org.apache.heron.packing.roundrobin.RoundRobinPacking) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 30 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class MetricsCacheMetricsProviderTest method createMetricsProviderSpy.

private MetricsCacheMetricsProvider createMetricsProviderSpy() {
    MetricsCacheLocation location = MetricsCacheLocation.newBuilder().setTopologyName("testTopo").setTopologyId("topoId").setHost("localhost").setControllerPort(0).setServerPort(0).build();
    SchedulerStateManagerAdaptor stateMgr = Mockito.mock(SchedulerStateManagerAdaptor.class);
    when(stateMgr.getMetricsCacheLocation("testTopo")).thenReturn(location);
    MetricsCacheMetricsProvider metricsProvider = new MetricsCacheMetricsProvider(stateMgr, "testTopo");
    return spy(metricsProvider);
}
Also used : MetricsCacheLocation(org.apache.heron.proto.tmanager.TopologyManager.MetricsCacheLocation) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor)

Aggregations

SchedulerStateManagerAdaptor (org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor)46 Test (org.junit.Test)28 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)24 PackingPlan (org.apache.heron.spi.packing.PackingPlan)15 Config (org.apache.heron.spi.common.Config)14 PackingPlans (org.apache.heron.proto.system.PackingPlans)10 ISchedulerClient (org.apache.heron.scheduler.client.ISchedulerClient)8 Scheduler (org.apache.heron.proto.scheduler.Scheduler)7 ILauncher (org.apache.heron.spi.scheduler.ILauncher)6 HashSet (java.util.HashSet)5 TopologyAPI (org.apache.heron.api.generated.TopologyAPI)5 ExecutionEnvironment (org.apache.heron.proto.system.ExecutionEnvironment)5 IStateManager (org.apache.heron.spi.statemgr.IStateManager)5 VisibleForTesting (com.google.common.annotations.VisibleForTesting)4 HeronTopology (org.apache.heron.api.HeronTopology)4 RoundRobinPacking (org.apache.heron.packing.roundrobin.RoundRobinPacking)4 IScheduler (org.apache.heron.spi.scheduler.IScheduler)3 HashMap (java.util.HashMap)2 LauncherUtils (org.apache.heron.scheduler.utils.LauncherUtils)2 PackingException (org.apache.heron.spi.packing.PackingException)2