Search in sources :

Example 41 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class SubmitterMainTest method testValidateSubmitAlreadyRunning.

@Test(expected = TopologySubmissionException.class)
public void testValidateSubmitAlreadyRunning() throws Exception {
    SubmitterMain submitterMain = new SubmitterMain(config, topology);
    SchedulerStateManagerAdaptor adaptor = mock(SchedulerStateManagerAdaptor.class);
    // Topology is running
    when(adaptor.isTopologyRunning(eq(TOPOLOGY_NAME))).thenReturn(true);
    submitterMain.validateSubmit(adaptor, TOPOLOGY_NAME);
}
Also used : SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 42 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class RuntimeManagerRunner method updateTopologyContainerCount.

@VisibleForTesting
void updateTopologyContainerCount(String topologyName, String newContainerNumber, String newParallelism) throws PackingException, UpdateDryRunResponse {
    LOG.fine(String.format("updateTopologyHandler called for %s with %s and %s", topologyName, newContainerNumber, newParallelism));
    Integer containerNum = Integer.parseInt(newContainerNumber);
    Map<String, Integer> changeRequests = new HashMap<String, Integer>();
    if (newParallelism != null && !newParallelism.isEmpty()) {
        changeRequests = parseNewParallelismParam(newParallelism);
    }
    SchedulerStateManagerAdaptor manager = Runtime.schedulerStateManagerAdaptor(runtime);
    TopologyAPI.Topology topology = manager.getTopology(topologyName);
    PackingPlans.PackingPlan currentPlan = manager.getPackingPlan(topologyName);
    if (!containersNumChangeDetected(currentPlan, containerNum) && !parallelismChangeDetected(currentPlan, changeRequests)) {
        throw new TopologyRuntimeManagementException(String.format("Both component parallelism request and container number are the " + "same as in the running topology."));
    }
    PackingPlans.PackingPlan proposedPlan = buildNewPackingPlan(currentPlan, changeRequests, containerNum, topology);
    sendUpdateRequest(topology, changeRequests, currentPlan, proposedPlan);
}
Also used : PackingPlans(org.apache.heron.proto.system.PackingPlans) HashMap(java.util.HashMap) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) TopologyAPI(org.apache.heron.api.generated.TopologyAPI) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 43 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class SchedulerClientFactoryTest method testGetServiceSchedulerClientOk.

@Test
public void testGetServiceSchedulerClientOk() {
    // Instantiate mock objects
    Config config = Mockito.mock(Config.class);
    Config runtime = Mockito.mock(Config.class);
    SchedulerStateManagerAdaptor statemgr = Mockito.mock(SchedulerStateManagerAdaptor.class);
    // Get a ServiceSchedulerClient
    Mockito.when(config.getBooleanValue(Key.SCHEDULER_IS_SERVICE)).thenReturn(true);
    // Mock the runtime object
    Mockito.when(runtime.get(Key.SCHEDULER_STATE_MANAGER_ADAPTOR)).thenReturn(statemgr);
    Mockito.when(runtime.getStringValue(Key.TOPOLOGY_NAME)).thenReturn(TOPOLOGY_NAME);
    // Get a schedulerLocation successfully
    Mockito.when(statemgr.getSchedulerLocation(Mockito.eq(TOPOLOGY_NAME))).thenReturn(Scheduler.SchedulerLocation.getDefaultInstance());
    Assert.assertNotNull(new SchedulerClientFactory(config, runtime).getSchedulerClient());
}
Also used : Config(org.apache.heron.spi.common.Config) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) Test(org.junit.Test) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest)

Example 44 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class RuntimeManagerRunner method getCurrentContainerNumber.

private int getCurrentContainerNumber(String topologyName) {
    SchedulerStateManagerAdaptor manager = Runtime.schedulerStateManagerAdaptor(runtime);
    PackingPlans.PackingPlan currentPlan = manager.getPackingPlan(topologyName);
    PackingPlanProtoDeserializer deserializer = new PackingPlanProtoDeserializer();
    PackingPlan cPlan = deserializer.fromProto(currentPlan);
    return cPlan.getContainers().size();
}
Also used : PackingPlans(org.apache.heron.proto.system.PackingPlans) PackingPlanProtoDeserializer(org.apache.heron.spi.packing.PackingPlanProtoDeserializer) PackingPlan(org.apache.heron.spi.packing.PackingPlan) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor)

Example 45 with SchedulerStateManagerAdaptor

use of org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor in project heron by twitter.

the class UpdateTopologyManagerTest method requestsToAddAndRemoveContainers.

/**
 * Test scalable scheduler invocation
 */
@Test
@PrepareForTest(TManagerUtils.class)
public void requestsToAddAndRemoveContainers() throws Exception {
    Lock lock = mockLock(true);
    SchedulerStateManagerAdaptor mockStateMgr = mockStateManager(testTopology, this.currentProtoPlan, lock);
    IScalable mockScheduler = mock(IScalable.class);
    HashSet<PackingPlan.ContainerPlan> mockRetrunSet = new HashSet<>();
    mockRetrunSet.add(new PackingPlan.ContainerPlan(0, new HashSet<>(), new Resource(5, ByteAmount.ZERO, ByteAmount.ZERO)));
    mockRetrunSet.add(new PackingPlan.ContainerPlan(1, new HashSet<>(), new Resource(6, ByteAmount.ZERO, ByteAmount.ZERO)));
    when(mockScheduler.addContainers(any())).thenReturn(mockRetrunSet);
    UpdateTopologyManager spyUpdateManager = spyUpdateManager(mockStateMgr, mockScheduler, testTopology);
    PowerMockito.spy(TManagerUtils.class);
    PowerMockito.doNothing().when(TManagerUtils.class, "sendToTManager", any(String.class), eq(TOPOLOGY_NAME), eq(mockStateMgr), any(NetworkUtils.TunnelConfig.class));
    // reactivation won't happen since topology state is still running due to mock state manager
    PowerMockito.doNothing().when(TManagerUtils.class, "transitionTopologyState", eq(TOPOLOGY_NAME), eq(TManagerUtils.TManagerCommand.ACTIVATE), eq(mockStateMgr), eq(TopologyAPI.TopologyState.PAUSED), eq(TopologyAPI.TopologyState.RUNNING), any(NetworkUtils.TunnelConfig.class));
    spyUpdateManager.updateTopology(currentProtoPlan, proposedProtoPlan);
    verify(spyUpdateManager).deactivateTopology(eq(mockStateMgr), eq(testTopology), eq(proposedPacking));
    verify(spyUpdateManager).reactivateTopology(eq(mockStateMgr), eq(testTopology), eq(2));
    verify(mockScheduler).addContainers(expectedContainersToAdd);
    verify(mockScheduler).removeContainers(expectedContainersToRemove);
    verify(lock).tryLock(any(Long.class), any(TimeUnit.class));
    verify(lock).unlock();
    PowerMockito.verifyStatic(times(1));
    TManagerUtils.transitionTopologyState(eq(TOPOLOGY_NAME), eq(TManagerUtils.TManagerCommand.DEACTIVATE), eq(mockStateMgr), eq(TopologyAPI.TopologyState.RUNNING), eq(TopologyAPI.TopologyState.PAUSED), any(NetworkUtils.TunnelConfig.class));
    PowerMockito.verifyStatic(times(1));
    TManagerUtils.transitionTopologyState(eq(TOPOLOGY_NAME), eq(TManagerUtils.TManagerCommand.ACTIVATE), eq(mockStateMgr), eq(TopologyAPI.TopologyState.PAUSED), eq(TopologyAPI.TopologyState.RUNNING), any(NetworkUtils.TunnelConfig.class));
}
Also used : PackingPlan(org.apache.heron.spi.packing.PackingPlan) Resource(org.apache.heron.spi.packing.Resource) IScalable(org.apache.heron.spi.scheduler.IScalable) Lock(org.apache.heron.spi.statemgr.Lock) SchedulerStateManagerAdaptor(org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor) TimeUnit(java.util.concurrent.TimeUnit) HashSet(java.util.HashSet) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest)

Aggregations

SchedulerStateManagerAdaptor (org.apache.heron.spi.statemgr.SchedulerStateManagerAdaptor)46 Test (org.junit.Test)28 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)24 PackingPlan (org.apache.heron.spi.packing.PackingPlan)15 Config (org.apache.heron.spi.common.Config)14 PackingPlans (org.apache.heron.proto.system.PackingPlans)10 ISchedulerClient (org.apache.heron.scheduler.client.ISchedulerClient)8 Scheduler (org.apache.heron.proto.scheduler.Scheduler)7 ILauncher (org.apache.heron.spi.scheduler.ILauncher)6 HashSet (java.util.HashSet)5 TopologyAPI (org.apache.heron.api.generated.TopologyAPI)5 ExecutionEnvironment (org.apache.heron.proto.system.ExecutionEnvironment)5 IStateManager (org.apache.heron.spi.statemgr.IStateManager)5 VisibleForTesting (com.google.common.annotations.VisibleForTesting)4 HeronTopology (org.apache.heron.api.HeronTopology)4 RoundRobinPacking (org.apache.heron.packing.roundrobin.RoundRobinPacking)4 IScheduler (org.apache.heron.spi.scheduler.IScheduler)3 HashMap (java.util.HashMap)2 LauncherUtils (org.apache.heron.scheduler.utils.LauncherUtils)2 PackingException (org.apache.heron.spi.packing.PackingException)2