Search in sources :

Example 1 with MockRMContext

use of org.apache.myriad.state.MockRMContext in project incubator-myriad by apache.

the class TestObjectFactory method getRMNode.

public static RMNode getRMNode(String host, int port, Resource resource) {
    NodeId id = NodeId.newInstance(host, port);
    RMContext context = new MockRMContext();
    return new RMNodeImpl(id, context, id.getHost(), id.getPort(), id.getPort(), new NodeBase(host, "/tmp"), resource, "version-one");
}
Also used : RMContext(org.apache.hadoop.yarn.server.resourcemanager.RMContext) MockRMContext(org.apache.myriad.state.MockRMContext) NodeBase(org.apache.hadoop.net.NodeBase) MockRMContext(org.apache.myriad.state.MockRMContext) NodeId(org.apache.hadoop.yarn.api.records.NodeId) RMNodeImpl(org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl)

Example 2 with MockRMContext

use of org.apache.myriad.state.MockRMContext in project incubator-myriad by apache.

the class MyriadOperationsTest method generateRMContext.

private RMContext generateRMContext(AbstractYarnScheduler<FiCaSchedulerApp, FiCaSchedulerNode> scheduler) throws Exception {
    Configuration conf = new Configuration();
    MockRMContext context = null;
    Dispatcher dispatcher = new MockDispatcher();
    RMApplicationHistoryWriter rmApplicationHistoryWriter = new RMApplicationHistoryWriter();
    AMLivelinessMonitor amLivelinessMonitor = new AMLivelinessMonitor(dispatcher);
    AMLivelinessMonitor amFinishingMonitor = new AMLivelinessMonitor(dispatcher);
    RMDelegationTokenSecretManager delegationTokenSecretManager = new RMDelegationTokenSecretManager(1, 1, 1, 1, context);
    context = new MockRMContext();
    context.setStateStore(TestObjectFactory.getStateStore(conf, "tmp/myriad-operations-test"));
    context.setAmLivelinessMonitor(amLivelinessMonitor);
    context.setAmFinishingMonitor(amFinishingMonitor);
    context.setRMApplicationHistoryWriter(rmApplicationHistoryWriter);
    context.setRMDelegationTokenSecretManager(delegationTokenSecretManager);
    return context;
}
Also used : MockRMContext(org.apache.myriad.state.MockRMContext) Configuration(org.apache.hadoop.conf.Configuration) RMDelegationTokenSecretManager(org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager) AMLivelinessMonitor(org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.AMLivelinessMonitor) MockDispatcher(org.apache.myriad.state.MockDispatcher) RMApplicationHistoryWriter(org.apache.hadoop.yarn.server.resourcemanager.ahs.RMApplicationHistoryWriter) MockDispatcher(org.apache.myriad.state.MockDispatcher) Dispatcher(org.apache.hadoop.yarn.event.Dispatcher)

Example 3 with MockRMContext

use of org.apache.myriad.state.MockRMContext in project incubator-myriad by apache.

the class NMHeartBeatHandlerTest method setUp.

@Override
@Before
public void setUp() throws Exception {
    super.setUp();
    this.baseStateStoreDirectory = "/tmp/nm-heartbeat-handler-test";
    context = new MockRMContext();
    context.setDispatcher(TestObjectFactory.getMockDispatcher());
    context.setSystemMetricsPublisher(new SystemMetricsPublisher());
    profileZero = TestObjectFactory.getServiceResourceProfile("zero", Double.valueOf(0.0), Double.valueOf(0.0), Long.valueOf(0), Long.valueOf(0));
    profileSmall = TestObjectFactory.getServiceResourceProfile("small", Double.valueOf(2.0), Double.valueOf(2048.0), Long.valueOf(1), Long.valueOf(1024));
    nodeOne = TestObjectFactory.getRMNode("localhost-one", 8800, Resource.newInstance(0, 0));
    nodeTwo = TestObjectFactory.getRMNode("localhost-two", 8800, Resource.newInstance(1024, 2));
    sNodeOne = new FSSchedulerNode(nodeOne, false);
    sNodeTwo = new FSSchedulerNode(nodeTwo, false);
    nodeTaskOne = TestObjectFactory.getNodeTask("localhost-one", profileZero);
    nodeTaskTwo = TestObjectFactory.getNodeTask("localhost-two", profileSmall);
    ConcurrentMap<NodeId, RMNode> rmNodes = new ConcurrentHashMap<NodeId, RMNode>();
    rmNodes.put(nodeOne.getNodeID(), nodeOne);
    rmNodes.put(nodeTwo.getNodeID(), nodeTwo);
    context.setRMNodes(rmNodes);
    store = new NodeStore();
    store.add(sNodeOne);
    store.add(sNodeTwo);
    MyriadDriver driver = TestObjectFactory.getMyriadDriver(new MockSchedulerDriver());
    olManager = new OfferLifecycleManager(store, driver);
    state = TestObjectFactory.getSchedulerState(new MyriadConfiguration(), "/tmp/nm-heartbeat-handler-test");
    state.addNodes(Lists.newArrayList(nodeTaskOne, nodeTaskTwo));
    MyriadFairScheduler scheduler = TestObjectFactory.getMyriadFairScheduler(context);
    scheduler.addNode(sNodeOne);
    scheduler.addNode(sNodeTwo);
    manager = new YarnNodeCapacityManager(new CompositeInterceptor(), scheduler, context, driver, olManager, store, state, new TaskUtils(this.cfg));
    handler = new NMHeartBeatHandler(new CompositeInterceptor(), scheduler, driver, manager, olManager, store, state, cfg.getNodeManagerConfiguration());
}
Also used : MyriadDriver(org.apache.myriad.scheduler.MyriadDriver) CompositeInterceptor(org.apache.myriad.scheduler.yarn.interceptor.CompositeInterceptor) TaskUtils(org.apache.myriad.scheduler.TaskUtils) MockRMContext(org.apache.myriad.state.MockRMContext) MyriadFairScheduler(org.apache.myriad.scheduler.yarn.MyriadFairScheduler) FSSchedulerNode(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode) RMNode(org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNode) MyriadConfiguration(org.apache.myriad.configuration.MyriadConfiguration) SystemMetricsPublisher(org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher) NodeId(org.apache.hadoop.yarn.api.records.NodeId) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) MockSchedulerDriver(org.apache.myriad.scheduler.MockSchedulerDriver) Before(org.junit.Before)

Example 4 with MockRMContext

use of org.apache.myriad.state.MockRMContext in project incubator-myriad by apache.

the class OfferLifeCycleManagerTest method setUp.

@Before
public void setUp() throws Exception {
    NodeStore store = new NodeStore();
    NodeIdProto nodeId = NodeIdProto.newBuilder().setHost("localhost").setPort(8000).build();
    RMNode rmNode = new RMNodeImpl(new NodeIdPBImpl(nodeId), new MockRMContext(), "localhost", 8000, 8070, new NodeBase(), new ResourcePBImpl(), "1.0");
    SchedulerNode node = new FiCaSchedulerNode(rmNode, false);
    store.add(node);
    manager = new OfferLifecycleManager(store, new MyriadDriver(new MockSchedulerDriver()));
}
Also used : MyriadDriver(org.apache.myriad.scheduler.MyriadDriver) MockRMContext(org.apache.myriad.state.MockRMContext) FiCaSchedulerNode(org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode) SchedulerNode(org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode) FiCaSchedulerNode(org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode) NodeIdPBImpl(org.apache.hadoop.yarn.api.records.impl.pb.NodeIdPBImpl) NodeIdProto(org.apache.hadoop.yarn.proto.YarnProtos.NodeIdProto) ResourcePBImpl(org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl) NodeBase(org.apache.hadoop.net.NodeBase) RMNode(org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNode) RMNodeImpl(org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl) MockSchedulerDriver(org.apache.myriad.scheduler.MockSchedulerDriver) Before(org.junit.Before)

Example 5 with MockRMContext

use of org.apache.myriad.state.MockRMContext in project incubator-myriad by apache.

the class YarnNodeCapacityManagerTest method setUp.

@Override
@Before
public void setUp() throws Exception {
    super.setUp();
    this.baseStateStoreDirectory = "/tmp/yarn-node-capacity-manager-test";
    context = new MockRMContext();
    context.setDispatcher(TestObjectFactory.getMockDispatcher());
    context.setSystemMetricsPublisher(new SystemMetricsPublisher());
    nodeOne = TestObjectFactory.getRMNode("localhost-one", 8800, Resource.newInstance(2048, 4));
    nodeTwo = TestObjectFactory.getRMNode("localhost-two", 8800, Resource.newInstance(1024, 2));
    sNodeOne = new FSSchedulerNode(nodeOne, false);
    sNodeTwo = new FSSchedulerNode(nodeTwo, false);
    containerOne = TestObjectFactory.getRMContainer(nodeOne, context, 1, 2, 1024);
    store = new NodeStore();
    store.add(sNodeOne);
    store.add(sNodeTwo);
    MyriadDriver driver = TestObjectFactory.getMyriadDriver(new MockSchedulerDriver());
    olManager = new OfferLifecycleManager(store, driver);
    state = TestObjectFactory.getSchedulerState(new MyriadConfiguration(), "/tmp/yarn-node-capacity-manager-test");
    MyriadFairScheduler scheduler = TestObjectFactory.getMyriadFairScheduler(context);
    scheduler.addNode(sNodeOne);
    scheduler.addNode(sNodeTwo);
    manager = new YarnNodeCapacityManager(new CompositeInterceptor(), scheduler, context, driver, olManager, store, state, new TaskUtils(this.cfg));
}
Also used : MyriadDriver(org.apache.myriad.scheduler.MyriadDriver) CompositeInterceptor(org.apache.myriad.scheduler.yarn.interceptor.CompositeInterceptor) TaskUtils(org.apache.myriad.scheduler.TaskUtils) MockRMContext(org.apache.myriad.state.MockRMContext) FSSchedulerNode(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode) MyriadConfiguration(org.apache.myriad.configuration.MyriadConfiguration) MyriadFairScheduler(org.apache.myriad.scheduler.yarn.MyriadFairScheduler) SystemMetricsPublisher(org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher) MockSchedulerDriver(org.apache.myriad.scheduler.MockSchedulerDriver) Before(org.junit.Before)

Aggregations

MockRMContext (org.apache.myriad.state.MockRMContext)6 Before (org.junit.Before)4 SystemMetricsPublisher (org.apache.hadoop.yarn.server.resourcemanager.metrics.SystemMetricsPublisher)3 FSSchedulerNode (org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode)3 MockSchedulerDriver (org.apache.myriad.scheduler.MockSchedulerDriver)3 MyriadDriver (org.apache.myriad.scheduler.MyriadDriver)3 NodeBase (org.apache.hadoop.net.NodeBase)2 NodeId (org.apache.hadoop.yarn.api.records.NodeId)2 RMNode (org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNode)2 RMNodeImpl (org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl)2 MyriadConfiguration (org.apache.myriad.configuration.MyriadConfiguration)2 TaskUtils (org.apache.myriad.scheduler.TaskUtils)2 MyriadFairScheduler (org.apache.myriad.scheduler.yarn.MyriadFairScheduler)2 CompositeInterceptor (org.apache.myriad.scheduler.yarn.interceptor.CompositeInterceptor)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 Configuration (org.apache.hadoop.conf.Configuration)1 NodeIdPBImpl (org.apache.hadoop.yarn.api.records.impl.pb.NodeIdPBImpl)1 ResourcePBImpl (org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl)1 Dispatcher (org.apache.hadoop.yarn.event.Dispatcher)1 NodeIdProto (org.apache.hadoop.yarn.proto.YarnProtos.NodeIdProto)1