Search in sources :

Example 1 with ServiceResourceProfile

use of org.apache.myriad.scheduler.ServiceResourceProfile in project incubator-myriad by apache.

the class Main method initProfiles.

private void initProfiles(Injector injector) {
    LOGGER.info("Initializing Profiles");
    ServiceProfileManager profileManager = injector.getInstance(ServiceProfileManager.class);
    Map<String, Map<String, String>> profiles = injector.getInstance(MyriadConfiguration.class).getProfiles();
    TaskUtils taskUtils = injector.getInstance(TaskUtils.class);
    if (MapUtils.isNotEmpty(profiles)) {
        for (Map.Entry<String, Map<String, String>> profile : profiles.entrySet()) {
            Map<String, String> profileResourceMap = profile.getValue();
            if (MapUtils.isNotEmpty(profiles) && profileResourceMap.containsKey("cpu") && profileResourceMap.containsKey("mem")) {
                Long cpu = Long.parseLong(profileResourceMap.get("cpu"));
                Long mem = Long.parseLong(profileResourceMap.get("mem"));
                ServiceResourceProfile serviceProfile = new ExtendedResourceProfile(new NMProfile(profile.getKey(), cpu, mem), taskUtils.getNodeManagerCpus(), taskUtils.getNodeManagerMemory(), taskUtils.getNodeManagerPorts());
                profileManager.add(serviceProfile);
            } else {
                LOGGER.error("Invalid definition for profile: " + profile.getKey());
            }
        }
    }
}
Also used : ServiceProfileManager(org.apache.myriad.scheduler.ServiceProfileManager) ExtendedResourceProfile(org.apache.myriad.scheduler.ExtendedResourceProfile) TaskUtils(org.apache.myriad.scheduler.TaskUtils) NMProfile(org.apache.myriad.scheduler.NMProfile) MyriadConfiguration(org.apache.myriad.configuration.MyriadConfiguration) ServiceResourceProfile(org.apache.myriad.scheduler.ServiceResourceProfile) Map(java.util.Map)

Example 2 with ServiceResourceProfile

use of org.apache.myriad.scheduler.ServiceResourceProfile in project incubator-myriad by apache.

the class SchedulerStateResourceTest method getSchedulerState.

private SchedulerState getSchedulerState() throws Exception {
    SchedulerState state = new SchedulerState(new MyriadFileSystemRMStateStore());
    idOne = Protos.TaskID.newBuilder().setValue("nt-1").build();
    idTwo = Protos.TaskID.newBuilder().setValue("nt-2").build();
    idThree = Protos.TaskID.newBuilder().setValue("nt-3").build();
    TreeMap<String, Long> ports = new TreeMap<>();
    state.addTask(idOne, new NodeTask(new ServiceResourceProfile("profile1", 0.2, 1024.0, ports), new LikeConstraint("localhost", "host-[0-9]*.example.com")));
    state.addTask(idTwo, new NodeTask(new ServiceResourceProfile("profile2", 0.4, 2048.0, ports), new LikeConstraint("localhost", "host-[0-9]*.example.com")));
    state.addTask(idThree, new NodeTask(new ServiceResourceProfile("profile3", 0.6, 3072.0, ports), new LikeConstraint("localhost", "host-[0-9]*.example.com")));
    state.setFrameworkId(FrameworkID.newBuilder().setValue("mock-framework").build());
    state.makeTaskActive(idOne);
    state.makeTaskPending(idTwo);
    state.makeTaskStaging(idThree);
    return state;
}
Also used : SchedulerState(org.apache.myriad.state.SchedulerState) MyriadFileSystemRMStateStore(org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore) ServiceResourceProfile(org.apache.myriad.scheduler.ServiceResourceProfile) TreeMap(java.util.TreeMap) NodeTask(org.apache.myriad.state.NodeTask) LikeConstraint(org.apache.myriad.scheduler.constraints.LikeConstraint)

Example 3 with ServiceResourceProfile

use of org.apache.myriad.scheduler.ServiceResourceProfile in project incubator-myriad by apache.

the class NodeTaskTest method setUp.

@Before
public void setUp() throws Exception {
    TreeMap<String, Long> ports = new TreeMap<>();
    task = new NodeTask(new ServiceResourceProfile("profile", 0.1, 1024.0, ports), new LikeConstraint("hostname", "host-[0-9]*.example.com"));
    task.setHostname("localhost");
    task.setTaskPrefix("prefix");
    task.setProfile(new ServiceResourceProfile("ServiceResourceProfile", 0.1, 1024.0, ports));
}
Also used : ServiceResourceProfile(org.apache.myriad.scheduler.ServiceResourceProfile) TreeMap(java.util.TreeMap) LikeConstraint(org.apache.myriad.scheduler.constraints.LikeConstraint) Before(org.junit.Before)

Example 4 with ServiceResourceProfile

use of org.apache.myriad.scheduler.ServiceResourceProfile in project incubator-myriad by apache.

the class TestResourceOfferContainer method testResourceOfferContainerForAuxServiceWithOutRole.

@Test
public void testResourceOfferContainerForAuxServiceWithOutRole() {
    Protos.Offer offer = new OfferBuilder("test.com").addScalarResource("cpus", 2.0).addScalarResource("mem", 8000).addRangeResource("ports", 3500, 3600).build();
    Map<String, Long> ports = new HashMap<>(4);
    ports.put("test1.address", 0L);
    ports.put("test2.address", 0L);
    ports.put("test3.address", 0L);
    ports.put("test4.port", 3501L);
    ServiceResourceProfile profile = new ServiceResourceProfile("jobhistory", 2.0, 8000.0, ports);
    ResourceOfferContainer roc = new ResourceOfferContainer(offer, profile, null);
    assertTrue(roc.getHostName().equals("test.com"));
    assertTrue("Should be satisfied if offer contains request", roc.satisfies(profile));
}
Also used : Protos(org.apache.mesos.Protos) HashMap(java.util.HashMap) OfferBuilder(org.apache.myriad.scheduler.offer.OfferBuilder) ServiceResourceProfile(org.apache.myriad.scheduler.ServiceResourceProfile) Test(org.junit.Test)

Example 5 with ServiceResourceProfile

use of org.apache.myriad.scheduler.ServiceResourceProfile in project incubator-myriad by apache.

the class TestResourceOfferContainer method testResourceOfferContainerForAuxServiceWithRole.

@Test
public void testResourceOfferContainerForAuxServiceWithRole() {
    Protos.Offer offer = new OfferBuilder("test.com").addScalarResource("cpus", 2.0).addScalarResource("mem", 8000).addScalarResource("cpus", "test", 4.0).addScalarResource("mem", "test", 32000.0).addRangeResource("ports", 3500, 3600).addRangeResource("ports", "test", 1500, 1600).build();
    Map<String, Long> ports = new HashMap<>(4);
    ports.put("test1.address", 0L);
    ports.put("test2.address", 0L);
    ports.put("test3.address", 1500L);
    ports.put("test4.port", 3501L);
    ServiceResourceProfile profile = new ServiceResourceProfile("jobhistory", 2.0, 8000.0, ports);
    ResourceOfferContainer roc = new ResourceOfferContainer(offer, profile, null);
    assertTrue(roc.getHostName().equals("test.com"));
    assertTrue("Should be satisfied if offer contains request", roc.satisfies(profile));
}
Also used : Protos(org.apache.mesos.Protos) HashMap(java.util.HashMap) OfferBuilder(org.apache.myriad.scheduler.offer.OfferBuilder) ServiceResourceProfile(org.apache.myriad.scheduler.ServiceResourceProfile) Test(org.junit.Test)

Aggregations

ServiceResourceProfile (org.apache.myriad.scheduler.ServiceResourceProfile)11 Protos (org.apache.mesos.Protos)5 HashMap (java.util.HashMap)4 OfferBuilder (org.apache.myriad.scheduler.offer.OfferBuilder)4 Test (org.junit.Test)4 Map (java.util.Map)3 MyriadConfiguration (org.apache.myriad.configuration.MyriadConfiguration)3 ExtendedResourceProfile (org.apache.myriad.scheduler.ExtendedResourceProfile)3 NMProfile (org.apache.myriad.scheduler.NMProfile)3 ServiceProfileManager (org.apache.myriad.scheduler.ServiceProfileManager)3 LikeConstraint (org.apache.myriad.scheduler.constraints.LikeConstraint)3 NodeTask (org.apache.myriad.state.NodeTask)3 TreeMap (java.util.TreeMap)2 Before (org.junit.Before)2 MyriadFileSystemRMStateStore (org.apache.hadoop.yarn.server.resourcemanager.recovery.MyriadFileSystemRMStateStore)1 Offer (org.apache.mesos.Protos.Offer)1 TaskInfo (org.apache.mesos.Protos.TaskInfo)1 SchedulerDriver (org.apache.mesos.SchedulerDriver)1 ServiceConfiguration (org.apache.myriad.configuration.ServiceConfiguration)1 TaskUtils (org.apache.myriad.scheduler.TaskUtils)1