Examples with ResourceContainer edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer used on opensource projects

Example 16 with ResourceContainer

use of edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer in project twister2 by DSC-SPIDAL.

the class BasicTaskgraphJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-taskgraphJob").setContainerClass(SimpleTaskgraph.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 2).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}

Example 17 with ResourceContainer

use of edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer in project twister2 by DSC-SPIDAL.

the class MemoryManagerGatherJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-gather-batch-MM").setContainerClass(BasicMemoryManagerGatherCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}

Example 18 with ResourceContainer

use of edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer in project twister2 by DSC-SPIDAL.

the class MemoryManagerTestJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-gather-MM").setContainerClass(BasicMemoryManagerContainer.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 1).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}

Example 19 with ResourceContainer

use of edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer in project twister2 by DSC-SPIDAL.

the class Utils method createTaskPlan.

/**
 * Let assume we have 1 task per container
 * @param plan the resource plan from scheduler
 * @return task plan
 */
public static TaskPlan createTaskPlan(Config cfg, ResourcePlan plan) {
    int noOfProcs = plan.noOfContainers();
    LOG.log(Level.INFO, "No of containers: " + noOfProcs);
    Map<Integer, Set<Integer>> executorToGraphNodes = new HashMap<>();
    Map<Integer, Set<Integer>> groupsToExeuctors = new HashMap<>();
    int thisExecutor = plan.getThisId();
    List<ResourceContainer> containers = plan.getContainers();
    Map<String, List<ResourceContainer>> containersPerNode = new HashMap<>();
    for (ResourceContainer c : containers) {
        String name = (String) c.getProperty(SchedulerContext.WORKER_NAME);
        List<ResourceContainer> containerList;
        if (!containersPerNode.containsKey(name)) {
            containerList = new ArrayList<>();
            containersPerNode.put(name, containerList);
        } else {
            containerList = containersPerNode.get(name);
        }
        containerList.add(c);
    }
    for (int i = 0; i < noOfProcs; i++) {
        Set<Integer> nodesOfExecutor = new HashSet<>();
        nodesOfExecutor.add(i);
        executorToGraphNodes.put(i, nodesOfExecutor);
    }
    int i = 0;
    // we take each container as an executor
    for (Map.Entry<String, List<ResourceContainer>> e : containersPerNode.entrySet()) {
        Set<Integer> executorsOfGroup = new HashSet<>();
        for (ResourceContainer c : e.getValue()) {
            executorsOfGroup.add(c.getId());
        }
        groupsToExeuctors.put(i, executorsOfGroup);
        i++;
    }
    String print = printMap(executorToGraphNodes);
    LOG.fine("Executor To Graph: " + print);
    print = printMap(groupsToExeuctors);
    LOG.fine("Groups to executors: " + print);
    // and reduce task in 0th process
    return new TaskPlan(executorToGraphNodes, groupsToExeuctors, thisExecutor);
}

Example 20 with ResourceContainer

use of edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer in project twister2 by DSC-SPIDAL.

the class MemoryManagerByteKeyedGatherJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-gather-batch-MM").setContainerClass(BasicMemoryManagerByteKeyedGatherCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}