Examples with Config edu.iu.dsc.tws.common.config.Config used on opensource projects

Example 51 with Config

use of edu.iu.dsc.tws.common.config.Config in project twister2 by DSC-SPIDAL.

the class Test method testJobCreate.

/**
 * testJobCreate
 */
public static void testJobCreate() {
    String auroraFile = "/root/twister2/twister2test.aurora";
    String cluster = "example";
    String role = "www-data";
    String env = "devel";
    String jobName = "hello";
    AuroraClientController controller = new AuroraClientController(cluster, role, env, jobName, true);
    Config.Builder builder = Config.newBuilder();
    builder.put(AuroraContext.AURORA_CLUSTER_NAME, cluster);
    builder.put(AuroraContext.ROLE, role);
    builder.put(AuroraContext.ENVIRONMENT, env);
    builder.put(SchedulerContext.JOB_NAME, jobName);
    builder.put(AuroraContext.AURORA_SCRIPT.getKey(), auroraFile);
    // builder.put(AuroraContext.TWISTER2_PACKAGE_PATH, "/root/twister2");
    // builder.put(AuroraContext.TWISTER2_PACKAGE_FILE, "twister2-client.tar.gz");
    builder.put(AuroraContext.NUMBER_OF_CONTAINERS, "1");
    builder.put(AuroraContext.CPUS_PER_CONTAINER, "1");
    // 1GB in bytes
    String ramAndDiskSize = "" + 1 * 1024 * 1024 * 1024;
    builder.put(AuroraContext.RAM_PER_CONTAINER, ramAndDiskSize);
    builder.put(AuroraContext.DISK_PER_CONTAINER, ramAndDiskSize);
    Config config = builder.build();
    System.out.println("number of config parameters: " + config.size());
    System.out.println(config);
    // get environment variables from config
    Map<AuroraField, String> bindings = AuroraLauncher.constructEnvVariables(config);
    // print all environment variables for debugging
    AuroraLauncher.printEnvs(bindings);
    boolean result = controller.createJob(bindings, auroraFile);
    if (result) {
        System.out.println("job submission is successfull");
    } else {
        System.out.println("job submission is unsuccessfull");
    }
}

Example 52 with Config

use of edu.iu.dsc.tws.common.config.Config in project twister2 by DSC-SPIDAL.

the class WordCountJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    BasicJob.BasicJobBuilder jobBuilder = BasicJob.newBuilder();
    jobBuilder.setName("batch-wordcount");
    jobBuilder.setContainerClass(WordCountContainer.class.getName());
    jobBuilder.setRequestResource(new ResourceContainer(2, 1024), 4);
    jobBuilder.setConfig(jobConfig);
    // now submit the job
    Twister2Submitter.submitContainerJob(jobBuilder.build(), config);
}

Example 53 with Config

use of edu.iu.dsc.tws.common.config.Config in project twister2 by DSC-SPIDAL.

the class BaseAllReduceCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-all-reduce").setContainerClass(BaseAllReduceCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}

Example 54 with Config

use of edu.iu.dsc.tws.common.config.Config in project twister2 by DSC-SPIDAL.

the class BaseBroadcastCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-broadcast").setContainerClass(BaseBroadcastCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}

Example 55 with Config

use of edu.iu.dsc.tws.common.config.Config in project twister2 by DSC-SPIDAL.

the class BaseReduceBatchCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-batch-reduce").setContainerClass(BaseReduceBatchCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}