Search in sources :

Example 26 with JobConfig

use of edu.iu.dsc.tws.api.JobConfig in project twister2 by DSC-SPIDAL.

the class WordCountJob method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    BasicJob.BasicJobBuilder jobBuilder = BasicJob.newBuilder();
    jobBuilder.setName("batch-wordcount");
    jobBuilder.setContainerClass(WordCountContainer.class.getName());
    jobBuilder.setRequestResource(new ResourceContainer(2, 1024), 4);
    jobBuilder.setConfig(jobConfig);
    // now submit the job
    Twister2Submitter.submitContainerJob(jobBuilder.build(), config);
}
Also used : BasicJob(edu.iu.dsc.tws.api.basic.job.BasicJob) Config(edu.iu.dsc.tws.common.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) JobConfig(edu.iu.dsc.tws.api.JobConfig) ResourceContainer(edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)

Example 27 with JobConfig

use of edu.iu.dsc.tws.api.JobConfig in project twister2 by DSC-SPIDAL.

the class BaseAllReduceCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-all-reduce").setContainerClass(BaseAllReduceCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}
Also used : BasicJob(edu.iu.dsc.tws.api.basic.job.BasicJob) JobConfig(edu.iu.dsc.tws.api.JobConfig) Config(edu.iu.dsc.tws.common.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) ResourceContainer(edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)

Example 28 with JobConfig

use of edu.iu.dsc.tws.api.JobConfig in project twister2 by DSC-SPIDAL.

the class BaseBroadcastCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-broadcast").setContainerClass(BaseBroadcastCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}
Also used : BasicJob(edu.iu.dsc.tws.api.basic.job.BasicJob) JobConfig(edu.iu.dsc.tws.api.JobConfig) Config(edu.iu.dsc.tws.common.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) ResourceContainer(edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)

Example 29 with JobConfig

use of edu.iu.dsc.tws.api.JobConfig in project twister2 by DSC-SPIDAL.

the class BaseReduceBatchCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-batch-reduce").setContainerClass(BaseReduceBatchCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}
Also used : BasicJob(edu.iu.dsc.tws.api.basic.job.BasicJob) JobConfig(edu.iu.dsc.tws.api.JobConfig) Config(edu.iu.dsc.tws.common.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) ResourceContainer(edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)

Example 30 with JobConfig

use of edu.iu.dsc.tws.api.JobConfig in project twister2 by DSC-SPIDAL.

the class BaseReduceHLCommunication method main.

public static void main(String[] args) {
    // first load the configurations from command line and config files
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    // build JobConfig
    JobConfig jobConfig = new JobConfig();
    // build the job
    BasicJob basicJob = BasicJob.newBuilder().setName("basic-hl-reduce").setContainerClass(BaseReduceHLCommunication.class.getName()).setRequestResource(new ResourceContainer(2, 1024), 4).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitContainerJob(basicJob, config);
}
Also used : BasicJob(edu.iu.dsc.tws.api.basic.job.BasicJob) JobConfig(edu.iu.dsc.tws.api.JobConfig) Config(edu.iu.dsc.tws.common.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) ResourceContainer(edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)

Aggregations

JobConfig (edu.iu.dsc.tws.api.JobConfig)33 BasicJob (edu.iu.dsc.tws.api.basic.job.BasicJob)31 ResourceContainer (edu.iu.dsc.tws.rsched.spi.resource.ResourceContainer)31 Config (edu.iu.dsc.tws.common.config.Config)30 Twister2Job (edu.iu.dsc.tws.api.Twister2Job)2 Config (edu.iu.dsc.tws.api.config.Config)2 Twister2JobState (edu.iu.dsc.tws.api.scheduler.Twister2JobState)2 HashMap (java.util.HashMap)2 LinkedHashMap (java.util.LinkedHashMap)2 Map (java.util.Map)2 Twister2RuntimeException (edu.iu.dsc.tws.api.exceptions.Twister2RuntimeException)1 PTransformOverride (org.apache.beam.sdk.runners.PTransformOverride)1