Examples with Twister2Job edu.iu.dsc.tws.api.Twister2Job used on opensource projects

Example 31 with Twister2Job

use of edu.iu.dsc.tws.api.Twister2Job in project twister2 by DSC-SPIDAL.

the class TSetFTExample method main.

public static void main(String[] args) {
    Config config = ResourceAllocator.loadConfig(new HashMap<>());
    JobConfig jobConfig = new JobConfig();
    Twister2Job twister2Job = Twister2Job.newBuilder().setJobName("TSetFTExample").setWorkerClass(TSetFTExample.class).addComputeResource(1, 512, 2).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitJob(twister2Job, config);
}

Example 32 with Twister2Job

use of edu.iu.dsc.tws.api.Twister2Job in project twister2 by DSC-SPIDAL.

the class TSetCheckptExample method main.

public static void main(String[] args) {
    JobConfig jobConfig = new JobConfig();
    Twister2Job job = Twister2Job.newBuilder().setJobName(TSetCheckptExample.class.getName()).setConfig(jobConfig).setWorkerClass(TSetCheckptExample.class).addComputeResource(1, 512, 4).build();
    Twister2Submitter.submitJob(job);
}

Example 33 with Twister2Job

use of edu.iu.dsc.tws.api.Twister2Job in project twister2 by DSC-SPIDAL.

the class TSetComputeExample method main.

public static void main(String[] args) {
    JobConfig jobConfig = new JobConfig();
    Twister2Job job = Twister2Job.newBuilder().setJobName(TSetComputeExample.class.getName()).setConfig(jobConfig).setWorkerClass(TSetComputeExample.class).addComputeResource(1, 512, 4).build();
    Twister2Submitter.submitJob(job);
}

Example 34 with Twister2Job

use of edu.iu.dsc.tws.api.Twister2Job in project twister2 by DSC-SPIDAL.

the class LocalSubmitter method startWorker.

/**
 * This method starts a new worker instance on a separate thread.
 */
private static Thread startWorker(Twister2Job twister2Job, Config config, int workerId, CyclicBarrier cyclicBarrier) {
    Thread.UncaughtExceptionHandler hndler = (th, ex) -> {
        failed = true;
        fault = ex;
    };
    LocalClassLoader localClassLoader = new LocalClassLoader(LocalSubmitter.class.getClassLoader());
    localClassLoader.addJobClass(twister2Job.getWorkerClass());
    try {
        Object o = localClassLoader.loadClass(MockWorker.class.getName()).getConstructor(twister2Job.getClass(), config.getClass(), Integer.class, CyclicBarrier.class).newInstance(twister2Job, config, workerId, cyclicBarrier);
        Thread thread = new Thread((Runnable) o);
        thread.setName("worker-" + workerId);
        thread.setUncaughtExceptionHandler(hndler);
        thread.start();
        return thread;
    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | InstantiationException | InvocationTargetException e) {
        throw new Twister2RuntimeException("Failed to start the worker thread", e);
    }
}

Example 35 with Twister2Job

use of edu.iu.dsc.tws.api.Twister2Job in project twister2 by DSC-SPIDAL.

the class DataParallelJob method submitJob.

private static void submitJob(Config config, int containers, JobConfig jobConfig, String clazz) {
    LOG.info("Submitting Job ...");
    Twister2Job twister2Job;
    twister2Job = Twister2Job.newBuilder().setJobName(clazz).setWorkerClass(clazz).addComputeResource(1, 512, containers).setConfig(jobConfig).build();
    // now submit the job
    Twister2Submitter.submitJob(twister2Job, config);
}