Search in sources :

Example 1 with InvalidArguments

use of edu.iu.dsc.tws.examples.ml.svm.exceptions.InvalidArguments in project twister2 by DSC-SPIDAL.

the class SourceTaskDataLoaderMain method main.

public static void main(String[] args) throws InvalidArguments {
    if (args.length == 3) {
        workers = Integer.parseInt(args[0]);
        parallelism = Integer.parseInt(args[1]);
        dataSource = args[2];
        Config config = ResourceAllocator.loadConfig(new HashMap<>());
        JobConfig jobConfig = new JobConfig();
        jobConfig.put(WorkerConstants.WORKERS, workers);
        jobConfig.put(WorkerConstants.PARALLELISM, parallelism);
        jobConfig.put(MLDataObjectConstants.TRAINING_DATA_DIR, dataSource);
        Twister2Job.Twister2JobBuilder jobBuilder = Twister2Job.newBuilder();
        jobBuilder.setJobName("SourceTaskDataLoader");
        jobBuilder.setWorkerClass(SourceTaskDataLoader.class.getName());
        jobBuilder.addComputeResource(2, 512, 1.0, workers);
        jobBuilder.setConfig(jobConfig);
        // now submit the job
        Twister2Submitter.submitJob(jobBuilder.build(), config);
    } else {
        throw new InvalidArguments("Invalid Arguments : <workers> <parallelism> <datafile_path>");
    }
}
Also used : InvalidArguments(edu.iu.dsc.tws.examples.ml.svm.exceptions.InvalidArguments) Config(edu.iu.dsc.tws.api.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) JobConfig(edu.iu.dsc.tws.api.JobConfig) Twister2Job(edu.iu.dsc.tws.api.Twister2Job)

Example 2 with InvalidArguments

use of edu.iu.dsc.tws.examples.ml.svm.exceptions.InvalidArguments in project twister2 by DSC-SPIDAL.

the class TaskWorkerDataLoaderMain method main.

public static void main(String[] args) throws InvalidArguments {
    if (args.length == 3) {
        workers = Integer.parseInt(args[0]);
        parallelism = Integer.parseInt(args[1]);
        dataSource = args[2];
        Config config = ResourceAllocator.loadConfig(new HashMap<>());
        JobConfig jobConfig = new JobConfig();
        jobConfig.put(WorkerConstants.WORKERS, workers);
        jobConfig.put(WorkerConstants.PARALLELISM, parallelism);
        jobConfig.put(MLDataObjectConstants.TRAINING_DATA_DIR, dataSource);
        Twister2Job.Twister2JobBuilder jobBuilder = Twister2Job.newBuilder();
        jobBuilder.setJobName("TaskWorkerDataLoader");
        jobBuilder.setWorkerClass(TaskWorkerDataLoader.class.getName());
        jobBuilder.addComputeResource(2, 512, 1.0, workers);
        jobBuilder.setConfig(jobConfig);
        // now submit the job
        Twister2Submitter.submitJob(jobBuilder.build(), config);
    } else {
        throw new InvalidArguments("Invalid Arguments : <workers> <parallelism> <datafile_path>");
    }
}
Also used : InvalidArguments(edu.iu.dsc.tws.examples.ml.svm.exceptions.InvalidArguments) Config(edu.iu.dsc.tws.api.config.Config) JobConfig(edu.iu.dsc.tws.api.JobConfig) JobConfig(edu.iu.dsc.tws.api.JobConfig) Twister2Job(edu.iu.dsc.tws.api.Twister2Job)

Aggregations

JobConfig (edu.iu.dsc.tws.api.JobConfig)2 Twister2Job (edu.iu.dsc.tws.api.Twister2Job)2 Config (edu.iu.dsc.tws.api.config.Config)2 InvalidArguments (edu.iu.dsc.tws.examples.ml.svm.exceptions.InvalidArguments)2