Examples with Path edu.iu.dsc.tws.api.data.Path used on opensource projects

Example 26 with Path

use of edu.iu.dsc.tws.api.data.Path in project twister2 by DSC-SPIDAL.

the class SVMDataObjectSource method prepare.

@Override
public void prepare(Config cfg, TaskContext ctx) {
    super.prepare(cfg, ctx);
    ExecutionRuntime runtime = (ExecutionRuntime) cfg.get(ExecutorContext.TWISTER2_RUNTIME_OBJECT);
    this.source = runtime.createInput(cfg, context, new LocalFixedInputPartitioner(new Path(getDataDirectory()), context.getParallelism(), config, this.datasize));
}

Example 27 with Path

use of edu.iu.dsc.tws.api.data.Path in project twister2 by DSC-SPIDAL.

the class DataFileSink method prepare.

@Override
public void prepare(Config cfg, TaskContext context) {
    super.prepare(cfg, context);
    String outDir = cfg.getStringValue(DataObjectConstants.OUTPUT_DIRECTORY);
    this.datasink = new DataSink<>(cfg, new TextOutputWriter(FileSystem.WriteMode.OVERWRITE, new Path(outDir)));
}

Example 28 with Path

use of edu.iu.dsc.tws.api.data.Path in project twister2 by DSC-SPIDAL.

the class DataObjectCSVSource method prepare.

@Override
public void prepare(Config cfg, TaskContext context) {
    super.prepare(cfg, context);
    ExecutionRuntime runtime = (ExecutionRuntime) cfg.get(ExecutorContext.TWISTER2_RUNTIME_OBJECT);
    this.source = runtime.createInput(cfg, context, new LocalCSVInputPartitioner(new Path(getDataDirectory()), context.getParallelism(), cfg));
}

Example 29 with Path

use of edu.iu.dsc.tws.api.data.Path in project twister2 by DSC-SPIDAL.

the class DataFileReplicatedReadSource method prepare.

public void prepare(Config cfg, TaskContext context) {
    super.prepare(cfg, context);
    ExecutionRuntime runtime = (ExecutionRuntime) cfg.get(ExecutorContext.TWISTER2_RUNTIME_OBJECT);
    this.source = runtime.createInput(cfg, context, new LocalCompleteTextInputPartitioner(new Path(getDataDirectory()), context.getParallelism(), config));
}

Example 30 with Path

use of edu.iu.dsc.tws.api.data.Path in project twister2 by DSC-SPIDAL.

the class DataFileSplittedReadSource method prepare.

/**
 * Retrieve the data input directory using the DataObjectConstants and the config file.
 * @param cfg
 * @param context
 */
@Override
public void prepare(Config cfg, TaskContext context) {
    super.prepare(cfg, context);
    ExecutionRuntime runtime = (ExecutionRuntime) cfg.get(ExecutorContext.TWISTER2_RUNTIME_OBJECT);
    this.source = runtime.createInput(cfg, context, new LocalTextInputPartitioner(new Path(getDataDirectory()), context.getParallelism(), config));
}