Search in sources :

Example 66 with ComputeGraphBuilder

use of edu.iu.dsc.tws.task.impl.ComputeGraphBuilder in project twister2 by DSC-SPIDAL.

the class KMeansComputeJob method buildDataPointsTG.

public static ComputeGraph buildDataPointsTG(String dataDirectory, int dsize, int parallelismValue, int dimension, Config conf, String filetype) {
    PointDataSource ps = new PointDataSource(Context.TWISTER2_DIRECT_EDGE, dataDirectory, "points", dimension, dsize, filetype);
    ComputeGraphBuilder datapointsComputeGraphBuilder = ComputeGraphBuilder.newBuilder(conf);
    // Add source, compute, and sink tasks to the task graph builder for the first task graph
    datapointsComputeGraphBuilder.addSource("datapointsource", ps, parallelismValue);
    datapointsComputeGraphBuilder.setMode(OperationMode.BATCH);
    datapointsComputeGraphBuilder.setTaskGraphName("datapointsTG");
    return datapointsComputeGraphBuilder.build();
}
Also used : ComputeGraphBuilder(edu.iu.dsc.tws.task.impl.ComputeGraphBuilder)

Aggregations

ComputeGraphBuilder (edu.iu.dsc.tws.task.impl.ComputeGraphBuilder)66 ComputeConnection (edu.iu.dsc.tws.task.impl.ComputeConnection)55 ComputeGraph (edu.iu.dsc.tws.api.compute.graph.ComputeGraph)39 TaskSchedulerClassTest (edu.iu.dsc.tws.tsched.utils.TaskSchedulerClassTest)16 ExecutionPlan (edu.iu.dsc.tws.api.compute.executor.ExecutionPlan)10 DataFlowGraph (edu.iu.dsc.tws.task.cdfw.DataFlowGraph)8 ComputeEnvironment (edu.iu.dsc.tws.task.ComputeEnvironment)7 GraphDataSource (edu.iu.dsc.tws.graphapi.partition.GraphDataSource)6 Config (edu.iu.dsc.tws.api.config.Config)4 ConnectedSink (edu.iu.dsc.tws.task.cdfw.task.ConnectedSink)4 DataFileReplicatedReadSource (edu.iu.dsc.tws.task.dataobjects.DataFileReplicatedReadSource)4 Path (edu.iu.dsc.tws.api.data.Path)3 DataObjectSource (edu.iu.dsc.tws.task.dataobjects.DataObjectSource)3 ReduceFn (edu.iu.dsc.tws.task.impl.function.ReduceFn)3 JobConfig (edu.iu.dsc.tws.api.JobConfig)2 DataObject (edu.iu.dsc.tws.api.dataset.DataObject)2 TextInputSplit (edu.iu.dsc.tws.data.api.splits.TextInputSplit)2 IterativeSVMDataObjectCompute (edu.iu.dsc.tws.examples.ml.svm.data.IterativeSVMDataObjectCompute)2 IterativeSVMDataObjectDirectSink (edu.iu.dsc.tws.examples.ml.svm.data.IterativeSVMDataObjectDirectSink)2 IterativeSVMWeightVectorObjectCompute (edu.iu.dsc.tws.examples.ml.svm.data.IterativeSVMWeightVectorObjectCompute)2