Examples with BaseSource edu.iu.dsc.tws.api.compute.nodes.BaseSource used on opensource projects

Example 1 with BaseSource

use of edu.iu.dsc.tws.api.compute.nodes.BaseSource in project twister2 by DSC-SPIDAL.

the class BTAllReduceExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ICompute r = new AllReduceSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.allreduce(SOURCE).viaEdge(edge).withOperation(Op.SUM, MessageTypes.INTEGER_ARRAY);
    return computeGraphBuilder;
}

Example 2 with BaseSource

use of edu.iu.dsc.tws.api.compute.nodes.BaseSource in project twister2 by DSC-SPIDAL.

the class BTReduceExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ICompute r = new ReduceSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.reduce(SOURCE).viaEdge(edge).withOperation(Op.SUM, MessageTypes.INTEGER_ARRAY);
    return computeGraphBuilder;
}

Example 3 with BaseSource

use of edu.iu.dsc.tws.api.compute.nodes.BaseSource in project twister2 by DSC-SPIDAL.

the class STAllGatherExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int psource = taskStages.get(0);
    int psink = taskStages.get(1);
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ICompute r = new AllGatherSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, psource);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, psink);
    computeConnection.allgather(SOURCE).viaEdge(edge).withDataType(dataType);
    return computeGraphBuilder;
}

Example 4 with BaseSource

use of edu.iu.dsc.tws.api.compute.nodes.BaseSource in project twister2 by DSC-SPIDAL.

the class STPartitionExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ((SourceTask) g).setMarkTimingOnlyForLowestTarget(true);
    ICompute r = new PartitionSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.partition(SOURCE).viaEdge(edge).withDataType(dataType);
    return computeGraphBuilder;
}

Example 5 with BaseSource

use of edu.iu.dsc.tws.api.compute.nodes.BaseSource in project twister2 by DSC-SPIDAL.

the class BTPartitionKeyedExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    MessageType keyType = MessageTypes.INTEGER;
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    BaseSource g = new SourceTask(edge, true);
    ICompute r = new BKeyedPartitionSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.keyedPartition(SOURCE).viaEdge(edge).withKeyType(keyType).withTaskPartitioner(new DeterministicTaskPartitioner()).withDataType(dataType);
    return computeGraphBuilder;
}