Search in sources :

Example 21 with ICompute

use of edu.iu.dsc.tws.api.compute.nodes.ICompute in project twister2 by DSC-SPIDAL.

the class BTAllGatherExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    ISource g = new SourceTask(edge);
    ICompute r = new AllGatherSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.allgather(SOURCE).viaEdge(edge).withDataType(dataType);
    return computeGraphBuilder;
}
Also used : ISource(edu.iu.dsc.tws.api.compute.nodes.ISource) ICompute(edu.iu.dsc.tws.api.compute.nodes.ICompute) MessageType(edu.iu.dsc.tws.api.comms.messaging.types.MessageType)

Example 22 with ICompute

use of edu.iu.dsc.tws.api.compute.nodes.ICompute in project twister2 by DSC-SPIDAL.

the class BTBroadCastExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ICompute r = new BroadcastSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.broadcast(SOURCE).viaEdge(edge);
    return computeGraphBuilder;
}
Also used : BaseSource(edu.iu.dsc.tws.api.compute.nodes.BaseSource) ICompute(edu.iu.dsc.tws.api.compute.nodes.ICompute)

Aggregations

ICompute (edu.iu.dsc.tws.api.compute.nodes.ICompute)22 BaseSource (edu.iu.dsc.tws.api.compute.nodes.BaseSource)19 MessageType (edu.iu.dsc.tws.api.comms.messaging.types.MessageType)14 ISource (edu.iu.dsc.tws.api.compute.nodes.ISource)3 Op (edu.iu.dsc.tws.api.comms.Op)2 INode (edu.iu.dsc.tws.api.compute.nodes.INode)2 SourceBatchInstance (edu.iu.dsc.tws.executor.core.batch.SourceBatchInstance)2 TaskBatchInstance (edu.iu.dsc.tws.executor.core.batch.TaskBatchInstance)2 SourceStreamingInstance (edu.iu.dsc.tws.executor.core.streaming.SourceStreamingInstance)2 TaskStreamingInstance (edu.iu.dsc.tws.executor.core.streaming.TaskStreamingInstance)2 Checkpoint (edu.iu.dsc.tws.proto.checkpoint.Checkpoint)2 ArrayList (java.util.ArrayList)2 ArrayBlockingQueue (java.util.concurrent.ArrayBlockingQueue)2 LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)2 HashBasedTable (com.google.common.collect.HashBasedTable)1 Table (com.google.common.collect.Table)1 CheckpointingClient (edu.iu.dsc.tws.api.checkpointing.CheckpointingClient)1 Communicator (edu.iu.dsc.tws.api.comms.Communicator)1 LogicalPlan (edu.iu.dsc.tws.api.comms.LogicalPlan)1 TaskPartitioner (edu.iu.dsc.tws.api.compute.TaskPartitioner)1