Search in sources :

Example 36 with MessageType

use of edu.iu.dsc.tws.api.comms.messaging.types.MessageType in project twister2 by DSC-SPIDAL.

the class BTPartitionExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    BaseSource g = new SourceTask(edge);
    ICompute r = new PartitionSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.partition(SOURCE).viaEdge(edge).withDataType(dataType);
    return computeGraphBuilder;
}
Also used : BaseSource(edu.iu.dsc.tws.api.compute.nodes.BaseSource) ICompute(edu.iu.dsc.tws.api.compute.nodes.ICompute) MessageType(edu.iu.dsc.tws.api.comms.messaging.types.MessageType)

Example 37 with MessageType

use of edu.iu.dsc.tws.api.comms.messaging.types.MessageType in project twister2 by DSC-SPIDAL.

the class BTAllGatherExample method buildTaskGraph.

@Override
public ComputeGraphBuilder buildTaskGraph() {
    List<Integer> taskStages = jobParameters.getTaskStages();
    int sourceParallelism = taskStages.get(0);
    int sinkParallelism = taskStages.get(1);
    MessageType dataType = MessageTypes.INTEGER_ARRAY;
    String edge = "edge";
    ISource g = new SourceTask(edge);
    ICompute r = new AllGatherSinkTask();
    computeGraphBuilder.addSource(SOURCE, g, sourceParallelism);
    computeConnection = computeGraphBuilder.addCompute(SINK, r, sinkParallelism);
    computeConnection.allgather(SOURCE).viaEdge(edge).withDataType(dataType);
    return computeGraphBuilder;
}
Also used : ISource(edu.iu.dsc.tws.api.compute.nodes.ISource) ICompute(edu.iu.dsc.tws.api.compute.nodes.ICompute) MessageType(edu.iu.dsc.tws.api.comms.messaging.types.MessageType)

Aggregations

MessageType (edu.iu.dsc.tws.api.comms.messaging.types.MessageType)37 InMessage (edu.iu.dsc.tws.comms.dfw.InMessage)15 ICompute (edu.iu.dsc.tws.api.compute.nodes.ICompute)14 BaseSource (edu.iu.dsc.tws.api.compute.nodes.BaseSource)13 Test (org.junit.Test)13 Tuple (edu.iu.dsc.tws.api.comms.structs.Tuple)11 ArrayList (java.util.ArrayList)8 List (java.util.List)5 CommunicationContext (edu.iu.dsc.tws.api.comms.CommunicationContext)4 MessageHeader (edu.iu.dsc.tws.api.comms.messaging.MessageHeader)4 JoinedTuple (edu.iu.dsc.tws.api.comms.structs.JoinedTuple)4 Twister2RuntimeException (edu.iu.dsc.tws.api.exceptions.Twister2RuntimeException)4 ResettableIterator (edu.iu.dsc.tws.comms.shuffle.ResettableIterator)4 Collections (java.util.Collections)4 Iterator (java.util.Iterator)4 Map (java.util.Map)4 Logger (java.util.logging.Logger)4 Op (edu.iu.dsc.tws.api.comms.Op)2 ChannelMessage (edu.iu.dsc.tws.api.comms.messaging.ChannelMessage)2 DataBuffer (edu.iu.dsc.tws.api.comms.packing.DataBuffer)2