Search in sources :

Example 1 with DirectRouter

use of edu.iu.dsc.tws.comms.routing.DirectRouter in project twister2 by DSC-SPIDAL.

the class MPIDirectDataFlowCommunication method init.

/**
 * Initialize
 * @param cfg
 * @param t
 * @param taskPlan
 * @param edge
 */
public void init(Config cfg, MessageType t, TaskPlan taskPlan, int edge) {
    this.router = new DirectRouter(taskPlan, sources, destination);
    if (this.finalReceiver != null && isLastReceiver()) {
        this.finalReceiver.init(cfg, this, receiveExpectedTaskIds());
    }
    Map<Integer, ArrayBlockingQueue<Pair<Object, MPISendMessage>>> pendingSendMessagesPerSource = new HashMap<>();
    Map<Integer, Queue<Pair<Object, MPIMessage>>> pendingReceiveMessagesPerSource = new HashMap<>();
    Map<Integer, Queue<MPIMessage>> pendingReceiveDeSerializations = new HashMap<>();
    Map<Integer, MessageSerializer> serializerMap = new HashMap<>();
    Map<Integer, MessageDeSerializer> deSerializerMap = new HashMap<>();
    Set<Integer> srcs = TaskPlanUtils.getTasksOfThisExecutor(taskPlan, sources);
    for (int s : srcs) {
        // later look at how not to allocate pairs for this each time
        ArrayBlockingQueue<Pair<Object, MPISendMessage>> pendingSendMessages = new ArrayBlockingQueue<Pair<Object, MPISendMessage>>(MPIContext.sendPendingMax(cfg));
        pendingSendMessagesPerSource.put(s, pendingSendMessages);
        pendingReceiveDeSerializations.put(s, new ArrayBlockingQueue<MPIMessage>(MPIContext.sendPendingMax(cfg)));
        serializerMap.put(s, new MPIMessageSerializer(new KryoSerializer()));
    }
    MessageDeSerializer messageDeSerializer = new MPIMessageDeSerializer(new KryoSerializer());
    deSerializerMap.put(destination, messageDeSerializer);
    delegete.init(cfg, t, taskPlan, edge, router.receivingExecutors(), isLastReceiver(), this, pendingSendMessagesPerSource, pendingReceiveMessagesPerSource, pendingReceiveDeSerializations, serializerMap, deSerializerMap, false);
}
Also used : MPIMessageDeSerializer(edu.iu.dsc.tws.comms.mpi.io.MPIMessageDeSerializer) MPIMessageSerializer(edu.iu.dsc.tws.comms.mpi.io.MPIMessageSerializer) MessageSerializer(edu.iu.dsc.tws.comms.mpi.io.MessageSerializer) HashMap(java.util.HashMap) DirectRouter(edu.iu.dsc.tws.comms.routing.DirectRouter) KryoSerializer(edu.iu.dsc.tws.comms.utils.KryoSerializer) MPIMessageDeSerializer(edu.iu.dsc.tws.comms.mpi.io.MPIMessageDeSerializer) MessageDeSerializer(edu.iu.dsc.tws.comms.mpi.io.MessageDeSerializer) MPIMessageSerializer(edu.iu.dsc.tws.comms.mpi.io.MPIMessageSerializer) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) Queue(java.util.Queue) Pair(org.apache.commons.lang3.tuple.Pair)

Aggregations

MPIMessageDeSerializer (edu.iu.dsc.tws.comms.mpi.io.MPIMessageDeSerializer)1 MPIMessageSerializer (edu.iu.dsc.tws.comms.mpi.io.MPIMessageSerializer)1 MessageDeSerializer (edu.iu.dsc.tws.comms.mpi.io.MessageDeSerializer)1 MessageSerializer (edu.iu.dsc.tws.comms.mpi.io.MessageSerializer)1 DirectRouter (edu.iu.dsc.tws.comms.routing.DirectRouter)1 KryoSerializer (edu.iu.dsc.tws.comms.utils.KryoSerializer)1 HashMap (java.util.HashMap)1 Queue (java.util.Queue)1 ArrayBlockingQueue (java.util.concurrent.ArrayBlockingQueue)1 Pair (org.apache.commons.lang3.tuple.Pair)1