Search in sources :

Example 1 with OrbPartitionCommunicationProtocol

use of org.goldenorb.OrbPartitionCommunicationProtocol in project goldenorb by jzachr.

the class OutboundVertexThread method testOutBoundVertexQueue.

@Test
public void testOutBoundVertexQueue() throws Exception {
    // OutboundVertexQueue settings
    int numberOfPartitions = 100;
    // max number of Vertices to be sent by a Thread
    int numOfVerticesToSendPerThread = 10500;
    // max number of Vertices to trigger a send operation by the queue
    int numOfVerticesPerBlock = 1000;
    int partitionId = 1;
    Class<? extends Vertex<?, ?, ?>> vertexClass = TestVertex.class;
    Map<Integer, OrbPartitionCommunicationProtocol> orbClients = new HashMap<Integer, OrbPartitionCommunicationProtocol>();
    for (int i = 0; i < numberOfPartitions; i++) {
        orbClients.put(new Integer(i), infoCollector);
    }
    OutboundVertexQueue ovq = new OutboundVertexQueue(numberOfPartitions, numOfVerticesPerBlock, orbClients, vertexClass, partitionId);
    // initialize the Threads and pass them their test Vertices
    CountDownLatch startLatch = new CountDownLatch(1);
    CountDownLatch everyoneDoneLatch = new CountDownLatch(numberOfPartitions);
    for (int i = 0; i < numberOfPartitions; i++) {
        Vertices vrts = new Vertices(vertexClass);
        for (int p = 0; p < numOfVerticesToSendPerThread; p++) {
            String vertexID = "vertex " + p;
            IntWritable vertexValue = new IntWritable(p);
            List<Edge<IntWritable>> edgesList = new ArrayList<Edge<IntWritable>>();
            TestVertex vrt = new TestVertex(vertexID, vertexValue, edgesList);
            vrts.add(vrt);
        }
        OutboundVertexThread obmThread = new OutboundVertexThread(vrts, ovq, startLatch, everyoneDoneLatch);
        // initialize a Thread
        obmThread.start();
    }
    // start all Threads simultaneously
    startLatch.countDown();
    // wait until all Threads are done
    everyoneDoneLatch.await();
    ovq.sendRemainingVertices();
    System.out.println(infoCollector.vList.size());
    assertThat(ovq, notNullValue());
    assertTrue(infoCollector.vList.size() == (numberOfPartitions * numOfVerticesToSendPerThread));
}
Also used : OrbPartitionCommunicationProtocol(org.goldenorb.OrbPartitionCommunicationProtocol) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) CountDownLatch(java.util.concurrent.CountDownLatch) Vertices(org.goldenorb.Vertices) Edge(org.goldenorb.Edge) IntWritable(org.apache.hadoop.io.IntWritable) Test(org.junit.Test)

Example 2 with OrbPartitionCommunicationProtocol

use of org.goldenorb.OrbPartitionCommunicationProtocol in project goldenorb by jzachr.

the class OutboundMessageThread method testOutBoundMessageQueue.

@Test
public void testOutBoundMessageQueue() throws Exception {
    // OutboundMessageQueue settings
    int numberOfPartitions = 100;
    // max number of Messages to be sent by a Thread
    int numOfMessagesToSendPerThread = 10500;
    // max number of Messages to trigger a send operation by the queue
    int numOfMessagesPerBlock = 1000;
    int partitionId = 1;
    Class<? extends Message<? extends Writable>> messageClass = TextMessage.class;
    Map<Integer, OrbPartitionCommunicationProtocol> orbClients = new HashMap<Integer, OrbPartitionCommunicationProtocol>();
    for (int i = 0; i < numberOfPartitions; i++) {
        orbClients.put(new Integer(i), infoCollector);
    }
    OutboundMessageQueue omq = new OutboundMessageQueue(numberOfPartitions, numOfMessagesPerBlock, orbClients, messageClass, partitionId);
    // initialize the Threads and pass them their test Messages
    CountDownLatch startLatch = new CountDownLatch(1);
    CountDownLatch everyoneDoneLatch = new CountDownLatch(numberOfPartitions);
    for (int i = 0; i < numberOfPartitions; i++) {
        Messages msgs = new Messages(TextMessage.class);
        for (int p = 0; p < numOfMessagesToSendPerThread; p++) {
            TextMessage txtmsg = new TextMessage(Integer.toString(i), new Text("test message " + Integer.toString(p)));
            msgs.add(txtmsg);
        }
        OutboundMessageThread obmThread = new OutboundMessageThread(msgs, omq, startLatch, everyoneDoneLatch);
        // initialize a Thread
        obmThread.start();
    }
    // start all Threads simultaneously
    startLatch.countDown();
    // wait until all Threads are done
    everyoneDoneLatch.await();
    omq.sendRemainingMessages();
    assertThat(omq, notNullValue());
    assertTrue(infoCollector.mList.size() == (numberOfPartitions * numOfMessagesToSendPerThread));
}
Also used : OrbPartitionCommunicationProtocol(org.goldenorb.OrbPartitionCommunicationProtocol) Messages(org.goldenorb.Messages) HashMap(java.util.HashMap) Text(org.apache.hadoop.io.Text) CountDownLatch(java.util.concurrent.CountDownLatch) TextMessage(org.goldenorb.types.message.TextMessage) Test(org.junit.Test)

Aggregations

HashMap (java.util.HashMap)2 CountDownLatch (java.util.concurrent.CountDownLatch)2 OrbPartitionCommunicationProtocol (org.goldenorb.OrbPartitionCommunicationProtocol)2 Test (org.junit.Test)2 ArrayList (java.util.ArrayList)1 IntWritable (org.apache.hadoop.io.IntWritable)1 Text (org.apache.hadoop.io.Text)1 Edge (org.goldenorb.Edge)1 Messages (org.goldenorb.Messages)1 Vertices (org.goldenorb.Vertices)1 TextMessage (org.goldenorb.types.message.TextMessage)1