Examples with Direction edu.snu.mist.formats.avro.Direction used on opensource projects

Example 1 with Direction

use of edu.snu.mist.formats.avro.Direction in project mist by snuspl.

the class OperatorOutputEmitter method emitData.

/**
 * This method emits the outputs to next OperatorChains.
 * If the Executor of the current OperatorChain is same as that of next OperatorChain,
 * the OutputEmitter directly forwards outputs of the current OperatorChain
 * as inputs of the next OperatorChain.
 * Otherwise, the OutputEmitter submits a job to the Executor of the next OperatorChain.
 * @param output an output
 */
@Override
public void emitData(final MistDataEvent output) {
    // Optimization: do not create new MistEvent and reuse it if it has one downstream operator chain.
    if (nextOperators.size() == 1) {
        for (final Map.Entry<ExecutionVertex, MISTEdge> nextChain : nextOperators.entrySet()) {
            final Direction direction = nextChain.getValue().getDirection();
            sendData(output, direction, nextChain.getKey());
        }
    } else {
        for (final Map.Entry<ExecutionVertex, MISTEdge> nextChain : nextOperators.entrySet()) {
            final MistDataEvent event = new MistDataEvent(output.getValue(), output.getTimestamp());
            final Direction direction = nextChain.getValue().getDirection();
            sendData(event, direction, nextChain.getKey());
        }
    }
}

Example 2 with Direction

use of edu.snu.mist.formats.avro.Direction in project mist by snuspl.

the class OperatorOutputEmitter method emitWatermark.

@Override
public void emitWatermark(final MistWatermarkEvent output) {
    // Watermark is not changed, so we just forward watermark to next operator chains.
    for (final Map.Entry<ExecutionVertex, MISTEdge> nextQuery : nextOperators.entrySet()) {
        final Direction direction = nextQuery.getValue().getDirection();
        sendWatermark(output, direction, nextQuery.getKey());
    }
}

Example 3 with Direction

use of edu.snu.mist.formats.avro.Direction in project mist by snuspl.

the class OperatorOutputEmitter method emitCheckpoint.

@Override
public void emitCheckpoint(final MistCheckpointEvent checkpoint) {
    // Checkpoint is not changed, so we just forward it to the next operator chains.
    for (final Map.Entry<ExecutionVertex, MISTEdge> nextQuery : nextOperators.entrySet()) {
        final Direction direction = nextQuery.getValue().getDirection();
        sendCheckpoint(checkpoint, direction, nextQuery.getKey());
    }
}

Example 4 with Direction

use of edu.snu.mist.formats.avro.Direction in project mist by snuspl.

the class AdjacentListConcurrentMapDAGTest method concurrentReadWriteEdgeTest.

/**
 * Test the DAG supports concurrent read-write on the edges.
 */
@Test
public void concurrentReadWriteEdgeTest() {
    final DAG<Integer, Direction> dag = new AdjacentListConcurrentMapDAG<>();
    dag.addVertex(1);
    dag.addVertex(2);
    dag.addVertex(3);
    dag.addVertex(4);
    dag.addEdge(1, 3, Direction.LEFT);
    dag.addEdge(3, 4, Direction.LEFT);
    dag.addEdge(2, 4, Direction.RIGHT);
    dag.addVertex(5);
    final AtomicBoolean finished = new AtomicBoolean(false);
    final Thread writeThread = new Thread(new Runnable() {

        @Override
        public void run() {
            while (!finished.get()) {
                dag.addEdge(1, 5, Direction.LEFT);
                dag.removeEdge(1, 5);
            }
        }
    });
    writeThread.start();
    final Map<Integer, Direction> edges = dag.getEdges(1);
    for (int i = 0; i < 20000; i++) {
        for (final Map.Entry<Integer, Direction> edge : edges.entrySet()) {
            LOG.fine("" + 1 + "->" + edge.getKey());
        }
    }
    finished.set(true);
}