Search in sources :

Example 6 with AddressedTuple

use of org.apache.storm.tuple.AddressedTuple in project storm by apache.

the class DeserializingConnectionCallback method recv.

@Override
public void recv(List<TaskMessage> batch) {
    KryoTupleDeserializer des = _des.get();
    ArrayList<AddressedTuple> ret = new ArrayList<>(batch.size());
    for (TaskMessage message : batch) {
        ret.add(new AddressedTuple(message.task(), des.deserialize(message.message())));
    }
    _cb.transfer(ret);
}
Also used : KryoTupleDeserializer(org.apache.storm.serialization.KryoTupleDeserializer) ArrayList(java.util.ArrayList) AddressedTuple(org.apache.storm.tuple.AddressedTuple)

Example 7 with AddressedTuple

use of org.apache.storm.tuple.AddressedTuple in project storm by apache.

the class WorkerState method transferLocal.

public void transferLocal(List<AddressedTuple> tupleBatch) {
    Map<Integer, List<AddressedTuple>> grouped = new HashMap<>();
    for (AddressedTuple tuple : tupleBatch) {
        Integer executor = taskToShortExecutor.get(tuple.dest);
        if (null == executor) {
            LOG.warn("Received invalid messages for unknown tasks. Dropping... ");
            continue;
        }
        List<AddressedTuple> current = grouped.get(executor);
        if (null == current) {
            current = new ArrayList<>();
            grouped.put(executor, current);
        }
        current.add(tuple);
    }
    for (Map.Entry<Integer, List<AddressedTuple>> entry : grouped.entrySet()) {
        DisruptorQueue queue = shortExecutorReceiveQueueMap.get(entry.getKey());
        if (null != queue) {
            queue.publish(entry.getValue());
        } else {
            LOG.warn("Received invalid messages for unknown tasks. Dropping... ");
        }
    }
}
Also used : DisruptorQueue(org.apache.storm.utils.DisruptorQueue) ImmutableMap(com.google.common.collect.ImmutableMap) AddressedTuple(org.apache.storm.tuple.AddressedTuple)

Example 8 with AddressedTuple

use of org.apache.storm.tuple.AddressedTuple in project storm by apache.

the class Executor method setupMetrics.

protected void setupMetrics() {
    for (final Integer interval : intervalToTaskToMetricToRegistry.keySet()) {
        StormTimer timerTask = workerData.getUserTimer();
        timerTask.scheduleRecurring(interval, interval, new Runnable() {

            @Override
            public void run() {
                TupleImpl tuple = new TupleImpl(workerTopologyContext, new Values(interval), (int) Constants.SYSTEM_TASK_ID, Constants.METRICS_TICK_STREAM_ID);
                List<AddressedTuple> metricsTickTuple = Lists.newArrayList(new AddressedTuple(AddressedTuple.BROADCAST_DEST, tuple));
                receiveQueue.publish(metricsTickTuple);
            }
        });
    }
}
Also used : StormTimer(org.apache.storm.StormTimer) Values(org.apache.storm.tuple.Values) List(java.util.List) ArrayList(java.util.ArrayList) TupleImpl(org.apache.storm.tuple.TupleImpl) AddressedTuple(org.apache.storm.tuple.AddressedTuple)

Aggregations

AddressedTuple (org.apache.storm.tuple.AddressedTuple)8 ArrayList (java.util.ArrayList)4 TupleImpl (org.apache.storm.tuple.TupleImpl)4 Values (org.apache.storm.tuple.Values)3 List (java.util.List)2 StormTimer (org.apache.storm.StormTimer)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)1 TaskMessage (org.apache.storm.messaging.TaskMessage)1 KryoTupleDeserializer (org.apache.storm.serialization.KryoTupleDeserializer)1 DisruptorQueue (org.apache.storm.utils.DisruptorQueue)1