Search in sources :

Example 6 with TupleImpl

use of org.apache.storm.tuple.TupleImpl in project storm by apache.

the class ExecutorShutdown method credentialsChanged.

@Override
public void credentialsChanged(Credentials credentials) {
    TupleImpl tuple = new TupleImpl(executor.getWorkerTopologyContext(), new Values(credentials), (int) Constants.SYSTEM_TASK_ID, Constants.CREDENTIALS_CHANGED_STREAM_ID);
    List<AddressedTuple> addressedTuple = Lists.newArrayList(new AddressedTuple(AddressedTuple.BROADCAST_DEST, tuple));
    executor.getReceiveQueue().publish(addressedTuple);
}
Also used : Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl) AddressedTuple(org.apache.storm.tuple.AddressedTuple)

Example 7 with TupleImpl

use of org.apache.storm.tuple.TupleImpl in project storm by apache.

the class BoltOutputCollectorImpl method ack.

@Override
public void ack(Tuple input) {
    long ackValue = ((TupleImpl) input).getAckVal();
    Map<Long, Long> anchorsToIds = input.getMessageId().getAnchorsToIds();
    for (Map.Entry<Long, Long> entry : anchorsToIds.entrySet()) {
        executor.sendUnanchored(taskData, Acker.ACKER_ACK_STREAM_ID, new Values(entry.getKey(), Utils.bitXor(entry.getValue(), ackValue)), executor.getExecutorTransfer());
    }
    long delta = tupleTimeDelta((TupleImpl) input);
    if (isDebug) {
        LOG.info("BOLT ack TASK: {} TIME: {} TUPLE: {}", taskId, delta, input);
    }
    BoltAckInfo boltAckInfo = new BoltAckInfo(input, taskId, delta);
    boltAckInfo.applyOn(taskData.getUserContext());
    if (delta != 0) {
        ((BoltExecutorStats) executor.getStats()).boltAckedTuple(input.getSourceComponent(), input.getSourceStreamId(), delta);
    }
}
Also used : BoltAckInfo(org.apache.storm.hooks.info.BoltAckInfo) BoltExecutorStats(org.apache.storm.stats.BoltExecutorStats) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl) HashMap(java.util.HashMap) Map(java.util.Map)

Example 8 with TupleImpl

use of org.apache.storm.tuple.TupleImpl in project storm by apache.

the class SpoutOutputCollectorImpl method sendSpoutMsg.

private List<Integer> sendSpoutMsg(String stream, List<Object> values, Object messageId, Integer outTaskId) {
    emittedCount.increment();
    List<Integer> outTasks;
    if (outTaskId != null) {
        outTasks = taskData.getOutgoingTasks(outTaskId, stream, values);
    } else {
        outTasks = taskData.getOutgoingTasks(stream, values);
    }
    List<Long> ackSeq = new ArrayList<>();
    boolean needAck = (messageId != null) && hasAckers;
    long rootId = MessageId.generateId(random);
    for (Integer t : outTasks) {
        MessageId msgId;
        if (needAck) {
            long as = MessageId.generateId(random);
            msgId = MessageId.makeRootId(rootId, as);
            ackSeq.add(as);
        } else {
            msgId = MessageId.makeUnanchored();
        }
        TupleImpl tuple = new TupleImpl(executor.getWorkerTopologyContext(), values, this.taskId, stream, msgId);
        executor.getExecutorTransfer().transfer(t, tuple);
    }
    if (isEventLoggers) {
        executor.sendToEventLogger(executor, taskData, values, executor.getComponentId(), messageId, random);
    }
    boolean sample = false;
    try {
        sample = executor.getSampler().call();
    } catch (Exception ignored) {
    }
    if (needAck) {
        TupleInfo info = new TupleInfo();
        info.setTaskId(this.taskId);
        info.setStream(stream);
        info.setMessageId(messageId);
        if (isDebug) {
            info.setValues(values);
        }
        if (sample) {
            info.setTimestamp(System.currentTimeMillis());
        }
        pending.put(rootId, info);
        List<Object> ackInitTuple = new Values(rootId, Utils.bitXorVals(ackSeq), this.taskId);
        executor.sendUnanchored(taskData, Acker.ACKER_INIT_STREAM_ID, ackInitTuple, executor.getExecutorTransfer());
    } else if (messageId != null) {
        TupleInfo info = new TupleInfo();
        info.setStream(stream);
        info.setValues(values);
        info.setMessageId(messageId);
        info.setTimestamp(0);
        Long timeDelta = sample ? 0L : null;
        info.setId("0:");
        executor.ackSpoutMsg(executor, taskData, timeDelta, info);
    }
    return outTasks;
}
Also used : TupleInfo(org.apache.storm.executor.TupleInfo) ArrayList(java.util.ArrayList) Values(org.apache.storm.tuple.Values) MutableLong(org.apache.storm.utils.MutableLong) TupleImpl(org.apache.storm.tuple.TupleImpl) MessageId(org.apache.storm.tuple.MessageId)

Example 9 with TupleImpl

use of org.apache.storm.tuple.TupleImpl in project storm by apache.

the class EsTestUtil method generateTestTuple.

public static Tuple generateTestTuple(String source, String index, String type, String id) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap(), new HashMap(), new HashMap(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("source", "index", "type", "id");
        }
    };
    return new TupleImpl(topologyContext, new Values(source, index, type, id), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 10 with TupleImpl

use of org.apache.storm.tuple.TupleImpl in project heron by twitter.

the class OutputCollectorImpl method emitDirect.

@Override
public void emitDirect(int taskId, String streamId, Collection<Tuple> anchors, List<Object> tuple) {
    if (anchors != null) {
        ArrayList<com.twitter.heron.api.tuple.Tuple> l = new ArrayList<com.twitter.heron.api.tuple.Tuple>();
        for (Tuple t : anchors) {
            TupleImpl i = (TupleImpl) t;
            l.add(i.getDelegate());
        }
        delegate.emitDirect(taskId, streamId, l, tuple);
    } else {
        delegate.emitDirect(taskId, streamId, (Collection<com.twitter.heron.api.tuple.Tuple>) null, tuple);
    }
}
Also used : ArrayList(java.util.ArrayList) TupleImpl(org.apache.storm.tuple.TupleImpl) Tuple(org.apache.storm.tuple.Tuple)

Aggregations

TupleImpl (org.apache.storm.tuple.TupleImpl)23 Values (org.apache.storm.tuple.Values)12 HashMap (java.util.HashMap)10 ArrayList (java.util.ArrayList)8 Fields (org.apache.storm.tuple.Fields)8 Config (org.apache.storm.Config)7 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)7 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)7 Tuple (org.apache.storm.tuple.Tuple)5 AddressedTuple (org.apache.storm.tuple.AddressedTuple)4 MessageId (org.apache.storm.tuple.MessageId)3 List (java.util.List)2 Map (java.util.Map)2 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 StormTimer (org.apache.storm.StormTimer)2 IOException (java.io.IOException)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 TupleInfo (org.apache.storm.executor.TupleInfo)1 BoltAckInfo (org.apache.storm.hooks.info.BoltAckInfo)1