Search in sources :

Example 16 with BatchGroupId

use of com.alibaba.jstorm.transactional.BatchGroupId in project jstorm by alibaba.

the class TransactionOutputCollector method flushBarrier.

public void flushBarrier() {
    // flush pending message in outputCollector
    flush();
    // Emit and flush barrier message to all downstream tasks
    for (Entry<Integer, CountValue> entry : currBatchTracker.sendMsgCount.entrySet()) {
        int taskId = entry.getKey();
        CountValue count = entry.getValue();
        BatchSnapshot snapshot = new BatchSnapshot(currBatchTracker.getBatchGroupId(), count.count);
        emitDirectByDelegate(taskId, TransactionCommon.BARRIER_STREAM_ID, null, new Values(new BatchGroupId(currBatchTracker.getBatchGroupId()), snapshot), null);
        count.count = 0;
    }
    flush();
}
Also used : BatchGroupId(com.alibaba.jstorm.transactional.BatchGroupId) CountValue(com.alibaba.jstorm.transactional.bolt.TransactionBolt.CountValue) Values(backtype.storm.tuple.Values) BatchSnapshot(com.alibaba.jstorm.transactional.BatchSnapshot)

Example 17 with BatchGroupId

use of com.alibaba.jstorm.transactional.BatchGroupId in project jstorm by alibaba.

the class TransactionOutputCollector method emitDirect.

@Override
public void emitDirect(int taskId, String streamId, Collection<Tuple> anchors, List<Object> tuple, ICollectorCallback callback) {
    List<Object> tupleWithId = new ArrayList<Object>();
    tupleWithId.add(new BatchGroupId(currBatchTracker.getBatchGroupId()));
    tupleWithId.addAll(tuple);
    delegate.emitDirect(taskId, streamId, anchors, tupleWithId, callback);
}
Also used : BatchGroupId(com.alibaba.jstorm.transactional.BatchGroupId) ArrayList(java.util.ArrayList)

Example 18 with BatchGroupId

use of com.alibaba.jstorm.transactional.BatchGroupId in project jstorm by alibaba.

the class TransactionStatefulBolt method rollback.

@Override
protected void rollback(TransactionState state) {
    LOG.info("Start to rollback, state={}, currentBatchStatus={}", state, currentBatchStatusInfo());
    boltExecutor.rollBack(Utils.maybe_deserialize((byte[]) state.getUserCheckpoint()));
    BatchGroupId batchGroupId = state.getCurrBatchGroupId();
    lastSuccessfulBatch.put(batchGroupId.groupId, batchGroupId.batchId);
    cleanupBuffer(state.getCurrBatchGroupId().groupId);
}
Also used : BatchGroupId(com.alibaba.jstorm.transactional.BatchGroupId)

Aggregations

BatchGroupId (com.alibaba.jstorm.transactional.BatchGroupId)18 Values (backtype.storm.tuple.Values)6 ArrayList (java.util.ArrayList)5 TopoMasterCtrlEvent (com.alibaba.jstorm.task.master.ctrlevent.TopoMasterCtrlEvent)4 BatchSnapshot (com.alibaba.jstorm.transactional.BatchSnapshot)3 TransactionState (com.alibaba.jstorm.transactional.state.TransactionState)3 InvalidTopologyException (backtype.storm.generated.InvalidTopologyException)2 ICollectorCallback (backtype.storm.task.ICollectorCallback)2 BoltCollector (com.alibaba.jstorm.task.execute.BoltCollector)2 Tuple (backtype.storm.tuple.Tuple)1 CountValue (com.alibaba.jstorm.transactional.bolt.TransactionBolt.CountValue)1 BatchInfo (com.alibaba.jstorm.transactional.spout.TransactionSpoutOutputCollector.BatchInfo)1 Pair (com.alibaba.jstorm.utils.Pair)1 HashSet (java.util.HashSet)1