Search in sources :

Example 6 with OutputCollector

use of backtype.storm.task.OutputCollector in project storm by nathanmarz.

the class CoordinatedBolt method prepare.

public void prepare(Map config, TopologyContext context, OutputCollector collector) {
    TimeCacheMap.ExpiredCallback<Object, TrackingInfo> callback = null;
    if (_delegate instanceof TimeoutCallback) {
        callback = new TimeoutItems();
    }
    _tracked = new TimeCacheMap<Object, TrackingInfo>(context.maxTopologyMessageTimeout(), callback);
    _collector = collector;
    _delegate.prepare(config, context, new OutputCollector(new CoordinatedOutputCollector(collector)));
    for (String component : Utils.get(context.getThisTargets(), Constants.COORDINATED_STREAM_ID, new HashMap<String, Grouping>()).keySet()) {
        for (Integer task : context.getComponentTasks(component)) {
            _countOutTasks.add(task);
        }
    }
    if (!_sourceArgs.isEmpty()) {
        _numSourceReports = 0;
        for (Entry<String, SourceArgs> entry : _sourceArgs.entrySet()) {
            if (entry.getValue().singleCount) {
                _numSourceReports += 1;
            } else {
                _numSourceReports += context.getComponentTasks(entry.getKey()).size();
            }
        }
    }
}
Also used : IOutputCollector(backtype.storm.task.IOutputCollector) OutputCollector(backtype.storm.task.OutputCollector) HashMap(java.util.HashMap) TimeCacheMap(backtype.storm.utils.TimeCacheMap)

Example 7 with OutputCollector

use of backtype.storm.task.OutputCollector in project pulsar by yahoo.

the class PulsarBoltTest method setup.

@Override
protected void setup() throws Exception {
    super.internalSetup();
    super.producerBaseSetup();
    pulsarBoltConf = new PulsarBoltConfiguration();
    pulsarBoltConf.setServiceUrl(serviceUrl);
    pulsarBoltConf.setTopic(topic);
    pulsarBoltConf.setTupleToMessageMapper(tupleToMessageMapper);
    pulsarBoltConf.setMetricsTimeIntervalInSecs(60);
    bolt = new PulsarBolt(pulsarBoltConf, new ClientConfiguration());
    mockCollector = new MockOutputCollector();
    OutputCollector collector = new OutputCollector(mockCollector);
    TopologyContext context = mock(TopologyContext.class);
    when(context.getThisComponentId()).thenReturn("test-bolt-" + methodName);
    when(context.getThisTaskId()).thenReturn(0);
    bolt.prepare(Maps.newHashMap(), context, collector);
    consumer = pulsarClient.subscribe(topic, subscriptionName);
}
Also used : OutputCollector(backtype.storm.task.OutputCollector) TopologyContext(backtype.storm.task.TopologyContext) ClientConfiguration(com.yahoo.pulsar.client.api.ClientConfiguration)

Aggregations

OutputCollector (backtype.storm.task.OutputCollector)7 IOutputCollector (backtype.storm.task.IOutputCollector)4 HashMap (java.util.HashMap)4 BatchOutputCollector (backtype.storm.coordination.BatchOutputCollector)2 BatchOutputCollectorImpl (backtype.storm.coordination.BatchOutputCollectorImpl)2 TopologyContext (backtype.storm.task.TopologyContext)2 TimeCacheMap (backtype.storm.utils.TimeCacheMap)2 ClientConfiguration (com.yahoo.pulsar.client.api.ClientConfiguration)2 GlobalStreamId (backtype.storm.generated.GlobalStreamId)1 InvalidTopologyException (backtype.storm.generated.InvalidTopologyException)1 SerializationFactory (backtype.storm.serialization.SerializationFactory)1 RotatingMap (backtype.storm.utils.RotatingMap)1 BatchCache (com.alibaba.jstorm.transactional.BatchCache)1 Input (com.esotericsoftware.kryo.io.Input)1 PersistentTopicStats (com.yahoo.pulsar.common.policies.data.PersistentTopicStats)1 Map (java.util.Map)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 Test (org.testng.annotations.Test)1