Search in sources :

Example 81 with Tuple

use of org.apache.storm.tuple.Tuple in project storm by apache.

the class RollingCountBoltTest method shouldEmitSomethingIfAtLeastOneObjectWasCountedAndTickTupleIsReceived.

@SuppressWarnings("rawtypes")
@Test
public void shouldEmitSomethingIfAtLeastOneObjectWasCountedAndTickTupleIsReceived() {
    // given
    Tuple normalTuple = mockNormalTuple(new Object());
    Tuple tickTuple = MockTupleHelpers.mockTickTuple();
    RollingCountBolt bolt = new RollingCountBolt();
    Map conf = mock(Map.class);
    TopologyContext context = mock(TopologyContext.class);
    OutputCollector collector = mock(OutputCollector.class);
    bolt.prepare(conf, context, collector);
    // when
    bolt.execute(normalTuple);
    bolt.execute(tickTuple);
    // then
    verify(collector).emit(any(Values.class));
}
Also used : OutputCollector(org.apache.storm.task.OutputCollector) Values(org.apache.storm.tuple.Values) TopologyContext(org.apache.storm.task.TopologyContext) Map(java.util.Map) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test)

Example 82 with Tuple

use of org.apache.storm.tuple.Tuple in project storm by apache.

the class TotalRankingsBoltTest method mockRankingsTuple.

private Tuple mockRankingsTuple(Object obj, long count) {
    Tuple tuple = MockTupleHelpers.mockTuple(ANY_NON_SYSTEM_COMPONENT_ID, ANY_NON_SYSTEM_STREAM_ID);
    Rankings rankings = mock(Rankings.class);
    when(tuple.getValue(0)).thenReturn(rankings);
    return tuple;
}
Also used : Rankings(org.apache.storm.starter.tools.Rankings) Tuple(org.apache.storm.tuple.Tuple)

Example 83 with Tuple

use of org.apache.storm.tuple.Tuple in project storm by apache.

the class TotalRankingsBoltTest method shouldEmitSomethingIfTickTupleIsReceived.

@Test
public void shouldEmitSomethingIfTickTupleIsReceived() {
    // given
    Tuple tickTuple = MockTupleHelpers.mockTickTuple();
    BasicOutputCollector collector = mock(BasicOutputCollector.class);
    TotalRankingsBolt bolt = new TotalRankingsBolt();
    // when
    bolt.execute(tickTuple, collector);
    // then
    // verifyZeroInteractions(collector);
    verify(collector).emit(any(Values.class));
}
Also used : Values(org.apache.storm.tuple.Values) BasicOutputCollector(org.apache.storm.topology.BasicOutputCollector) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test)

Example 84 with Tuple

use of org.apache.storm.tuple.Tuple in project storm by apache.

the class SingleJoinBolt method execute.

@Override
public void execute(Tuple tuple) {
    List<Object> id = tuple.select(_idFields);
    GlobalStreamId streamId = new GlobalStreamId(tuple.getSourceComponent(), tuple.getSourceStreamId());
    if (!_pending.containsKey(id)) {
        _pending.put(id, new HashMap<GlobalStreamId, Tuple>());
    }
    Map<GlobalStreamId, Tuple> parts = _pending.get(id);
    if (parts.containsKey(streamId))
        throw new RuntimeException("Received same side of single join twice");
    parts.put(streamId, tuple);
    if (parts.size() == _numSources) {
        _pending.remove(id);
        List<Object> joinResult = new ArrayList<Object>();
        for (String outField : _outFields) {
            GlobalStreamId loc = _fieldLocations.get(outField);
            joinResult.add(parts.get(loc).getValueByField(outField));
        }
        _collector.emit(new ArrayList<Tuple>(parts.values()), joinResult);
        for (Tuple part : parts.values()) {
            _collector.ack(part);
        }
    }
}
Also used : GlobalStreamId(org.apache.storm.generated.GlobalStreamId) Tuple(org.apache.storm.tuple.Tuple)

Example 85 with Tuple

use of org.apache.storm.tuple.Tuple in project storm by apache.

the class SlidingWindowSumBolt method execute.

@Override
public void execute(TupleWindow inputWindow) {
    /*
             * The inputWindow gives a view of
             * (a) all the events in the window
             * (b) events that expired since last activation of the window
             * (c) events that newly arrived since last activation of the window
             */
    List<Tuple> tuplesInWindow = inputWindow.get();
    List<Tuple> newTuples = inputWindow.getNew();
    List<Tuple> expiredTuples = inputWindow.getExpired();
    LOG.debug("Events in current window: " + tuplesInWindow.size());
    /*
             * Instead of iterating over all the tuples in the window to compute
             * the sum, the values for the new events are added and old events are
             * subtracted. Similar optimizations might be possible in other
             * windowing computations.
             */
    for (Tuple tuple : newTuples) {
        sum += (int) tuple.getValue(0);
    }
    for (Tuple tuple : expiredTuples) {
        sum -= (int) tuple.getValue(0);
    }
    collector.emit(new Values(sum));
}
Also used : Values(org.apache.storm.tuple.Values) Tuple(org.apache.storm.tuple.Tuple)

Aggregations

Tuple (org.apache.storm.tuple.Tuple)85 Test (org.junit.Test)30 Fields (org.apache.storm.tuple.Fields)13 OutputCollector (org.apache.storm.task.OutputCollector)11 Values (org.apache.storm.tuple.Values)11 ArrayList (java.util.ArrayList)10 HiveOptions (org.apache.storm.hive.common.HiveOptions)10 TupleWindow (org.apache.storm.windowing.TupleWindow)9 HashMap (java.util.HashMap)7 Test (org.testng.annotations.Test)7 GlobalStreamId (org.apache.storm.generated.GlobalStreamId)6 DelimitedRecordHiveMapper (org.apache.storm.hive.bolt.mapper.DelimitedRecordHiveMapper)6 HashSet (java.util.HashSet)5 JsonRecordHiveMapper (org.apache.storm.hive.bolt.mapper.JsonRecordHiveMapper)5 TopologyContext (org.apache.storm.task.TopologyContext)5 TupleImpl (org.apache.storm.tuple.TupleImpl)5 BasicOutputCollector (org.apache.storm.topology.BasicOutputCollector)4 Map (java.util.Map)3 Callback (org.apache.kafka.clients.producer.Callback)3 ProducerRecord (org.apache.kafka.clients.producer.ProducerRecord)3