Search in sources :

Example 1 with Count

use of org.apache.storm.streams.operations.aggregators.Count in project storm by apache.

the class StreamBuilderTest method testRepartition.

@Test
public void testRepartition() throws Exception {
    Stream<String> stream = streamBuilder.newStream(newSpout(Utils.DEFAULT_STREAM_ID), new ValueMapper<>(0));
    stream.repartition(3).filter(x -> true).repartition(2).filter(x -> true).aggregate(new Count<>());
    StormTopology topology = streamBuilder.build();
    assertEquals(1, topology.get_spouts_size());
    SpoutSpec spout = topology.get_spouts().get("spout1");
    assertEquals(4, topology.get_bolts_size());
    Bolt bolt1 = topology.get_bolts().get("bolt1");
    Bolt bolt2 = topology.get_bolts().get("bolt2");
    Bolt bolt3 = topology.get_bolts().get("bolt3");
    Bolt bolt4 = topology.get_bolts().get("bolt4");
    assertEquals(1, spout.get_common().get_parallelism_hint());
    assertEquals(1, bolt1.get_common().get_parallelism_hint());
    assertEquals(3, bolt2.get_common().get_parallelism_hint());
    assertEquals(2, bolt3.get_common().get_parallelism_hint());
    assertEquals(2, bolt4.get_common().get_parallelism_hint());
}
Also used : OutputFieldsDeclarer(org.apache.storm.topology.OutputFieldsDeclarer) BaseRichSpout(org.apache.storm.topology.base.BaseRichSpout) IRichSpout(org.apache.storm.topology.IRichSpout) BaseWindowedBolt(org.apache.storm.topology.base.BaseWindowedBolt) TopologyContext(org.apache.storm.task.TopologyContext) HashMap(java.util.HashMap) Count(org.apache.storm.streams.operations.aggregators.Count) Bolt(org.apache.storm.generated.Bolt) Tuple(org.apache.storm.tuple.Tuple) OutputCollector(org.apache.storm.task.OutputCollector) StormTopology(org.apache.storm.generated.StormTopology) Map(java.util.Map) BranchProcessor(org.apache.storm.streams.processors.BranchProcessor) ValueMapper(org.apache.storm.streams.operations.mappers.ValueMapper) Before(org.junit.Before) BaseRichBolt(org.apache.storm.topology.base.BaseRichBolt) PairValueMapper(org.apache.storm.streams.operations.mappers.PairValueMapper) Grouping(org.apache.storm.generated.Grouping) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Fields(org.apache.storm.tuple.Fields) Utils(org.apache.storm.utils.Utils) GlobalStreamId(org.apache.storm.generated.GlobalStreamId) Mockito(org.mockito.Mockito) TumblingWindows(org.apache.storm.streams.windowing.TumblingWindows) SpoutSpec(org.apache.storm.generated.SpoutSpec) IRichBolt(org.apache.storm.topology.IRichBolt) NullStruct(org.apache.storm.generated.NullStruct) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) Collections(java.util.Collections) Assert.assertEquals(org.junit.Assert.assertEquals) SpoutSpec(org.apache.storm.generated.SpoutSpec) StormTopology(org.apache.storm.generated.StormTopology) BaseWindowedBolt(org.apache.storm.topology.base.BaseWindowedBolt) Bolt(org.apache.storm.generated.Bolt) BaseRichBolt(org.apache.storm.topology.base.BaseRichBolt) IRichBolt(org.apache.storm.topology.IRichBolt) Test(org.junit.Test)

Example 2 with Count

use of org.apache.storm.streams.operations.aggregators.Count in project storm by apache.

the class WindowedProcessorBoltTest method testEmit.

@Test
public void testEmit() throws Exception {
    Window<?, ?> window = TumblingWindows.of(BaseWindowedBolt.Count.of(2));
    setUpWindowedProcessorBolt(new AggregateProcessor<>(new Count<>()), window);
    bolt.execute(getMockTupleWindow(mockTuple1, mockTuple2, mockTuple3));
    ArgumentCaptor<Values> values = ArgumentCaptor.forClass(Values.class);
    ArgumentCaptor<String> os = ArgumentCaptor.forClass(String.class);
    Mockito.verify(mockOutputCollector, Mockito.times(2)).emit(os.capture(), values.capture());
    assertEquals("outputstream", os.getAllValues().get(0));
    assertEquals(new Values(3L), values.getAllValues().get(0));
    assertEquals("outputstream__punctuation", os.getAllValues().get(1));
    assertEquals(new Values(WindowNode.PUNCTUATION), values.getAllValues().get(1));
}
Also used : Values(org.apache.storm.tuple.Values) Count(org.apache.storm.streams.operations.aggregators.Count) Test(org.junit.Test)

Aggregations

Count (org.apache.storm.streams.operations.aggregators.Count)2 Test (org.junit.Test)2 Collections (java.util.Collections)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 Bolt (org.apache.storm.generated.Bolt)1 GlobalStreamId (org.apache.storm.generated.GlobalStreamId)1 Grouping (org.apache.storm.generated.Grouping)1 NullStruct (org.apache.storm.generated.NullStruct)1 SpoutSpec (org.apache.storm.generated.SpoutSpec)1 StormTopology (org.apache.storm.generated.StormTopology)1 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)1 PairValueMapper (org.apache.storm.streams.operations.mappers.PairValueMapper)1 ValueMapper (org.apache.storm.streams.operations.mappers.ValueMapper)1 BranchProcessor (org.apache.storm.streams.processors.BranchProcessor)1 TumblingWindows (org.apache.storm.streams.windowing.TumblingWindows)1 OutputCollector (org.apache.storm.task.OutputCollector)1 TopologyContext (org.apache.storm.task.TopologyContext)1 IRichBolt (org.apache.storm.topology.IRichBolt)1 IRichSpout (org.apache.storm.topology.IRichSpout)1