Search in sources :

Example 6 with JStormUnitTestValidator

use of com.jstorm.example.unittests.utils.JStormUnitTestValidator in project jstorm by alibaba.

the class SlidingWindowTopologyTest method testSlidingWindowTopology.

@Test
public void testSlidingWindowTopology() {
    TopologyBuilder topologyBuilder = new TopologyBuilder();
    topologyBuilder.setSpout("spout", new SlidingWindowTestRandomSpout(SPOUT_LIMIT), 1);
    // the following bolt sums all the elements in the window. The window has length of 30 elements
    // and slide every 10 elements.
    // for example, if the spout generate 1, 2, 3, 4 ... then the SumBolt generate 55, 210, 465, 765 ...
    topologyBuilder.setBolt("sum", new SlidingWindowTestSumBolt().withWindow(new BaseWindowedBolt.Count(SUM_BOLT_WINDOW_LENGTH), new BaseWindowedBolt.Count(SUM_BOLT_WINDOW_SLIDE)), 1).shuffleGrouping("spout");
    // the following bolt calculate the average value of elements in the window. The window has length
    // of 3. So it generates the average of 3 elements and then wait for another 3 elements.
    topologyBuilder.setBolt("avg", new SlidingWindowTestAvgBolt().withTumblingWindow(new BaseWindowedBolt.Count(AVG_BOLT_WINDOW_LENGTH)), 1).shuffleGrouping("sum");
    Set<String> userDefineMetrics = new HashSet<String>();
    userDefineMetrics.add("SlidingWindowTopologyTest.SpoutAvgSum");
    userDefineMetrics.add("SlidingWindowTopologyTest.BoltAvgSum");
    Map config = new HashMap();
    config.put(Config.TOPOLOGY_NAME, "SlidingWindowTopologyTest");
    JStormUnitTestValidator validator = new JStormUnitTestMetricValidator(userDefineMetrics) {

        @Override
        public boolean validateMetrics(Map<String, Double> metrics) {
            int spoutAvgSum = (int) metrics.get("SlidingWindowTopologyTest.SpoutAvgSum").doubleValue();
            int boltAvgSum = (int) metrics.get("SlidingWindowTopologyTest.BoltAvgSum").doubleValue();
            System.out.println(spoutAvgSum + " " + boltAvgSum);
            assertEquals(spoutAvgSum, boltAvgSum);
            return true;
        }
    };
    JStormUnitTestRunner.submitTopology(topologyBuilder.createTopology(), config, 120, validator);
}
Also used : JStormUnitTestValidator(com.jstorm.example.unittests.utils.JStormUnitTestValidator) TopologyBuilder(backtype.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) JStormUnitTestMetricValidator(com.jstorm.example.unittests.utils.JStormUnitTestMetricValidator) Map(java.util.Map) HashMap(java.util.HashMap) BaseWindowedBolt(backtype.storm.topology.base.BaseWindowedBolt) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

JStormUnitTestValidator (com.jstorm.example.unittests.utils.JStormUnitTestValidator)6 Test (org.junit.Test)6 TopologyBuilder (backtype.storm.topology.TopologyBuilder)4 Map (java.util.Map)4 LocalDRPC (backtype.storm.LocalDRPC)3 JStormUnitTestDRPCValidator (com.jstorm.example.unittests.utils.JStormUnitTestDRPCValidator)3 JStormUnitTestMetricValidator (com.jstorm.example.unittests.utils.JStormUnitTestMetricValidator)3 Config (backtype.storm.Config)2 BaseWindowedBolt (backtype.storm.topology.base.BaseWindowedBolt)2 Fields (backtype.storm.tuple.Fields)2 Values (backtype.storm.tuple.Values)2 HashMap (java.util.HashMap)2 HashSet (java.util.HashSet)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 TridentState (storm.trident.TridentState)2 TridentTopology (storm.trident.TridentTopology)2 Count (storm.trident.operation.builtin.Count)2 FilterNull (storm.trident.operation.builtin.FilterNull)2 MapGet (storm.trident.operation.builtin.MapGet)2