Search in sources :

Example 1 with MkClusterParam

use of backtype.storm.testing.MkClusterParam in project storm-lib by xumingming.

the class TestingApiDemo method testBasicTopology.

public void testBasicTopology() {
    MkClusterParam mkClusterParam = new MkClusterParam();
    mkClusterParam.setSupervisors(4);
    Config daemonConf = new Config();
    daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
    mkClusterParam.setDaemonConf(daemonConf);
    /**
		 * This is a combination of <code>Testing.withLocalCluster</code> and <code>Testing.withSimulatedTime</code>.
		 */
    Testing.withSimulatedTimeLocalCluster(mkClusterParam, new TestJob() {

        @Override
        public void run(ILocalCluster cluster) {
            // build the test topology
            TopologyBuilder builder = new TopologyBuilder();
            builder.setSpout("1", new TestWordSpout(true), 3);
            builder.setBolt("2", new TestWordCounter(), 4).fieldsGrouping("1", new Fields("word"));
            builder.setBolt("3", new TestGlobalCount()).globalGrouping("1");
            builder.setBolt("4", new TestAggregatesCounter()).globalGrouping("2");
            StormTopology topology = builder.createTopology();
            // complete the topology
            // prepare the mock data
            MockedSources mockedSources = new MockedSources();
            mockedSources.addMockData("1", new Values("nathan"), new Values("bob"), new Values("joey"), new Values("nathan"));
            // prepare the config
            Config conf = new Config();
            conf.setNumWorkers(2);
            CompleteTopologyParam completeTopologyParam = new CompleteTopologyParam();
            completeTopologyParam.setMockedSources(mockedSources);
            completeTopologyParam.setStormConf(conf);
            /**
				 * TODO
				 */
            Map result = Testing.completeTopology(cluster, topology, completeTopologyParam);
            // check whether the result is right
            assertTrue(Testing.multiseteq(new Values(new Values("nathan"), new Values("bob"), new Values("joey"), new Values("nathan")), Testing.readTuples(result, "1")));
            assertTrue(Testing.multiseteq(new Values(new Values("nathan", 1), new Values("nathan", 2), new Values("bob", 1), new Values("joey", 1)), Testing.readTuples(result, "2")));
            assertTrue(Testing.multiseteq(new Values(new Values(1), new Values(2), new Values(3), new Values(4)), Testing.readTuples(result, "3")));
            assertTrue(Testing.multiseteq(new Values(new Values(1), new Values(2), new Values(3), new Values(4)), Testing.readTuples(result, "4")));
        }
    });
}
Also used : TestJob(backtype.storm.testing.TestJob) TopologyBuilder(backtype.storm.topology.TopologyBuilder) Config(backtype.storm.Config) StormTopology(backtype.storm.generated.StormTopology) Values(backtype.storm.tuple.Values) TestWordCounter(backtype.storm.testing.TestWordCounter) TestAggregatesCounter(backtype.storm.testing.TestAggregatesCounter) MkClusterParam(backtype.storm.testing.MkClusterParam) ILocalCluster(backtype.storm.ILocalCluster) MockedSources(backtype.storm.testing.MockedSources) Fields(backtype.storm.tuple.Fields) TestGlobalCount(backtype.storm.testing.TestGlobalCount) CompleteTopologyParam(backtype.storm.testing.CompleteTopologyParam) TestWordSpout(backtype.storm.testing.TestWordSpout) Map(java.util.Map)

Example 2 with MkClusterParam

use of backtype.storm.testing.MkClusterParam in project storm-lib by xumingming.

the class TestingApiDemo method testWithLocalCluster.

public void testWithLocalCluster() {
    MkClusterParam mkClusterParam = new MkClusterParam();
    mkClusterParam.setSupervisors(2);
    mkClusterParam.setPortsPerSupervisor(5);
    Config daemonConf = new Config();
    daemonConf.put(Config.SUPERVISOR_ENABLE, false);
    daemonConf.put(Config.TOPOLOGY_ACKER_EXECUTORS, 0);
    /**
		 * when testing your topology, you need a <code>LocalCluster</code> to run your topologies, you need
		 * to create it, after using it, you need to stop it. Using <code>Testing.withLocalCluster</code> you
		 * don't need to do any of this, just use the <code>cluster</code> provided through the param of 
		 * <code>TestJob.run</code>.
		 */
    Testing.withLocalCluster(mkClusterParam, new TestJob() {

        @Override
        public void run(ILocalCluster cluster) {
            assertNotNull(cluster.getState());
        }
    });
}
Also used : ILocalCluster(backtype.storm.ILocalCluster) TestJob(backtype.storm.testing.TestJob) Config(backtype.storm.Config) MkClusterParam(backtype.storm.testing.MkClusterParam)

Example 3 with MkClusterParam

use of backtype.storm.testing.MkClusterParam in project storm-lib by xumingming.

the class TestingApiDemo method testTimeout.

public void testTimeout() {
    Config daemonConfig = new Config();
    daemonConfig.put(Config.TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS, true);
    MkClusterParam mkClusterParam = new MkClusterParam();
    mkClusterParam.setDaemonConf(daemonConfig);
    Testing.withSimulatedTimeLocalCluster(mkClusterParam, new TestJob() {

        @Override
        public void run(ILocalCluster cluster) {
            AckFailMapTracker tracker = new AckFailMapTracker();
            FeederSpout feeder = createFeederSpout("field1");
            feeder.setAckFailDelegate(tracker);
            TopologyBuilder builder = new TopologyBuilder();
            builder.setSpout("1", feeder);
            builder.setBolt("2", new AckEveryOtherBolt()).globalGrouping("1");
            StormTopology topology = builder.createTopology();
            Config topologyConfig = new Config();
            topologyConfig.setMessageTimeoutSecs(10);
            /**
				 * TODO
				 */
            try {
                cluster.submitTopology("timeout-tester", topologyConfig, topology);
            } catch (AlreadyAliveException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            } catch (InvalidTopologyException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            feeder.feed(new Values("a"), 1);
            feeder.feed(new Values("b"), 2);
            feeder.feed(new Values("c"), 3);
            /**
				 * TODO
				 */
            Testing.advanceClusterTime(cluster, 9);
            assertAcked(tracker, 1, 3);
            assertFalse(tracker.isFailed(2));
            Testing.advanceClusterTime(cluster, 12);
            assertFailed(tracker, 2);
        }
    });
}
Also used : TestJob(backtype.storm.testing.TestJob) TopologyBuilder(backtype.storm.topology.TopologyBuilder) Config(backtype.storm.Config) AckFailMapTracker(backtype.storm.testing.AckFailMapTracker) StormTopology(backtype.storm.generated.StormTopology) InvalidTopologyException(backtype.storm.generated.InvalidTopologyException) Values(backtype.storm.tuple.Values) AlreadyAliveException(backtype.storm.generated.AlreadyAliveException) MkClusterParam(backtype.storm.testing.MkClusterParam) ILocalCluster(backtype.storm.ILocalCluster) FeederSpout(backtype.storm.testing.FeederSpout)

Aggregations

Config (backtype.storm.Config)3 ILocalCluster (backtype.storm.ILocalCluster)3 MkClusterParam (backtype.storm.testing.MkClusterParam)3 TestJob (backtype.storm.testing.TestJob)3 StormTopology (backtype.storm.generated.StormTopology)2 TopologyBuilder (backtype.storm.topology.TopologyBuilder)2 Values (backtype.storm.tuple.Values)2 AlreadyAliveException (backtype.storm.generated.AlreadyAliveException)1 InvalidTopologyException (backtype.storm.generated.InvalidTopologyException)1 AckFailMapTracker (backtype.storm.testing.AckFailMapTracker)1 CompleteTopologyParam (backtype.storm.testing.CompleteTopologyParam)1 FeederSpout (backtype.storm.testing.FeederSpout)1 MockedSources (backtype.storm.testing.MockedSources)1 TestAggregatesCounter (backtype.storm.testing.TestAggregatesCounter)1 TestGlobalCount (backtype.storm.testing.TestGlobalCount)1 TestWordCounter (backtype.storm.testing.TestWordCounter)1 TestWordSpout (backtype.storm.testing.TestWordSpout)1 Fields (backtype.storm.tuple.Fields)1 Map (java.util.Map)1