Search in sources :

Example 76 with LocalCluster

use of org.apache.storm.LocalCluster in project storm by apache.

the class ResourceAwareExampleTopology method main.

public static void main(String[] args) throws Exception {
    TopologyBuilder builder = new TopologyBuilder();
    SpoutDeclarer spout = builder.setSpout("word", new TestWordSpout(), 5);
    //set cpu requirement
    spout.setCPULoad(20);
    //set onheap and offheap memory requirement
    spout.setMemoryLoad(64, 16);
    BoltDeclarer bolt1 = builder.setBolt("exclaim1", new ExclamationBolt(), 3).shuffleGrouping("word");
    //sets cpu requirement.  Not neccessary to set both CPU and memory.
    //For requirements not set, a default value will be used
    bolt1.setCPULoad(15);
    BoltDeclarer bolt2 = builder.setBolt("exclaim2", new ExclamationBolt(), 2).shuffleGrouping("exclaim1");
    bolt2.setMemoryLoad(100);
    Config conf = new Config();
    conf.setDebug(true);
    /**
     * Use to limit the maximum amount of memory (in MB) allocated to one worker process.
     * Can be used to spread executors to to multiple workers
     */
    conf.setTopologyWorkerMaxHeapSize(1024.0);
    //topology priority describing the importance of the topology in decreasing importance starting from 0 (i.e. 0 is the highest priority and the priority importance decreases as the priority number increases).
    //Recommended range of 0-29 but no hard limit set.
    conf.setTopologyPriority(29);
    // Set strategy to schedule topology. If not specified, default to org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy
    conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
    if (args != null && args.length > 0) {
        conf.setNumWorkers(3);
        StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
    } else {
        try (LocalCluster cluster = new LocalCluster();
            LocalTopology topo = cluster.submitTopology("test", conf, builder.createTopology())) {
            Utils.sleep(10000);
        }
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) BoltDeclarer(org.apache.storm.topology.BoltDeclarer) Config(org.apache.storm.Config) TestWordSpout(org.apache.storm.testing.TestWordSpout) SpoutDeclarer(org.apache.storm.topology.SpoutDeclarer) LocalTopology(org.apache.storm.LocalCluster.LocalTopology)

Aggregations

LocalCluster (org.apache.storm.LocalCluster)76 Config (org.apache.storm.Config)70 LocalTopology (org.apache.storm.LocalCluster.LocalTopology)52 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)28 Fields (org.apache.storm.tuple.Fields)22 Map (java.util.Map)14 StreamBuilder (org.apache.storm.streams.StreamBuilder)9 RandomIntegerSpout (org.apache.storm.starter.spout.RandomIntegerSpout)7 HashMap (java.util.HashMap)6 LocalDRPC (org.apache.storm.LocalDRPC)6 JedisPoolConfig (org.apache.storm.redis.common.config.JedisPoolConfig)6 TestWordSpout (org.apache.storm.testing.TestWordSpout)5 FileInputStream (java.io.FileInputStream)4 InputStream (java.io.InputStream)4 StormSubmitter (org.apache.storm.StormSubmitter)4 StormTopology (org.apache.storm.generated.StormTopology)4 ValueMapper (org.apache.storm.streams.operations.mappers.ValueMapper)4 TopologyContext (org.apache.storm.task.TopologyContext)4 Utils (org.apache.storm.utils.Utils)4 Yaml (org.yaml.snakeyaml.Yaml)4