Search in sources :

Example 16 with Config

use of backtype.storm.Config in project jstorm by alibaba.

the class FluxBuilder method buildExternalTopology.

private static StormTopology buildExternalTopology(ObjectDef def, ExecutionContext context) throws ClassNotFoundException, IllegalAccessException, InstantiationException, NoSuchMethodException, InvocationTargetException {
    Object topologySource = buildObject(def, context);
    String methodName = context.getTopologyDef().getTopologySource().getMethodName();
    Method getTopology = findGetTopologyMethod(topologySource, methodName);
    if (getTopology.getParameterTypes()[0].equals(Config.class)) {
        Config config = new Config();
        config.putAll(context.getTopologyDef().getConfig());
        return (StormTopology) getTopology.invoke(topologySource, config);
    } else {
        return (StormTopology) getTopology.invoke(topologySource, context.getTopologyDef().getConfig());
    }
}
Also used : Config(backtype.storm.Config) StormTopology(backtype.storm.generated.StormTopology)

Example 17 with Config

use of backtype.storm.Config in project jstorm by alibaba.

the class FluxBuilder method buildConfig.

/**
     * Given a topology definition, return a populated `org.apache.storm.Config` instance.
     *
     * @param topologyDef
     * @return
     */
public static Config buildConfig(TopologyDef topologyDef) {
    // merge contents of `config` into topology config
    Config conf = new Config();
    conf.putAll(topologyDef.getConfig());
    return conf;
}
Also used : Config(backtype.storm.Config)

Example 18 with Config

use of backtype.storm.Config in project jstorm by alibaba.

the class TestTridentTopology method main.

public static void main(String[] args) throws Exception {
    if (args.length == 0) {
        System.err.println("Please input configuration file");
        System.exit(-1);
    }
    Map conf = LoadConfig.LoadConf(args[0]);
    if (conf == null) {
        LOG.error("Failed to load config");
    } else {
        Config config = new Config();
        config.putAll(conf);
        config.setMaxSpoutPending(10);
        config.put(LoadConfig.TOPOLOGY_TYPE, "Trident");
        StormSubmitter.submitTopology("WordCount", config, buildTopology());
    }
}
Also used : Config(backtype.storm.Config) HashMap(java.util.HashMap) Map(java.util.Map)

Example 19 with Config

use of backtype.storm.Config in project storm-signals by ptgoetz.

the class SignalTopology method main.

/**
     * @param args
     */
public static void main(String[] args) {
    // TODO Auto-generated method stub
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("signal-spout", new TestSignalSpout("test-signal-spout"));
    Config conf = new Config();
    conf.setDebug(true);
    LocalCluster cluster = new LocalCluster();
    cluster.submitTopology("test", conf, builder.createTopology());
    Utils.sleep(120000);
    cluster.killTopology("test");
    cluster.shutdown();
}
Also used : LocalCluster(backtype.storm.LocalCluster) TopologyBuilder(backtype.storm.topology.TopologyBuilder) Config(backtype.storm.Config)

Example 20 with Config

use of backtype.storm.Config in project avro-kafka-storm by ransilberman.

the class AvroTopology method main.

public static void main(String[] args) throws AlreadyAliveException, InvalidTopologyException {
    //Add those lines to prevent too much logging noise in the console
    Logger.getLogger("storm.kafka.PartitionManager").setLevel(Level.ERROR);
    Logger.getLogger("backtype.storm").setLevel(Level.ERROR);
    Logger.getLogger("storm.kafka").setLevel(Level.ERROR);
    TopologyBuilder builder = new TopologyBuilder();
    int partitions = 1;
    final String offsetPath = "/liveperson-avro-test";
    final String consumerId = "v1";
    final String topic = "avro-test";
    List<String> hosts = new ArrayList<String>();
    hosts.add("tlvwhale1");
    hosts.add("tlvwhale2");
    hosts.add("tlvwhale3");
    SpoutConfig kafkaConfig = new SpoutConfig(KafkaConfig.StaticHosts.fromHostString(hosts, partitions), topic, offsetPath, consumerId);
    KafkaSpout kafkaSpout = new KafkaSpout(kafkaConfig);
    builder.setSpout("spout", kafkaSpout);
    builder.setBolt("bolt", new AvroBolt()).shuffleGrouping("spout");
    Config conf = new Config();
    conf.setDebug(true);
    if (args != null && args.length > 0) {
        conf.setNumWorkers(3);
        StormSubmitter.submitTopology(args[0], conf, builder.createTopology());
    } else {
        LocalCluster cluster = new LocalCluster();
        cluster.submitTopology("avroTopology", conf, builder.createTopology());
        Utils.sleep(100000);
        cluster.killTopology("avroTopology");
        cluster.shutdown();
    }
}
Also used : LocalCluster(backtype.storm.LocalCluster) TopologyBuilder(backtype.storm.topology.TopologyBuilder) KafkaConfig(storm.kafka.KafkaConfig) SpoutConfig(storm.kafka.SpoutConfig) Config(backtype.storm.Config) SpoutConfig(storm.kafka.SpoutConfig) ArrayList(java.util.ArrayList) KafkaSpout(storm.kafka.KafkaSpout)

Aggregations

Config (backtype.storm.Config)75 TopologyBuilder (backtype.storm.topology.TopologyBuilder)38 LocalCluster (backtype.storm.LocalCluster)31 Fields (backtype.storm.tuple.Fields)19 Test (org.junit.Test)8 StormTopology (backtype.storm.generated.StormTopology)7 Map (java.util.Map)7 LocalDRPC (backtype.storm.LocalDRPC)6 ArrayList (java.util.ArrayList)4 ILocalCluster (backtype.storm.ILocalCluster)3 TupleTableConfig (backtype.storm.contrib.hbase.utils.TupleTableConfig)3 LinearDRPCTopologyBuilder (backtype.storm.drpc.LinearDRPCTopologyBuilder)3 MkClusterParam (backtype.storm.testing.MkClusterParam)3 TestJob (backtype.storm.testing.TestJob)3 TransactionalTopologyBuilder (backtype.storm.transactional.TransactionalTopologyBuilder)3 Values (backtype.storm.tuple.Values)3 TridentConfig (backtype.storm.contrib.hbase.utils.TridentConfig)2 DRPCSpout (backtype.storm.drpc.DRPCSpout)2 ReturnResults (backtype.storm.drpc.ReturnResults)2 FeederSpout (backtype.storm.testing.FeederSpout)2