Search in sources :

Example 1 with DefaultTopicSelector

use of org.apache.storm.rocketmq.common.selector.DefaultTopicSelector in project storm by apache.

the class WordCountTopology method buildTopology.

public static StormTopology buildTopology(String nameserverAddr, String topic) {
    Properties properties = new Properties();
    properties.setProperty(SpoutConfig.NAME_SERVER_ADDR, nameserverAddr);
    properties.setProperty(SpoutConfig.CONSUMER_GROUP, CONSUMER_GROUP);
    properties.setProperty(SpoutConfig.CONSUMER_TOPIC, CONSUMER_TOPIC);
    RocketMqSpout spout = new RocketMqSpout(properties);
    TupleToMessageMapper mapper = new FieldNameBasedTupleToMessageMapper("word", "count");
    TopicSelector selector = new DefaultTopicSelector(topic);
    properties = new Properties();
    properties.setProperty(RocketMqConfig.NAME_SERVER_ADDR, nameserverAddr);
    RocketMqBolt insertBolt = new RocketMqBolt().withMapper(mapper).withSelector(selector).withProperties(properties);
    // wordSpout ==> countBolt ==> insertBolt
    TopologyBuilder builder = new TopologyBuilder();
    WordCounter bolt = new WordCounter();
    builder.setSpout(WORD_SPOUT, spout, 1);
    builder.setBolt(COUNT_BOLT, bolt, 1).fieldsGrouping(WORD_SPOUT, new Fields("str"));
    builder.setBolt(INSERT_BOLT, insertBolt, 1).shuffleGrouping(COUNT_BOLT);
    return builder.createTopology();
}
Also used : FieldNameBasedTupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper) Fields(org.apache.storm.tuple.Fields) TupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.TupleToMessageMapper) FieldNameBasedTupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper) RocketMqBolt(org.apache.storm.rocketmq.bolt.RocketMqBolt) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) TopicSelector(org.apache.storm.rocketmq.common.selector.TopicSelector) DefaultTopicSelector(org.apache.storm.rocketmq.common.selector.DefaultTopicSelector) RocketMqSpout(org.apache.storm.rocketmq.spout.RocketMqSpout) DefaultTopicSelector(org.apache.storm.rocketmq.common.selector.DefaultTopicSelector) Properties(java.util.Properties)

Example 2 with DefaultTopicSelector

use of org.apache.storm.rocketmq.common.selector.DefaultTopicSelector in project storm by apache.

the class WordCountTrident method buildTopology.

public static StormTopology buildTopology(String nameserverAddr, String topic) {
    Fields fields = new Fields("word", "count");
    FixedBatchSpout spout = new FixedBatchSpout(fields, 4, new Values("storm", 1), new Values("trident", 1), new Values("needs", 1), new Values("javadoc", 1));
    spout.setCycle(true);
    TupleToMessageMapper mapper = new FieldNameBasedTupleToMessageMapper("word", "count");
    TopicSelector selector = new DefaultTopicSelector(topic);
    Properties properties = new Properties();
    properties.setProperty(RocketMqConfig.NAME_SERVER_ADDR, nameserverAddr);
    RocketMqState.Options options = new RocketMqState.Options().withMapper(mapper).withSelector(selector).withProperties(properties);
    StateFactory factory = new RocketMqStateFactory(options);
    TridentTopology topology = new TridentTopology();
    Stream stream = topology.newStream("spout1", spout);
    stream.partitionPersist(factory, fields, new RocketMqStateUpdater(), new Fields());
    return topology.build();
}
Also used : FieldNameBasedTupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper) TopicSelector(org.apache.storm.rocketmq.common.selector.TopicSelector) DefaultTopicSelector(org.apache.storm.rocketmq.common.selector.DefaultTopicSelector) Values(org.apache.storm.tuple.Values) Properties(java.util.Properties) FixedBatchSpout(org.apache.storm.trident.testing.FixedBatchSpout) Fields(org.apache.storm.tuple.Fields) StateFactory(org.apache.storm.trident.state.StateFactory) RocketMqStateFactory(org.apache.storm.rocketmq.trident.state.RocketMqStateFactory) TupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.TupleToMessageMapper) FieldNameBasedTupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper) TridentTopology(org.apache.storm.trident.TridentTopology) DefaultTopicSelector(org.apache.storm.rocketmq.common.selector.DefaultTopicSelector) Stream(org.apache.storm.trident.Stream) RocketMqStateUpdater(org.apache.storm.rocketmq.trident.state.RocketMqStateUpdater) RocketMqState(org.apache.storm.rocketmq.trident.state.RocketMqState) RocketMqStateFactory(org.apache.storm.rocketmq.trident.state.RocketMqStateFactory)

Example 3 with DefaultTopicSelector

use of org.apache.storm.rocketmq.common.selector.DefaultTopicSelector in project storm by apache.

the class RocketMqBoltTest method setUp.

@Before
public void setUp() throws Exception {
    rocketMqBolt = new RocketMqBolt();
    rocketMqBolt.withSelector(new DefaultTopicSelector("tpc"));
    rocketMqBolt.withMapper(new FieldNameBasedTupleToMessageMapper("f1", "f2"));
    rocketMqBolt.withBatch(false);
    rocketMqBolt.withBatchSize(5);
    rocketMqBolt.withAsync(true);
    rocketMqBolt.withFlushIntervalSecs(5);
    producer = mock(DefaultMQProducer.class);
    TestUtils.setFieldValue(rocketMqBolt, "producer", producer);
}
Also used : FieldNameBasedTupleToMessageMapper(org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper) DefaultTopicSelector(org.apache.storm.rocketmq.common.selector.DefaultTopicSelector) DefaultMQProducer(org.apache.rocketmq.client.producer.DefaultMQProducer) Before(org.junit.Before)

Aggregations

FieldNameBasedTupleToMessageMapper (org.apache.storm.rocketmq.common.mapper.FieldNameBasedTupleToMessageMapper)3 DefaultTopicSelector (org.apache.storm.rocketmq.common.selector.DefaultTopicSelector)3 Properties (java.util.Properties)2 TupleToMessageMapper (org.apache.storm.rocketmq.common.mapper.TupleToMessageMapper)2 TopicSelector (org.apache.storm.rocketmq.common.selector.TopicSelector)2 Fields (org.apache.storm.tuple.Fields)2 DefaultMQProducer (org.apache.rocketmq.client.producer.DefaultMQProducer)1 RocketMqBolt (org.apache.storm.rocketmq.bolt.RocketMqBolt)1 RocketMqSpout (org.apache.storm.rocketmq.spout.RocketMqSpout)1 RocketMqState (org.apache.storm.rocketmq.trident.state.RocketMqState)1 RocketMqStateFactory (org.apache.storm.rocketmq.trident.state.RocketMqStateFactory)1 RocketMqStateUpdater (org.apache.storm.rocketmq.trident.state.RocketMqStateUpdater)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1 Stream (org.apache.storm.trident.Stream)1 TridentTopology (org.apache.storm.trident.TridentTopology)1 StateFactory (org.apache.storm.trident.state.StateFactory)1 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)1 Values (org.apache.storm.tuple.Values)1 Before (org.junit.Before)1