Search in sources :

Example 1 with DefaultTopicSelector

use of org.apache.storm.kafka.trident.selector.DefaultTopicSelector in project storm by apache.

the class TridentKafkaTest method setup.

@Before
public void setup() {
    broker = new KafkaTestBroker();
    simpleConsumer = TestUtils.getKafkaConsumer(broker);
    TridentTupleToKafkaMapper<Object, Object> mapper = new FieldNameBasedTupleToKafkaMapper<Object, Object>("key", "message");
    KafkaTopicSelector topicSelector = new DefaultTopicSelector(TestUtils.TOPIC);
    state = new TridentKafkaState().withKafkaTopicSelector(topicSelector).withTridentTupleToKafkaMapper(mapper);
    state.prepare(TestUtils.getProducerProperties(broker.getBrokerConnectionString()));
}
Also used : TridentKafkaState(org.apache.storm.kafka.trident.TridentKafkaState) KafkaTopicSelector(org.apache.storm.kafka.trident.selector.KafkaTopicSelector) DefaultTopicSelector(org.apache.storm.kafka.trident.selector.DefaultTopicSelector) FieldNameBasedTupleToKafkaMapper(org.apache.storm.kafka.trident.mapper.FieldNameBasedTupleToKafkaMapper) Before(org.junit.Before)

Example 2 with DefaultTopicSelector

use of org.apache.storm.kafka.trident.selector.DefaultTopicSelector in project storm by apache.

the class TridentKafkaTopology method buildTopology.

private static StormTopology buildTopology(String brokerConnectionString) {
    Fields fields = new Fields("word", "count");
    FixedBatchSpout spout = new FixedBatchSpout(fields, 4, new Values("storm", "1"), new Values("trident", "1"), new Values("needs", "1"), new Values("javadoc", "1"));
    spout.setCycle(true);
    TridentTopology topology = new TridentTopology();
    Stream stream = topology.newStream("spout1", spout);
    Properties props = new Properties();
    props.put("bootstrap.servers", brokerConnectionString);
    props.put("acks", "1");
    props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
    props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
    TridentKafkaStateFactory stateFactory = new TridentKafkaStateFactory().withProducerProperties(props).withKafkaTopicSelector(new DefaultTopicSelector("test")).withTridentTupleToKafkaMapper(new FieldNameBasedTupleToKafkaMapper("word", "count"));
    stream.partitionPersist(stateFactory, fields, new TridentKafkaUpdater(), new Fields());
    return topology.build();
}
Also used : FixedBatchSpout(org.apache.storm.trident.testing.FixedBatchSpout) Fields(org.apache.storm.tuple.Fields) TridentTopology(org.apache.storm.trident.TridentTopology) Values(org.apache.storm.tuple.Values) Stream(org.apache.storm.trident.Stream) DefaultTopicSelector(org.apache.storm.kafka.trident.selector.DefaultTopicSelector) Properties(java.util.Properties) FieldNameBasedTupleToKafkaMapper(org.apache.storm.kafka.trident.mapper.FieldNameBasedTupleToKafkaMapper)

Aggregations

FieldNameBasedTupleToKafkaMapper (org.apache.storm.kafka.trident.mapper.FieldNameBasedTupleToKafkaMapper)2 DefaultTopicSelector (org.apache.storm.kafka.trident.selector.DefaultTopicSelector)2 Properties (java.util.Properties)1 TridentKafkaState (org.apache.storm.kafka.trident.TridentKafkaState)1 KafkaTopicSelector (org.apache.storm.kafka.trident.selector.KafkaTopicSelector)1 Stream (org.apache.storm.trident.Stream)1 TridentTopology (org.apache.storm.trident.TridentTopology)1 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)1 Fields (org.apache.storm.tuple.Fields)1 Values (org.apache.storm.tuple.Values)1 Before (org.junit.Before)1