Search in sources :

Example 1 with ExponentialBackoffRetrier

use of org.apache.storm.kinesis.spout.ExponentialBackoffRetrier in project storm by apache.

the class KinesisSpoutTopology method main.

public static void main(String[] args) throws InvalidTopologyException, AuthorizationException, AlreadyAliveException {
    String topologyName = args[0];
    RecordToTupleMapper recordToTupleMapper = new TestRecordToTupleMapper();
    KinesisConnectionInfo kinesisConnectionInfo = new KinesisConnectionInfo(new CredentialsProviderChain(), new ClientConfiguration(), Regions.US_WEST_2, 1000);
    ZkInfo zkInfo = new ZkInfo("localhost:2181", "/kinesisOffsets", 20000, 15000, 10000L, 3, 2000);
    KinesisConfig kinesisConfig = new KinesisConfig(args[1], ShardIteratorType.TRIM_HORIZON, recordToTupleMapper, new Date(), new ExponentialBackoffRetrier(), zkInfo, kinesisConnectionInfo, 10000L);
    KinesisSpout kinesisSpout = new KinesisSpout(kinesisConfig);
    TopologyBuilder topologyBuilder = new TopologyBuilder();
    topologyBuilder.setSpout("spout", kinesisSpout, 3);
    topologyBuilder.setBolt("bolt", new KinesisBoltTest(), 1).shuffleGrouping("spout");
    Config topologyConfig = new Config();
    topologyConfig.setDebug(true);
    topologyConfig.setNumWorkers(3);
    StormSubmitter.submitTopology(topologyName, topologyConfig, topologyBuilder.createTopology());
}
Also used : RecordToTupleMapper(org.apache.storm.kinesis.spout.RecordToTupleMapper) KinesisConfig(org.apache.storm.kinesis.spout.KinesisConfig) ZkInfo(org.apache.storm.kinesis.spout.ZkInfo) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) KinesisConfig(org.apache.storm.kinesis.spout.KinesisConfig) Config(org.apache.storm.Config) KinesisSpout(org.apache.storm.kinesis.spout.KinesisSpout) Date(java.util.Date) CredentialsProviderChain(org.apache.storm.kinesis.spout.CredentialsProviderChain) KinesisConnectionInfo(org.apache.storm.kinesis.spout.KinesisConnectionInfo) ExponentialBackoffRetrier(org.apache.storm.kinesis.spout.ExponentialBackoffRetrier) ClientConfiguration(com.amazonaws.ClientConfiguration)

Aggregations

ClientConfiguration (com.amazonaws.ClientConfiguration)1 Date (java.util.Date)1 Config (org.apache.storm.Config)1 CredentialsProviderChain (org.apache.storm.kinesis.spout.CredentialsProviderChain)1 ExponentialBackoffRetrier (org.apache.storm.kinesis.spout.ExponentialBackoffRetrier)1 KinesisConfig (org.apache.storm.kinesis.spout.KinesisConfig)1 KinesisConnectionInfo (org.apache.storm.kinesis.spout.KinesisConnectionInfo)1 KinesisSpout (org.apache.storm.kinesis.spout.KinesisSpout)1 RecordToTupleMapper (org.apache.storm.kinesis.spout.RecordToTupleMapper)1 ZkInfo (org.apache.storm.kinesis.spout.ZkInfo)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1