Search in sources :

Example 1 with OpenTsdbConfig

use of org.openkilda.wfm.topology.opentsdb.OpenTsdbTopologyConfig.OpenTsdbConfig in project open-kilda by telstra.

the class OpenTsdbTopology method createTopology.

@Override
public StormTopology createTopology() {
    logger.info("Creating OpenTsdbTopology - {}", topologyName);
    TopologyBuilder tb = new TopologyBuilder();
    ZooKeeperSpout zooKeeperSpout = new ZooKeeperSpout(getConfig().getBlueGreenMode(), getZkTopoName(), getZookeeperConfig());
    declareSpout(tb, zooKeeperSpout, ZooKeeperSpout.SPOUT_ID);
    ZooKeeperBolt zooKeeperBolt = new ZooKeeperBolt(getConfig().getBlueGreenMode(), getZkTopoName(), getZookeeperConfig(), getBoltInstancesCount(OTSDB_PARSE_BOLT_ID));
    declareBolt(tb, zooKeeperBolt, ZooKeeperBolt.BOLT_ID).allGrouping(OTSDB_PARSE_BOLT_ID, ZkStreams.ZK.toString());
    attachInput(tb);
    OpenTsdbConfig openTsdbConfig = topologyConfig.getOpenTsdbConfig();
    declareBolt(tb, new DatapointParseBolt(), OTSDB_PARSE_BOLT_ID).shuffleGrouping(OTSDB_SPOUT_ID).allGrouping(ZooKeeperSpout.SPOUT_ID);
    declareBolt(tb, new OpenTSDBFilterBolt(), OTSDB_FILTER_BOLT_ID).fieldsGrouping(OTSDB_PARSE_BOLT_ID, new Fields("hash"));
    OpenTsdbClient.Builder tsdbBuilder = OpenTsdbClient.newBuilder(openTsdbConfig.getHosts()).returnDetails();
    if (openTsdbConfig.getClientChunkedRequestsEnabled()) {
        tsdbBuilder.enableChunkedEncoding();
    }
    OpenTsdbBolt openTsdbBolt = new OpenTsdbBolt(tsdbBuilder, Collections.singletonList(TupleOpenTsdbDatapointMapper.DEFAULT_MAPPER));
    openTsdbBolt.withBatchSize(openTsdbConfig.getBatchSize()).withFlushInterval(openTsdbConfig.getFlushInterval());
    declareBolt(tb, openTsdbBolt, OTSDB_BOLT_ID).shuffleGrouping(OTSDB_FILTER_BOLT_ID);
    return tb.createTopology();
}
Also used : DatapointParseBolt(org.openkilda.wfm.topology.opentsdb.bolts.DatapointParseBolt) Fields(org.apache.storm.tuple.Fields) OpenTsdbClient(org.apache.storm.opentsdb.client.OpenTsdbClient) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) ZooKeeperSpout(org.openkilda.wfm.share.zk.ZooKeeperSpout) ZooKeeperBolt(org.openkilda.wfm.share.zk.ZooKeeperBolt) OpenTsdbBolt(org.apache.storm.opentsdb.bolt.OpenTsdbBolt) OpenTsdbConfig(org.openkilda.wfm.topology.opentsdb.OpenTsdbTopologyConfig.OpenTsdbConfig) OpenTSDBFilterBolt(org.openkilda.wfm.topology.opentsdb.bolts.OpenTSDBFilterBolt)

Aggregations

OpenTsdbBolt (org.apache.storm.opentsdb.bolt.OpenTsdbBolt)1 OpenTsdbClient (org.apache.storm.opentsdb.client.OpenTsdbClient)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1 Fields (org.apache.storm.tuple.Fields)1 ZooKeeperBolt (org.openkilda.wfm.share.zk.ZooKeeperBolt)1 ZooKeeperSpout (org.openkilda.wfm.share.zk.ZooKeeperSpout)1 OpenTsdbConfig (org.openkilda.wfm.topology.opentsdb.OpenTsdbTopologyConfig.OpenTsdbConfig)1 DatapointParseBolt (org.openkilda.wfm.topology.opentsdb.bolts.DatapointParseBolt)1 OpenTSDBFilterBolt (org.openkilda.wfm.topology.opentsdb.bolts.OpenTSDBFilterBolt)1