Search in sources :

Example 1 with Twister2Bolt

use of org.apache.storm.topology.twister2.Twister2Bolt in project twister2 by DSC-SPIDAL.

the class TopologyBuilder method createT2Bolt.

private BoltDeclarer createT2Bolt(String id, Object bolt, Number parallelismHint) {
    Twister2Bolt twister2Bolt = new Twister2Bolt(id, bolt, source -> {
        // the source is not a sink anymore,
        // it is a source for another node(compute node)
        boolean sourceWasInSinkNodes = this.sinkNodes.remove(source);
        if (sourceWasInSinkNodes) {
            this.computeNodes.add(source);
        }
    });
    twister2Bolt.setParallelism(parallelismHint.intValue());
    this.nodes.put(id, twister2Bolt);
    // add all to the sink nodes initially
    this.sinkNodes.add(id);
    return twister2Bolt.getBoltDeclarer();
}
Also used : Twister2Bolt(org.apache.storm.topology.twister2.Twister2Bolt)

Example 2 with Twister2Bolt

use of org.apache.storm.topology.twister2.Twister2Bolt in project twister2 by DSC-SPIDAL.

the class TopologyBuilder method createTopology.

public StormTopology createTopology() {
    this.sourceNodes.forEach(source -> {
        Twister2Spout twister2Spout = (Twister2Spout) nodes.get(source);
        LOG.info("Adding source : " + source);
        this.computeGraphBuilder.addSource(source, twister2Spout, twister2Spout.getParallelism());
    });
    this.computeNodes.forEach(compute -> {
        Twister2Bolt twister2Bolt = (Twister2Bolt) nodes.get(compute);
        ComputeConnection computeConnection = this.computeGraphBuilder.addCompute(compute, twister2Bolt, twister2Bolt.getParallelism());
        this.defineGrouping(twister2Bolt, computeConnection);
    });
    this.sinkNodes.forEach(sink -> {
        Twister2Bolt twister2Bolt = (Twister2Bolt) nodes.get(sink);
        ComputeConnection computeConnection = this.computeGraphBuilder.addCompute(sink, twister2Bolt, twister2Bolt.getParallelism());
        this.defineGrouping(twister2Bolt, computeConnection);
    });
    this.computeGraphBuilder.setMode(OperationMode.STREAMING);
    return new StormTopology(this.computeGraphBuilder.build());
}
Also used : Twister2Spout(org.apache.storm.topology.twister2.Twister2Spout) StormTopology(org.apache.storm.generated.StormTopology) Twister2Bolt(org.apache.storm.topology.twister2.Twister2Bolt) ComputeConnection(edu.iu.dsc.tws.task.impl.ComputeConnection)

Aggregations

Twister2Bolt (org.apache.storm.topology.twister2.Twister2Bolt)2 ComputeConnection (edu.iu.dsc.tws.task.impl.ComputeConnection)1 StormTopology (org.apache.storm.generated.StormTopology)1 Twister2Spout (org.apache.storm.topology.twister2.Twister2Spout)1