Search in sources :

Example 1 with TopologyContext

use of org.apache.storm.task.TopologyContext in project storm by apache.

the class WindowedBoltExecutorTest method testPrepareLateTUpleStreamWithoutBuilder.

@Test
public void testPrepareLateTUpleStreamWithoutBuilder() throws Exception {
    Map<String, Object> conf = new HashMap<>();
    conf.put(Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS, 100000);
    conf.put(Config.TOPOLOGY_BOLTS_WINDOW_LENGTH_DURATION_MS, 20);
    conf.put(Config.TOPOLOGY_BOLTS_SLIDING_INTERVAL_DURATION_MS, 10);
    conf.put(Config.TOPOLOGY_BOLTS_LATE_TUPLE_STREAM, "$late");
    conf.put(Config.TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_MAX_LAG_MS, 5);
    conf.put(Config.TOPOLOGY_BOLTS_WATERMARK_EVENT_INTERVAL_MS, 10);
    testWindowedBolt = new TestWindowedBolt();
    testWindowedBolt.withTimestampField("ts");
    executor = new WindowedBoltExecutor(testWindowedBolt);
    TopologyContext context = getTopologyContext();
    try {
        executor.prepare(conf, context, getOutputCollector());
        fail();
    } catch (IllegalArgumentException e) {
        assertThat(e.getMessage(), is("Stream for late tuples must be defined with the builder method withLateTupleStream"));
    }
}
Also used : HashMap(java.util.HashMap) TopologyContext(org.apache.storm.task.TopologyContext) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) Test(org.junit.Test)

Example 2 with TopologyContext

use of org.apache.storm.task.TopologyContext in project heron by twitter.

the class IRichSpoutDelegate method open.

@Override
@SuppressWarnings("rawtypes")
public void open(Map conf, com.twitter.heron.api.topology.TopologyContext context, SpoutOutputCollector collector) {
    topologyContextImpl = new TopologyContext(context);
    spoutOutputCollectorImpl = new SpoutOutputCollectorImpl(collector);
    delegate.open(conf, topologyContextImpl, spoutOutputCollectorImpl);
}
Also used : SpoutOutputCollectorImpl(org.apache.storm.spout.SpoutOutputCollectorImpl) TopologyContext(org.apache.storm.task.TopologyContext)

Example 3 with TopologyContext

use of org.apache.storm.task.TopologyContext in project flink by apache.

the class SpoutWrapper method run.

@Override
public final void run(final SourceContext<OUT> ctx) throws Exception {
    final GlobalJobParameters config = super.getRuntimeContext().getExecutionConfig().getGlobalJobParameters();
    StormConfig stormConfig = new StormConfig();
    if (config != null) {
        if (config instanceof StormConfig) {
            stormConfig = (StormConfig) config;
        } else {
            stormConfig.putAll(config.toMap());
        }
    }
    final TopologyContext stormTopologyContext = WrapperSetupHelper.createTopologyContext((StreamingRuntimeContext) super.getRuntimeContext(), this.spout, this.name, this.stormTopology, stormConfig);
    SpoutCollector<OUT> collector = new SpoutCollector<OUT>(this.numberOfAttributes, stormTopologyContext.getThisTaskId(), ctx);
    this.spout.open(stormConfig, stormTopologyContext, new SpoutOutputCollector(collector));
    this.spout.activate();
    if (numberOfInvocations == null) {
        if (this.spout instanceof FiniteSpout) {
            final FiniteSpout finiteSpout = (FiniteSpout) this.spout;
            while (this.isRunning && !finiteSpout.reachedEnd()) {
                finiteSpout.nextTuple();
            }
        } else {
            while (this.isRunning) {
                this.spout.nextTuple();
            }
        }
    } else {
        int counter = this.numberOfInvocations;
        if (counter >= 0) {
            while ((--counter >= 0) && this.isRunning) {
                this.spout.nextTuple();
            }
        } else {
            do {
                collector.tupleEmitted = false;
                this.spout.nextTuple();
            } while (collector.tupleEmitted && this.isRunning);
        }
    }
}
Also used : StormConfig(org.apache.flink.storm.util.StormConfig) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) TopologyContext(org.apache.storm.task.TopologyContext) GlobalJobParameters(org.apache.flink.api.common.ExecutionConfig.GlobalJobParameters) FiniteSpout(org.apache.flink.storm.util.FiniteSpout)

Example 4 with TopologyContext

use of org.apache.storm.task.TopologyContext in project open-kilda by telstra.

the class DumpStateAction method handle.

@Override
protected void handle() throws MessageFormatException, UnsupportedActionException, JsonProcessingException {
    AbstractDumpState state = getMaster().getBolt().dumpState();
    TopologyContext context = getBolt().getContext();
    emitResponse(new DumpStateResponseData(context.getThisComponentId(), context.getThisTaskId(), getMessage().getTopology(), state));
}
Also used : AbstractDumpState(org.openkilda.messaging.ctrl.AbstractDumpState) DumpStateResponseData(org.openkilda.messaging.ctrl.DumpStateResponseData) TopologyContext(org.apache.storm.task.TopologyContext)

Example 5 with TopologyContext

use of org.apache.storm.task.TopologyContext in project open-kilda by telstra.

the class ListAction method handle.

@Override
protected void handle() throws MessageFormatException, UnsupportedActionException, JsonProcessingException {
    TopologyContext context = getBolt().getContext();
    emitResponse(new ResponseData(context.getThisComponentId(), context.getThisTaskId(), getMessage().getTopology()));
}
Also used : ResponseData(org.openkilda.messaging.ctrl.ResponseData) TopologyContext(org.apache.storm.task.TopologyContext)

Aggregations

TopologyContext (org.apache.storm.task.TopologyContext)62 Test (org.junit.Test)29 HashMap (java.util.HashMap)25 OutputCollector (org.apache.storm.task.OutputCollector)19 Tuple (org.apache.storm.tuple.Tuple)16 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)15 Map (java.util.Map)14 GlobalStreamId (org.apache.storm.generated.GlobalStreamId)8 ClientConfiguration (org.apache.pulsar.client.api.ClientConfiguration)7 Test (org.testng.annotations.Test)7 WriterConfiguration (org.apache.metron.common.configuration.writer.WriterConfiguration)6 BulkWriterResponse (org.apache.metron.common.writer.BulkWriterResponse)6 Collections (java.util.Collections)5 Grouping (org.apache.storm.generated.Grouping)5 StormTopology (org.apache.storm.generated.StormTopology)5 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)5 OutputCollectorImpl (org.apache.storm.task.OutputCollectorImpl)5 IRichBolt (org.apache.storm.topology.IRichBolt)5 IRichSpout (org.apache.storm.topology.IRichSpout)5 ParserConfigurations (org.apache.metron.common.configuration.ParserConfigurations)4