Search in sources :

Example 11 with TopologyContext

use of backtype.storm.task.TopologyContext in project storm-elastic-search by hmsonline.

the class ElasticSearchTest method testBolt.

@Test
public void testBolt() {
    TopologyBuilder builder = new TopologyBuilder();
    ElasticSearchBolt bolt = new ElasticSearchBolt(new DefaultTupleMapper());
    builder.setBolt("TEST_BOLT", bolt);
    TopologyContext context = new MockTopologyContext(builder.createTopology());
    Config config = new Config();
    config.put(ElasticSearchBolt.ELASTIC_LOCAL_MODE, true);
    bolt.prepare(config, context, null);
    List<Object> values = new ArrayList<Object>();
    values.add("testIndex");
    values.add("entity");
    values.add("testId");
    values.add("{\"bolt\":\"lightning\"}");
    Tuple tuple = new TupleImpl(context, values, 5, "test");
    bolt.execute(tuple);
}
Also used : DefaultTupleMapper(com.hmsonline.storm.contrib.bolt.elasticsearch.mapper.DefaultTupleMapper) TopologyBuilder(backtype.storm.topology.TopologyBuilder) Config(backtype.storm.Config) ArrayList(java.util.ArrayList) TopologyContext(backtype.storm.task.TopologyContext) TupleImpl(backtype.storm.tuple.TupleImpl) Tuple(backtype.storm.tuple.Tuple) Test(org.junit.Test)

Example 12 with TopologyContext

use of backtype.storm.task.TopologyContext in project jstorm by alibaba.

the class TaskShutdownDameon method shutdown.

@Override
public void shutdown() {
    if (isClosing.compareAndSet(false, true)) {
        LOG.info("Begin to shut down task " + topology_id + ":" + task_id);
        TopologyContext userContext = task.getUserContext();
        for (ITaskHook iTaskHook : userContext.getHooks()) iTaskHook.cleanup();
        closeComponent(task_obj);
        taskHeartbeatTrigger.updateExecutorStatus(TaskStatus.SHUTDOWN);
        // To be assure send  the shutdown information TM
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
        }
        // all thread will check the taskStatus
        // once it has been set SHUTDOWN, it will quit
        taskStatus.setStatus(TaskStatus.SHUTDOWN);
        for (AsyncLoopThread thr : all_threads) {
            LOG.info("Begin to shutdown " + thr.getThread().getName());
            thr.cleanup();
            JStormUtils.sleepMs(10);
            thr.interrupt();
            // try {
            // //thr.join();
            // thr.getThread().stop(new RuntimeException());
            // } catch (Throwable e) {
            // }
            LOG.info("Successfully shutdown " + thr.getThread().getName());
        }
        try {
            zkCluster.disconnect();
        } catch (Exception e) {
            LOG.error("Failed to disconnect zk for task-" + task_id);
        }
        LOG.info("Successfully shutdown task " + topology_id + ":" + task_id);
    }
}
Also used : ITaskHook(backtype.storm.hooks.ITaskHook) TopologyContext(backtype.storm.task.TopologyContext) AsyncLoopThread(com.alibaba.jstorm.callback.AsyncLoopThread)

Example 13 with TopologyContext

use of backtype.storm.task.TopologyContext in project kafka-spout by HolmesNL.

the class KafkaSpoutConstructorTest method testOpenWithDefaultTopicName.

/**
     * Using the default constructor, a topic name must be in the storm config.
     */
@Test
public void testOpenWithDefaultTopicName() {
    KafkaSpout spout = spy(new KafkaSpout());
    TopologyContext topology = mock(TopologyContext.class);
    SpoutOutputCollector collector = mock(SpoutOutputCollector.class);
    Map<String, Object> config = new HashMap<String, Object>();
    doNothing().when(spout).createConsumer(config);
    spout.open(config, topology, collector);
    assertEquals("Wrong Topic Name", spout._topic, ConfigUtils.DEFAULT_TOPIC);
}
Also used : HashMap(java.util.HashMap) SpoutOutputCollector(backtype.storm.spout.SpoutOutputCollector) TopologyContext(backtype.storm.task.TopologyContext) Test(org.junit.Test)

Example 14 with TopologyContext

use of backtype.storm.task.TopologyContext in project kafka-spout by HolmesNL.

the class KafkaSpoutConstructorTest method testOpenWithOverloadedConstructor.

/**
     * If we use the overloaded constructor, do not even look at the storm config for the topic name.
     */
@Test
public void testOpenWithOverloadedConstructor() {
    KafkaSpout spout = spy(new KafkaSpout("OVERLOAD"));
    TopologyContext topology = mock(TopologyContext.class);
    SpoutOutputCollector collector = mock(SpoutOutputCollector.class);
    Map<String, Object> config = new HashMap<String, Object>();
    doNothing().when(spout).createConsumer(config);
    spout.open(config, topology, collector);
    assertEquals("Wrong Topic Name", spout._topic, "OVERLOAD");
}
Also used : HashMap(java.util.HashMap) SpoutOutputCollector(backtype.storm.spout.SpoutOutputCollector) TopologyContext(backtype.storm.task.TopologyContext) Test(org.junit.Test)

Example 15 with TopologyContext

use of backtype.storm.task.TopologyContext in project kafka-spout by HolmesNL.

the class KafkaSpoutConstructorTest method testOpenWithDefaultConstructor.

/**
     * Using the default constructor, a topic name must be in the storm config
     */
@Test
public void testOpenWithDefaultConstructor() {
    KafkaSpout spout = spy(new KafkaSpout());
    TopologyContext topology = mock(TopologyContext.class);
    SpoutOutputCollector collector = mock(SpoutOutputCollector.class);
    Map<String, Object> config = new HashMap<String, Object>();
    config.put(ConfigUtils.CONFIG_TOPIC, "topic");
    doNothing().when(spout).createConsumer(config);
    spout.open(config, topology, collector);
    assertEquals("Wrong Topic Name", spout._topic, "topic");
}
Also used : HashMap(java.util.HashMap) SpoutOutputCollector(backtype.storm.spout.SpoutOutputCollector) TopologyContext(backtype.storm.task.TopologyContext) Test(org.junit.Test)

Aggregations

TopologyContext (backtype.storm.task.TopologyContext)17 SpoutOutputCollector (backtype.storm.spout.SpoutOutputCollector)7 ClientConfiguration (com.yahoo.pulsar.client.api.ClientConfiguration)5 HashMap (java.util.HashMap)5 Test (org.junit.Test)5 PersistentTopicStats (com.yahoo.pulsar.common.policies.data.PersistentTopicStats)3 Test (org.testng.annotations.Test)3 OutputCollector (backtype.storm.task.OutputCollector)2 Config (backtype.storm.Config)1 GlobalStreamId (backtype.storm.generated.GlobalStreamId)1 Grouping (backtype.storm.generated.Grouping)1 StormTopology (backtype.storm.generated.StormTopology)1 ITaskHook (backtype.storm.hooks.ITaskHook)1 SpoutOutputCollectorImpl (backtype.storm.spout.SpoutOutputCollectorImpl)1 OutputCollectorImpl (backtype.storm.task.OutputCollectorImpl)1 WorkerTopologyContext (backtype.storm.task.WorkerTopologyContext)1 TopologyBuilder (backtype.storm.topology.TopologyBuilder)1 Tuple (backtype.storm.tuple.Tuple)1 TupleImpl (backtype.storm.tuple.TupleImpl)1 AsyncLoopThread (com.alibaba.jstorm.callback.AsyncLoopThread)1