Search in sources :

Example 56 with TopologyContext

use of org.apache.storm.task.TopologyContext in project storm by apache.

the class Testing method testTuple.

/**
 * Create a {@link org.apache.storm.tuple.Tuple} for use with testing.
 * @param values the values to appear in the tuple
 * @param param parametrs describing more details about the tuple
 */
public static Tuple testTuple(List<Object> values, MkTupleParam param) {
    String stream = param.getStream();
    if (stream == null) {
        stream = Utils.DEFAULT_STREAM_ID;
    }
    String component = param.getComponent();
    if (component == null) {
        component = "component";
    }
    int task = 1;
    List<String> fields = param.getFields();
    if (fields == null) {
        fields = new ArrayList<>(values.size());
        for (int i = 1; i <= values.size(); i++) {
            fields.add("field" + i);
        }
    }
    Map<Integer, String> taskToComp = new HashMap<>();
    taskToComp.put(task, component);
    Map<String, Map<String, Fields>> compToStreamToFields = new HashMap<>();
    Map<String, Fields> streamToFields = new HashMap<>();
    streamToFields.put(stream, new Fields(fields));
    compToStreamToFields.put(component, streamToFields);
    TopologyContext context = new TopologyContext(null, ConfigUtils.readStormConfig(), taskToComp, null, compToStreamToFields, null, "test-storm-id", null, null, 1, null, null, new HashMap<>(), new HashMap<>(), new HashMap<>(), new HashMap<>(), new AtomicBoolean(false), null);
    return new TupleImpl(context, values, component, 1, stream);
}
Also used : HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Fields(org.apache.storm.tuple.Fields) TopologyContext(org.apache.storm.task.TopologyContext) HashMap(java.util.HashMap) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 57 with TopologyContext

use of org.apache.storm.task.TopologyContext in project storm by apache.

the class KafkaBoltTest method testSimple.

@Test
public void testSimple() {
    MockProducer<String, String> producer = new MockProducer<>(Cluster.empty(), false, null, null, null);
    KafkaBolt<String, String> bolt = makeBolt(producer);
    OutputCollector collector = mock(OutputCollector.class);
    TopologyContext context = mock(TopologyContext.class);
    Map<String, Object> conf = new HashMap<>();
    bolt.prepare(conf, context, collector);
    String key = "KEY";
    String value = "VALUE";
    Tuple testTuple = createTestTuple(key, value);
    bolt.execute(testTuple);
    assertThat(producer.history().size(), is(1));
    ProducerRecord<String, String> arg = producer.history().get(0);
    LOG.info("GOT {} ->", arg);
    LOG.info("{}, {}, {}", arg.topic(), arg.key(), arg.value());
    assertThat(arg.topic(), is("MY_TOPIC"));
    assertThat(arg.key(), is(key));
    assertThat(arg.value(), is(value));
    // Complete the send
    producer.completeNext();
    verify(collector).ack(testTuple);
}
Also used : OutputCollector(org.apache.storm.task.OutputCollector) MockProducer(org.apache.kafka.clients.producer.MockProducer) HashMap(java.util.HashMap) TopologyContext(org.apache.storm.task.TopologyContext) Tuple(org.apache.storm.tuple.Tuple) Test(org.junit.Test)

Example 58 with TopologyContext

use of org.apache.storm.task.TopologyContext in project storm by apache.

the class RollingCountBoltTest method shouldEmitSomethingIfAtLeastOneObjectWasCountedAndTickTupleIsReceived.

@SuppressWarnings("rawtypes")
@Test
public void shouldEmitSomethingIfAtLeastOneObjectWasCountedAndTickTupleIsReceived() {
    // given
    Tuple normalTuple = mockNormalTuple(new Object());
    Tuple tickTuple = MockTupleHelpers.mockTickTuple();
    RollingCountBolt bolt = new RollingCountBolt();
    Map<String, Object> conf = mock(Map.class);
    TopologyContext context = mock(TopologyContext.class);
    OutputCollector collector = mock(OutputCollector.class);
    bolt.prepare(conf, context, collector);
    // when
    bolt.execute(normalTuple);
    bolt.execute(tickTuple);
    // then
    verify(collector).emit(any(Values.class));
}
Also used : OutputCollector(org.apache.storm.task.OutputCollector) Values(org.apache.storm.tuple.Values) TopologyContext(org.apache.storm.task.TopologyContext) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test)

Example 59 with TopologyContext

use of org.apache.storm.task.TopologyContext in project kafka-spout by HolmesNL.

the class KafkaSpoutConstructorTest method testOpenWithDefaultTopicName.

/**
 * Using the default constructor, a topic name must be in the storm config.
 */
@Test
public void testOpenWithDefaultTopicName() {
    KafkaSpout spout = spy(new KafkaSpout());
    TopologyContext topology = mock(TopologyContext.class);
    SpoutOutputCollector collector = mock(SpoutOutputCollector.class);
    Map<String, Object> config = new HashMap<String, Object>();
    doNothing().when(spout).createConsumer(config);
    spout.open(config, topology, collector);
    assertEquals("Wrong Topic Name", spout._topic, ConfigUtils.DEFAULT_TOPIC);
}
Also used : HashMap(java.util.HashMap) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) TopologyContext(org.apache.storm.task.TopologyContext) Test(org.junit.Test)

Example 60 with TopologyContext

use of org.apache.storm.task.TopologyContext in project kafka-spout by HolmesNL.

the class KafkaSpoutConstructorTest method testOpenWithOverloadedConstructor.

/**
 * If we use the overloaded constructor, do not even look at the storm config for the topic name.
 */
@Test
public void testOpenWithOverloadedConstructor() {
    KafkaSpout spout = spy(new KafkaSpout("OVERLOAD"));
    TopologyContext topology = mock(TopologyContext.class);
    SpoutOutputCollector collector = mock(SpoutOutputCollector.class);
    Map<String, Object> config = new HashMap<String, Object>();
    doNothing().when(spout).createConsumer(config);
    spout.open(config, topology, collector);
    assertEquals("Wrong Topic Name", spout._topic, "OVERLOAD");
}
Also used : HashMap(java.util.HashMap) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) TopologyContext(org.apache.storm.task.TopologyContext) Test(org.junit.Test)

Aggregations

TopologyContext (org.apache.storm.task.TopologyContext)62 Test (org.junit.Test)29 HashMap (java.util.HashMap)25 OutputCollector (org.apache.storm.task.OutputCollector)19 Tuple (org.apache.storm.tuple.Tuple)16 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)15 Map (java.util.Map)14 GlobalStreamId (org.apache.storm.generated.GlobalStreamId)8 ClientConfiguration (org.apache.pulsar.client.api.ClientConfiguration)7 Test (org.testng.annotations.Test)7 WriterConfiguration (org.apache.metron.common.configuration.writer.WriterConfiguration)6 BulkWriterResponse (org.apache.metron.common.writer.BulkWriterResponse)6 Collections (java.util.Collections)5 Grouping (org.apache.storm.generated.Grouping)5 StormTopology (org.apache.storm.generated.StormTopology)5 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)5 OutputCollectorImpl (org.apache.storm.task.OutputCollectorImpl)5 IRichBolt (org.apache.storm.topology.IRichBolt)5 IRichSpout (org.apache.storm.topology.IRichSpout)5 ParserConfigurations (org.apache.metron.common.configuration.ParserConfigurations)4