Search in sources :

Example 86 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class JdbcLookupBolt method process.

@Override
protected void process(Tuple tuple) {
    try {
        List<Column> columns = jdbcLookupMapper.getColumns(tuple);
        List<List<Column>> result = jdbcClient.select(this.selectQuery, columns);
        if (result != null && result.size() != 0) {
            for (List<Column> row : result) {
                List<Values> values = jdbcLookupMapper.toTuple(tuple, row);
                for (Values value : values) {
                    collector.emit(tuple, value);
                }
            }
        }
        this.collector.ack(tuple);
    } catch (Exception e) {
        this.collector.reportError(e);
        this.collector.fail(tuple);
    }
}
Also used : Column(org.apache.storm.jdbc.common.Column) Values(org.apache.storm.tuple.Values) List(java.util.List)

Example 87 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class SimpleJdbcLookupMapper method toTuple.

@Override
public List<Values> toTuple(ITuple input, List<Column> columns) {
    Values values = new Values();
    for (String field : outputFields) {
        if (input.contains(field)) {
            values.add(input.getValueByField(field));
        } else {
            for (Column column : columns) {
                if (column.getColumnName().equalsIgnoreCase(field)) {
                    values.add(column.getVal());
                }
            }
        }
    }
    List<Values> result = new ArrayList<Values>();
    result.add(values);
    return result;
}
Also used : Column(org.apache.storm.jdbc.common.Column) Values(org.apache.storm.tuple.Values) ArrayList(java.util.ArrayList)

Example 88 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class ByTopicRecordTranslatorTest method testTopicCollision.

@Test(expected = IllegalStateException.class)
public void testTopicCollision() {
    ByTopicRecordTranslator<String, String> trans = new ByTopicRecordTranslator<>((r) -> new Values(r.key()), new Fields("key"));
    trans.forTopic("foo", (r) -> new Values(r.value()), new Fields("value"), "foo1");
    trans.forTopic("foo", (r) -> new Values(r.key(), r.value()), new Fields("key", "value"), "foo2");
}
Also used : Fields(org.apache.storm.tuple.Fields) Values(org.apache.storm.tuple.Values) Test(org.junit.Test)

Example 89 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class ByTopicRecordTranslatorTest method testFieldCollision.

@Test(expected = IllegalArgumentException.class)
public void testFieldCollision() {
    ByTopicRecordTranslator<String, String> trans = new ByTopicRecordTranslator<>((r) -> new Values(r.key()), new Fields("key"));
    trans.forTopic("foo", (r) -> new Values(r.value()), new Fields("value"));
}
Also used : Fields(org.apache.storm.tuple.Fields) Values(org.apache.storm.tuple.Values) Test(org.junit.Test)

Example 90 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class SingleTopicKafkaSpoutTest method shouldEmitAllMessages.

@Test
public void shouldEmitAllMessages() throws Exception {
    try (SimulatedTime simulatedTime = new SimulatedTime()) {
        int messageCount = 10;
        initializeSpout(messageCount);
        //Emit all messages and check that they are emitted. Ack the messages too
        IntStream.range(0, messageCount).forEach(value -> {
            spout.nextTuple();
            ArgumentCaptor<Object> messageId = ArgumentCaptor.forClass(Object.class);
            verify(collector).emit(eq(SingleTopicKafkaSpoutConfiguration.STREAM), eq(new Values(SingleTopicKafkaSpoutConfiguration.TOPIC, Integer.toString(value), Integer.toString(value))), messageId.capture());
            spout.ack(messageId.getValue());
            reset(collector);
        });
        Time.advanceTime(commitOffsetPeriodMs + KafkaSpout.TIMER_DELAY_MS);
        //Commit offsets
        spout.nextTuple();
        verifyAllMessagesCommitted(messageCount);
    }
}
Also used : SimulatedTime(org.apache.storm.utils.Time.SimulatedTime) Values(org.apache.storm.tuple.Values) Matchers.anyObject(org.mockito.Matchers.anyObject) Test(org.junit.Test)

Aggregations

Values (org.apache.storm.tuple.Values)206 Test (org.junit.Test)89 ArrayList (java.util.ArrayList)42 Fields (org.apache.storm.tuple.Fields)40 HashMap (java.util.HashMap)39 ChannelHandler (org.apache.storm.sql.runtime.ChannelHandler)26 TridentTopology (org.apache.storm.trident.TridentTopology)21 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)14 Stream (org.apache.storm.trident.Stream)12 TupleImpl (org.apache.storm.tuple.TupleImpl)12 List (java.util.List)11 TestUtils (org.apache.storm.sql.TestUtils)11 TridentState (org.apache.storm.trident.TridentState)11 Tuple (org.apache.storm.tuple.Tuple)11 JSONObject (org.json.simple.JSONObject)10 Map (java.util.Map)9 Config (org.apache.storm.Config)9 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)9 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)8 TridentTuple (org.apache.storm.trident.tuple.TridentTuple)8