Search in sources :

Example 76 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class TestStormSql method testGroupbyBuiltin.

@Test
public void testGroupbyBuiltin() throws Exception {
    List<String> stmt = new ArrayList<>();
    stmt.add("CREATE EXTERNAL TABLE FOO (ID INT PRIMARY KEY, SALARY INT, PCT DOUBLE, NAME VARCHAR) LOCATION 'mockgroup:///foo'");
    stmt.add("SELECT STREAM ID, COUNT(*), SUM(SALARY), AVG(SALARY) FROM FOO GROUP BY (ID)");
    StormSql sql = StormSql.construct();
    List<Values> values = new ArrayList<>();
    ChannelHandler h = new TestUtils.CollectDataChannelHandler(values);
    sql.execute(stmt, h);
    Assert.assertEquals(4, values.size());
    Assert.assertEquals(3, values.get(0).get(2));
    Assert.assertEquals(12, values.get(1).get(2));
    Assert.assertEquals(21, values.get(2).get(2));
    Assert.assertEquals(9, values.get(3).get(2));
}
Also used : ArrayList(java.util.ArrayList) Values(org.apache.storm.tuple.Values) ChannelHandler(org.apache.storm.sql.runtime.ChannelHandler) Test(org.junit.Test)

Example 77 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class EvaluationFunction method execute.

@Override
public Values execute(TridentTuple input) {
    Context calciteContext = new StormContext(dataContext);
    calciteContext.values = input.getValues().toArray();
    projectionInstance.execute(calciteContext, outputValues);
    return new Values(outputValues);
}
Also used : Context(org.apache.calcite.interpreter.Context) DataContext(org.apache.calcite.DataContext) TridentOperationContext(org.apache.storm.trident.operation.TridentOperationContext) StormContext(org.apache.calcite.interpreter.StormContext) Values(org.apache.storm.tuple.Values) StormContext(org.apache.calcite.interpreter.StormContext)

Example 78 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class DruidBeamBolt method process.

@Override
protected void process(final Tuple tuple) {
    Future future = tranquilizer.send((druidEventMapper.getEvent(tuple)));
    LOG.debug("Sent tuple : [{}]", tuple);
    future.addEventListener(new FutureEventListener() {

        @Override
        public void onFailure(Throwable cause) {
            if (cause instanceof MessageDroppedException) {
                collector.ack(tuple);
                LOG.debug("Tuple Dropped due to MessageDroppedException : [{}]", tuple);
                if (druidConfig.getDiscardStreamId() != null)
                    collector.emit(druidConfig.getDiscardStreamId(), new Values(tuple, System.currentTimeMillis()));
            } else {
                collector.fail(tuple);
                LOG.debug("Tuple Processing Failed : [{}]", tuple);
            }
        }

        @Override
        public void onSuccess(Object value) {
            collector.ack(tuple);
            LOG.debug("Tuple Processing Success : [{}]", tuple);
        }
    });
}
Also used : MessageDroppedException(com.metamx.tranquility.tranquilizer.MessageDroppedException) Values(org.apache.storm.tuple.Values) Future(com.twitter.util.Future) FutureEventListener(com.twitter.util.FutureEventListener)

Example 79 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class EsPercolateBoltTest method testEsPercolateBolt.

@Test
public void testEsPercolateBolt() throws Exception {
    String source = "{\"user\":\"user1\"}";
    String index = "index1";
    String type = ".percolator";
    node.client().prepareIndex("index1", ".percolator").setId("1").setSource("{\"query\":{\"match\":{\"user\":\"user1\"}}}").execute().actionGet();
    Tuple tuple = EsTestUtil.generateTestTuple(source, index, type, null);
    bolt.execute(tuple);
    verify(outputCollector).ack(tuple);
    verify(outputCollector).emit(new Values(source, any(PercolateResponse.Match.class)));
}
Also used : Values(org.apache.storm.tuple.Values) Tuple(org.apache.storm.tuple.Tuple) PercolateResponse(org.elasticsearch.action.percolate.PercolateResponse) IntegrationTest(org.apache.storm.testing.IntegrationTest) Test(org.junit.Test)

Example 80 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class HBaseLookupBolt method execute.

@Override
public void execute(Tuple tuple) {
    if (TupleUtils.isTick(tuple)) {
        collector.ack(tuple);
        return;
    }
    byte[] rowKey = this.mapper.rowKey(tuple);
    Result result = null;
    try {
        if (cacheEnabled) {
            result = cache.get(rowKey);
        } else {
            Get get = hBaseClient.constructGetRequests(rowKey, projectionCriteria);
            result = hBaseClient.batchGet(Lists.newArrayList(get))[0];
        }
        for (Values values : rowToTupleMapper.toValues(tuple, result)) {
            this.collector.emit(tuple, values);
        }
        this.collector.ack(tuple);
    } catch (Exception e) {
        this.collector.reportError(e);
        this.collector.fail(tuple);
    }
}
Also used : Get(org.apache.hadoop.hbase.client.Get) Values(org.apache.storm.tuple.Values) Result(org.apache.hadoop.hbase.client.Result)

Aggregations

Values (org.apache.storm.tuple.Values)206 Test (org.junit.Test)89 ArrayList (java.util.ArrayList)42 Fields (org.apache.storm.tuple.Fields)40 HashMap (java.util.HashMap)39 ChannelHandler (org.apache.storm.sql.runtime.ChannelHandler)26 TridentTopology (org.apache.storm.trident.TridentTopology)21 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)14 Stream (org.apache.storm.trident.Stream)12 TupleImpl (org.apache.storm.tuple.TupleImpl)12 List (java.util.List)11 TestUtils (org.apache.storm.sql.TestUtils)11 TridentState (org.apache.storm.trident.TridentState)11 Tuple (org.apache.storm.tuple.Tuple)11 JSONObject (org.json.simple.JSONObject)10 Map (java.util.Map)9 Config (org.apache.storm.Config)9 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)9 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)8 TridentTuple (org.apache.storm.trident.tuple.TridentTuple)8