Search in sources :

Example 91 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class MongoState method batchRetrieve.

public List<List<Values>> batchRetrieve(List<TridentTuple> tridentTuples) {
    List<List<Values>> batchRetrieveResult = Lists.newArrayList();
    try {
        for (TridentTuple tuple : tridentTuples) {
            Bson filter = options.queryCreator.createFilter(tuple);
            Document doc = mongoClient.find(filter);
            List<Values> values = options.lookupMapper.toTuple(tuple, doc);
            batchRetrieveResult.add(values);
        }
    } catch (Exception e) {
        LOG.warn("Batch get operation failed. Triggering replay.", e);
        throw new FailedException(e);
    }
    return batchRetrieveResult;
}
Also used : FailedException(org.apache.storm.topology.FailedException) Values(org.apache.storm.tuple.Values) List(java.util.List) Document(org.bson.Document) FailedException(org.apache.storm.topology.FailedException) TridentTuple(org.apache.storm.trident.tuple.TridentTuple) Bson(org.bson.conversions.Bson)

Example 92 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class MongoLookupBolt method execute.

@Override
public void execute(Tuple tuple) {
    if (TupleUtils.isTick(tuple)) {
        return;
    }
    try {
        //get query filter
        Bson filter = queryCreator.createFilter(tuple);
        //find document from mongodb
        Document doc = mongoClient.find(filter);
        //get storm values and emit
        List<Values> valuesList = mapper.toTuple(tuple, doc);
        for (Values values : valuesList) {
            this.collector.emit(tuple, values);
        }
        this.collector.ack(tuple);
    } catch (Exception e) {
        this.collector.reportError(e);
        this.collector.fail(tuple);
    }
}
Also used : Values(org.apache.storm.tuple.Values) Document(org.bson.Document) Bson(org.bson.conversions.Bson)

Example 93 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class SimpleMongoLookupMapper method toTuple.

@Override
public List<Values> toTuple(ITuple input, Document doc) {
    Values values = new Values();
    for (String field : fields) {
        if (input.contains(field)) {
            values.add(input.getValueByField(field));
        } else {
            values.add(doc.get(field));
        }
    }
    List<Values> result = new ArrayList<Values>();
    result.add(values);
    return result;
}
Also used : Values(org.apache.storm.tuple.Values) ArrayList(java.util.ArrayList)

Example 94 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class PrepareRequest method execute.

@Override
public void execute(Tuple tuple, BasicOutputCollector collector) {
    String args = tuple.getString(0);
    String returnInfo = tuple.getString(1);
    long requestId = rand.nextLong();
    collector.emit(ARGS_STREAM, new Values(requestId, args));
    collector.emit(RETURN_STREAM, new Values(requestId, returnInfo));
    collector.emit(ID_STREAM, new Values(requestId));
}
Also used : Values(org.apache.storm.tuple.Values)

Example 95 with Values

use of org.apache.storm.tuple.Values in project storm by apache.

the class Executor method setupTicks.

protected void setupTicks(boolean isSpout) {
    final Integer tickTimeSecs = Utils.getInt(stormConf.get(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS), null);
    boolean enableMessageTimeout = (Boolean) stormConf.get(Config.TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS);
    if (tickTimeSecs != null) {
        if (Utils.isSystemId(componentId) || (!enableMessageTimeout && isSpout)) {
            LOG.info("Timeouts disabled for executor " + componentId + ":" + executorId);
        } else {
            StormTimer timerTask = workerData.getUserTimer();
            timerTask.scheduleRecurring(tickTimeSecs, tickTimeSecs, new Runnable() {

                @Override
                public void run() {
                    TupleImpl tuple = new TupleImpl(workerTopologyContext, new Values(tickTimeSecs), (int) Constants.SYSTEM_TASK_ID, Constants.SYSTEM_TICK_STREAM_ID);
                    List<AddressedTuple> tickTuple = Lists.newArrayList(new AddressedTuple(AddressedTuple.BROADCAST_DEST, tuple));
                    receiveQueue.publish(tickTuple);
                }
            });
        }
    }
}
Also used : StormTimer(org.apache.storm.StormTimer) Values(org.apache.storm.tuple.Values) List(java.util.List) ArrayList(java.util.ArrayList) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) TupleImpl(org.apache.storm.tuple.TupleImpl) AddressedTuple(org.apache.storm.tuple.AddressedTuple)

Aggregations

Values (org.apache.storm.tuple.Values)206 Test (org.junit.Test)89 ArrayList (java.util.ArrayList)42 Fields (org.apache.storm.tuple.Fields)40 HashMap (java.util.HashMap)39 ChannelHandler (org.apache.storm.sql.runtime.ChannelHandler)26 TridentTopology (org.apache.storm.trident.TridentTopology)21 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)14 Stream (org.apache.storm.trident.Stream)12 TupleImpl (org.apache.storm.tuple.TupleImpl)12 List (java.util.List)11 TestUtils (org.apache.storm.sql.TestUtils)11 TridentState (org.apache.storm.trident.TridentState)11 Tuple (org.apache.storm.tuple.Tuple)11 JSONObject (org.json.simple.JSONObject)10 Map (java.util.Map)9 Config (org.apache.storm.Config)9 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)9 GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)8 TridentTuple (org.apache.storm.trident.tuple.TridentTuple)8