Search in sources :

Example 1 with TimeCacheMap

use of com.alibaba.jstorm.utils.TimeCacheMap in project jstorm by alibaba.

the class SingleJoinBolt method prepare.

@Override
public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
    _fieldLocations = new HashMap<String, GlobalStreamId>();
    _collector = collector;
    int timeout = ((Number) conf.get(Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS)).intValue();
    _pending = new TimeCacheMap<List<Object>, Map<GlobalStreamId, Tuple>>(timeout, new ExpireCallback());
    _numSources = context.getThisSources().size();
    Set<String> idFields = null;
    for (GlobalStreamId source : context.getThisSources().keySet()) {
        Fields fields = context.getComponentOutputFields(source.get_componentId(), source.get_streamId());
        Set<String> setFields = new HashSet<String>(fields.toList());
        if (idFields == null)
            idFields = setFields;
        else
            idFields.retainAll(setFields);
        for (String outfield : _outFields) {
            for (String sourcefield : fields) {
                if (outfield.equals(sourcefield)) {
                    _fieldLocations.put(outfield, source);
                }
            }
        }
    }
    _idFields = new Fields(new ArrayList<String>(idFields));
    if (_fieldLocations.size() != _outFields.size()) {
        throw new RuntimeException("Cannot find all outfields among sources");
    }
}
Also used : ArrayList(java.util.ArrayList) Fields(backtype.storm.tuple.Fields) GlobalStreamId(backtype.storm.generated.GlobalStreamId) ArrayList(java.util.ArrayList) List(java.util.List) HashMap(java.util.HashMap) Map(java.util.Map) TimeCacheMap(com.alibaba.jstorm.utils.TimeCacheMap) HashSet(java.util.HashSet)

Aggregations

GlobalStreamId (backtype.storm.generated.GlobalStreamId)1 Fields (backtype.storm.tuple.Fields)1 TimeCacheMap (com.alibaba.jstorm.utils.TimeCacheMap)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Map (java.util.Map)1