Search in sources :

Example 1 with ValueUpdater

use of storm.trident.state.ValueUpdater in project jstorm by alibaba.

the class MapCombinerAggStateUpdater method updateState.

@Override
public void updateState(MapState map, List<TridentTuple> tuples, TridentCollector collector) {
    List<List<Object>> groups = new ArrayList<List<Object>>(tuples.size());
    List<ValueUpdater> updaters = new ArrayList<ValueUpdater>(tuples.size());
    for (TridentTuple t : tuples) {
        groups.add(_groupFactory.create(t));
        updaters.add(new CombinerValueUpdater(_agg, _inputFactory.create(t).getValue(0)));
    }
    List<Object> newVals = map.multiUpdate(groups, updaters);
    for (int i = 0; i < tuples.size(); i++) {
        List<Object> key = groups.get(i);
        Object result = newVals.get(i);
        collector.emit(_factory.create(new List[] { key, new Values(result) }));
    }
}
Also used : CombinerValueUpdater(storm.trident.state.CombinerValueUpdater) ValueUpdater(storm.trident.state.ValueUpdater) ArrayList(java.util.ArrayList) Values(backtype.storm.tuple.Values) CombinerValueUpdater(storm.trident.state.CombinerValueUpdater) ArrayList(java.util.ArrayList) List(java.util.List) ComboList(storm.trident.tuple.ComboList) TridentTuple(storm.trident.tuple.TridentTuple)

Example 2 with ValueUpdater

use of storm.trident.state.ValueUpdater in project storm by nathanmarz.

the class MapReducerAggStateUpdater method updateState.

@Override
public void updateState(MapState map, List<TridentTuple> tuples, TridentCollector collector) {
    Map<List<Object>, List<TridentTuple>> grouped = new HashMap();
    List<List<Object>> groups = new ArrayList<List<Object>>(tuples.size());
    List<Object> values = new ArrayList<Object>(tuples.size());
    for (TridentTuple t : tuples) {
        List<Object> group = _groupFactory.create(t);
        List<TridentTuple> groupTuples = grouped.get(group);
        if (groupTuples == null) {
            groupTuples = new ArrayList();
            grouped.put(group, groupTuples);
        }
        groupTuples.add(_inputFactory.create(t));
    }
    List<List<Object>> uniqueGroups = new ArrayList(grouped.keySet());
    List<ValueUpdater> updaters = new ArrayList(uniqueGroups.size());
    for (List<Object> group : uniqueGroups) {
        updaters.add(new ReducerValueUpdater(_agg, grouped.get(group)));
    }
    List<Object> results = map.multiUpdate(uniqueGroups, updaters);
    for (int i = 0; i < uniqueGroups.size(); i++) {
        List<Object> group = uniqueGroups.get(i);
        Object result = results.get(i);
        collector.emit(_factory.create(new List[] { group, new Values(result) }));
    }
}
Also used : HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Values(backtype.storm.tuple.Values) ReducerValueUpdater(storm.trident.state.ReducerValueUpdater) ValueUpdater(storm.trident.state.ValueUpdater) ReducerValueUpdater(storm.trident.state.ReducerValueUpdater) ArrayList(java.util.ArrayList) List(java.util.List) ComboList(storm.trident.tuple.ComboList) TridentTuple(storm.trident.tuple.TridentTuple)

Example 3 with ValueUpdater

use of storm.trident.state.ValueUpdater in project storm by nathanmarz.

the class MapCombinerAggStateUpdater method updateState.

@Override
public void updateState(MapState map, List<TridentTuple> tuples, TridentCollector collector) {
    List<List<Object>> groups = new ArrayList<List<Object>>(tuples.size());
    List<ValueUpdater> updaters = new ArrayList<ValueUpdater>(tuples.size());
    for (TridentTuple t : tuples) {
        groups.add(_groupFactory.create(t));
        updaters.add(new CombinerValueUpdater(_agg, _inputFactory.create(t).getValue(0)));
    }
    List<Object> newVals = map.multiUpdate(groups, updaters);
    for (int i = 0; i < tuples.size(); i++) {
        List<Object> key = groups.get(i);
        Object result = newVals.get(i);
        collector.emit(_factory.create(new List[] { key, new Values(result) }));
    }
}
Also used : CombinerValueUpdater(storm.trident.state.CombinerValueUpdater) ValueUpdater(storm.trident.state.ValueUpdater) ArrayList(java.util.ArrayList) Values(backtype.storm.tuple.Values) CombinerValueUpdater(storm.trident.state.CombinerValueUpdater) ArrayList(java.util.ArrayList) List(java.util.List) ComboList(storm.trident.tuple.ComboList) TridentTuple(storm.trident.tuple.TridentTuple)

Example 4 with ValueUpdater

use of storm.trident.state.ValueUpdater in project jstorm by alibaba.

the class MapReducerAggStateUpdater method updateState.

@Override
public void updateState(MapState map, List<TridentTuple> tuples, TridentCollector collector) {
    Map<List<Object>, List<TridentTuple>> grouped = new HashMap<>();
    for (TridentTuple t : tuples) {
        List<Object> group = _groupFactory.create(t);
        List<TridentTuple> groupTuples = grouped.get(group);
        if (groupTuples == null) {
            groupTuples = new ArrayList<>();
            grouped.put(group, groupTuples);
        }
        groupTuples.add(_inputFactory.create(t));
    }
    List<List<Object>> uniqueGroups = new ArrayList<>(grouped.keySet());
    List<ValueUpdater> updaters = new ArrayList<>(uniqueGroups.size());
    for (List<Object> group : uniqueGroups) {
        updaters.add(new ReducerValueUpdater(_agg, grouped.get(group)));
    }
    List<Object> results = map.multiUpdate(uniqueGroups, updaters);
    for (int i = 0; i < uniqueGroups.size(); i++) {
        List<Object> group = uniqueGroups.get(i);
        Object result = results.get(i);
        collector.emit(_factory.create(new List[] { group, new Values(result) }));
    }
}
Also used : HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Values(backtype.storm.tuple.Values) ReducerValueUpdater(storm.trident.state.ReducerValueUpdater) ValueUpdater(storm.trident.state.ValueUpdater) ReducerValueUpdater(storm.trident.state.ReducerValueUpdater) ArrayList(java.util.ArrayList) List(java.util.List) ComboList(storm.trident.tuple.ComboList) TridentTuple(storm.trident.tuple.TridentTuple)

Aggregations

Values (backtype.storm.tuple.Values)4 ArrayList (java.util.ArrayList)4 List (java.util.List)4 ValueUpdater (storm.trident.state.ValueUpdater)4 ComboList (storm.trident.tuple.ComboList)4 TridentTuple (storm.trident.tuple.TridentTuple)4 HashMap (java.util.HashMap)2 CombinerValueUpdater (storm.trident.state.CombinerValueUpdater)2 ReducerValueUpdater (storm.trident.state.ReducerValueUpdater)2