Search in sources :

Example 21 with Values

use of org.apache.heron.api.tuple.Values in project heron by twitter.

the class ReduceByKeyOperator method execute.

@SuppressWarnings("unchecked")
@Override
public void execute(Tuple tuple) {
    R obj = (R) tuple.getValue(0);
    K key = keyExtractor.apply(obj);
    T value = valueExtractor.apply(obj);
    T newValue;
    if (reduceMap.containsKey(key)) {
        newValue = reduceFn.apply(reduceMap.get(key), value);
    } else {
        newValue = value;
    }
    reduceMap.put(key, newValue);
    collector.emit(new Values(new KeyValue<K, T>(key, newValue)));
    collector.ack(tuple);
}
Also used : KeyValue(org.apache.heron.streamlet.KeyValue) Values(org.apache.heron.api.tuple.Values)

Example 22 with Values

use of org.apache.heron.api.tuple.Values in project heron by twitter.

the class SplitOperator method execute.

@SuppressWarnings("unchecked")
@Override
public void execute(Tuple tuple) {
    R obj = (R) tuple.getValue(0);
    for (Map.Entry<String, SerializablePredicate<R>> entry : splitFns.entrySet()) {
        if (entry.getValue().test(obj)) {
            collector.emit(entry.getKey(), new Values(obj));
        }
    }
    collector.ack(tuple);
}
Also used : Values(org.apache.heron.api.tuple.Values) SerializablePredicate(org.apache.heron.streamlet.SerializablePredicate) Map(java.util.Map)

Example 23 with Values

use of org.apache.heron.api.tuple.Values in project heron by twitter.

the class UnionOperator method execute.

@SuppressWarnings("unchecked")
@Override
public void execute(Tuple tuple) {
    I obj = (I) tuple.getValue(0);
    collector.emit(new Values(obj));
    collector.ack(tuple);
}
Also used : Values(org.apache.heron.api.tuple.Values)

Example 24 with Values

use of org.apache.heron.api.tuple.Values in project heron by twitter.

the class FlatMapOperator method execute.

@SuppressWarnings("unchecked")
@Override
public void execute(Tuple tuple) {
    R obj = (R) tuple.getValue(0);
    Iterable<? extends T> result = flatMapFn.apply(obj);
    for (T o : result) {
        collector.emit(new Values(o));
    }
    collector.ack(tuple);
}
Also used : Values(org.apache.heron.api.tuple.Values)

Example 25 with Values

use of org.apache.heron.api.tuple.Values in project heron by twitter.

the class SlidingWindowSumBolt method execute.

@Override
public void execute(TupleWindow inputWindow) {
    /*
             * The inputWindow gives a view of
             * (a) all the events in the window
             * (b) events that expired since last activation of the window
             * (c) events that newly arrived since last activation of the window
             */
    List<Tuple> tuplesInWindow = inputWindow.get();
    List<Tuple> newTuples = inputWindow.getNew();
    List<Tuple> expiredTuples = inputWindow.getExpired();
    LOG.fine("Events in current window: " + tuplesInWindow.size());
    /*
             * Instead of iterating over all the tuples in the window to compute
             * the sum, the values for the new events are added and old events are
             * subtracted. Similar optimizations might be possible in other
             * windowing computations.
             */
    for (Tuple tuple : newTuples) {
        sum += (int) tuple.getValue(0);
    }
    for (Tuple tuple : expiredTuples) {
        sum -= (int) tuple.getValue(0);
    }
    collector.emit(new Values(sum));
}
Also used : Values(org.apache.heron.api.tuple.Values) Tuple(org.apache.heron.api.tuple.Tuple)

Aggregations

Values (org.apache.heron.api.tuple.Values)37 Tuple (org.apache.heron.api.tuple.Tuple)8 KeyValue (org.apache.heron.streamlet.KeyValue)7 TupleWindow (org.apache.heron.api.windowing.TupleWindow)6 Fields (org.apache.heron.api.tuple.Fields)5 IOException (java.io.IOException)3 HashMap (java.util.HashMap)3 LinkedList (java.util.LinkedList)3 TopologyAPI (org.apache.heron.api.generated.TopologyAPI)3 TupleWindowImpl (org.apache.heron.api.windowing.TupleWindowImpl)3 KeyedWindow (org.apache.heron.streamlet.KeyedWindow)2 Window (org.apache.heron.streamlet.Window)2 Test (org.junit.Test)2 ArrayList (java.util.ArrayList)1 Map (java.util.Map)1 Random (java.util.Random)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 TopologyContext (org.apache.heron.api.topology.TopologyContext)1 SerializablePredicate (org.apache.heron.streamlet.SerializablePredicate)1