Search in sources :

Example 1 with SerializableFunction

use of com.twitter.heron.streamlet.SerializableFunction in project incubator-heron by apache.

the class JoinOperatorTest method getJoinOperator.

@SuppressWarnings({ "rawtypes", "unchecked" })
private JoinOperator<String, KeyValue<String, String>, KeyValue<String, String>, String> getJoinOperator(JoinType type) {
    SerializableFunction<KeyValue<String, String>, String> f = x -> x == null ? "null" : x.getKey();
    JoinOperator<String, KeyValue<String, String>, KeyValue<String, String>, String> joinOperator = new JoinOperator(type, "leftComponent", "rightComponent", f, f, (SerializableBiFunction<KeyValue<String, String>, KeyValue<String, String>, String>) (o, o2) -> (o == null ? "null" : o.getValue()) + (o2 == null ? "null" : o2.getValue()));
    joinOperator.prepare(new Config(), PowerMockito.mock(TopologyContext.class), new OutputCollector(new IOutputCollector() {

        @Override
        public void reportError(Throwable error) {
        }

        @Override
        public List<Integer> emit(String streamId, Collection<Tuple> anchors, List<Object> tuple) {
            emittedTuples.addAll(tuple);
            return null;
        }

        @Override
        public void emitDirect(int taskId, String streamId, Collection<Tuple> anchors, List<Object> tuple) {
        }

        @Override
        public void ack(Tuple input) {
        }

        @Override
        public void fail(Tuple input) {
        }
    }));
    return joinOperator;
}
Also used : TopologyBuilder(com.twitter.heron.api.topology.TopologyBuilder) TopologyAPI(com.twitter.heron.api.generated.TopologyAPI) Config(com.twitter.heron.api.Config) HashMap(java.util.HashMap) HashSet(java.util.HashSet) Values(com.twitter.heron.api.tuple.Values) TupleImpl(com.twitter.heron.common.utils.tuple.TupleImpl) TopologyContextImpl(com.twitter.heron.common.utils.topology.TopologyContextImpl) Map(java.util.Map) Fields(com.twitter.heron.api.tuple.Fields) SerializableFunction(com.twitter.heron.streamlet.SerializableFunction) LinkedList(java.util.LinkedList) PowerMockito(org.powermock.api.mockito.PowerMockito) SerializableBiFunction(com.twitter.heron.streamlet.SerializableBiFunction) TupleWindow(com.twitter.heron.api.windowing.TupleWindow) Before(org.junit.Before) JoinType(com.twitter.heron.streamlet.JoinType) Collection(java.util.Collection) KeyedWindow(com.twitter.heron.streamlet.KeyedWindow) Set(java.util.Set) Tuple(com.twitter.heron.api.tuple.Tuple) Test(org.junit.Test) TopologyContext(com.twitter.heron.api.topology.TopologyContext) Mockito(org.mockito.Mockito) IOutputCollector(com.twitter.heron.api.bolt.IOutputCollector) List(java.util.List) OutputCollector(com.twitter.heron.api.bolt.OutputCollector) KeyValue(com.twitter.heron.streamlet.KeyValue) Assert(org.junit.Assert) Collections(java.util.Collections) TupleWindowImpl(com.twitter.heron.api.windowing.TupleWindowImpl) IOutputCollector(com.twitter.heron.api.bolt.IOutputCollector) OutputCollector(com.twitter.heron.api.bolt.OutputCollector) KeyValue(com.twitter.heron.streamlet.KeyValue) Config(com.twitter.heron.api.Config) IOutputCollector(com.twitter.heron.api.bolt.IOutputCollector) Collection(java.util.Collection) LinkedList(java.util.LinkedList) List(java.util.List) TopologyContext(com.twitter.heron.api.topology.TopologyContext) Tuple(com.twitter.heron.api.tuple.Tuple)

Aggregations

Config (com.twitter.heron.api.Config)1 IOutputCollector (com.twitter.heron.api.bolt.IOutputCollector)1 OutputCollector (com.twitter.heron.api.bolt.OutputCollector)1 TopologyAPI (com.twitter.heron.api.generated.TopologyAPI)1 TopologyBuilder (com.twitter.heron.api.topology.TopologyBuilder)1 TopologyContext (com.twitter.heron.api.topology.TopologyContext)1 Fields (com.twitter.heron.api.tuple.Fields)1 Tuple (com.twitter.heron.api.tuple.Tuple)1 Values (com.twitter.heron.api.tuple.Values)1 TupleWindow (com.twitter.heron.api.windowing.TupleWindow)1 TupleWindowImpl (com.twitter.heron.api.windowing.TupleWindowImpl)1 TopologyContextImpl (com.twitter.heron.common.utils.topology.TopologyContextImpl)1 TupleImpl (com.twitter.heron.common.utils.tuple.TupleImpl)1 JoinType (com.twitter.heron.streamlet.JoinType)1 KeyValue (com.twitter.heron.streamlet.KeyValue)1 KeyedWindow (com.twitter.heron.streamlet.KeyedWindow)1 SerializableBiFunction (com.twitter.heron.streamlet.SerializableBiFunction)1 SerializableFunction (com.twitter.heron.streamlet.SerializableFunction)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1