Search in sources :

Example 41 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class UnionITCase method testUnion2IdenticalDataSets.

@Test
public void testUnion2IdenticalDataSets() throws Exception {
    /*
		 * Union of 2 Same Data Sets
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> unionDs = ds.union(CollectionDataSets.get3TupleDataSet(env));
    List<Tuple3<Integer, Long, String>> result = unionDs.collect();
    String expected = FULL_TUPLE_3_STRING + FULL_TUPLE_3_STRING;
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 42 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ValueCollectionDataSets method getGroupSortedNestedTupleDataSet2.

public static DataSet<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> getGroupSortedNestedTupleDataSet2(ExecutionEnvironment env) {
    List<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> data = new ArrayList<>();
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(1), new IntValue(3)), new StringValue("a"), new IntValue(2)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(1), new IntValue(2)), new StringValue("a"), new IntValue(1)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(2), new IntValue(1)), new StringValue("a"), new IntValue(3)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(2), new IntValue(2)), new StringValue("b"), new IntValue(4)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(3), new IntValue(3)), new StringValue("c"), new IntValue(5)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(3), new IntValue(6)), new StringValue("c"), new IntValue(6)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(4), new IntValue(9)), new StringValue("c"), new IntValue(7)));
    TupleTypeInfo<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> type = new TupleTypeInfo<>(new TupleTypeInfo<Tuple2<IntValue, IntValue>>(ValueTypeInfo.INT_VALUE_TYPE_INFO, ValueTypeInfo.INT_VALUE_TYPE_INFO), ValueTypeInfo.STRING_VALUE_TYPE_INFO, ValueTypeInfo.INT_VALUE_TYPE_INFO);
    return env.fromCollection(data, type);
}
Also used : Tuple2(org.apache.flink.api.java.tuple.Tuple2) Tuple3(org.apache.flink.api.java.tuple.Tuple3) ArrayList(java.util.ArrayList) StringValue(org.apache.flink.types.StringValue) IntValue(org.apache.flink.types.IntValue) TupleTypeInfo(org.apache.flink.api.java.typeutils.TupleTypeInfo)

Example 43 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ReduceITCase method testAllReduceForTuple.

@Test
public void testAllReduceForTuple() throws Exception {
    /*
		 * All-reduce for tuple
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> reduceDs = ds.reduce(new AllAddingTuple3Reduce());
    List<Tuple3<Integer, Long, String>> result = reduceDs.collect();
    String expected = "231,91,Hello World\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 44 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ReduceITCase method testReduceOnTuplesWithKeyFieldSelector.

@Test
public void testReduceOnTuplesWithKeyFieldSelector() throws Exception {
    /*
		 * Reduce on tuples with key field selector
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> reduceDs = ds.groupBy(1).reduce(new Tuple3Reduce("B-)"));
    List<Tuple3<Integer, Long, String>> result = reduceDs.collect();
    String expected = "1,1,Hi\n" + "5,2,B-)\n" + "15,3,B-)\n" + "34,4,B-)\n" + "65,5,B-)\n" + "111,6,B-)\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 45 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ReduceWithCombinerITCase method testReduceOnKeyedDataset.

@Test
public void testReduceOnKeyedDataset() throws Exception {
    // set up the execution environment
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    env.setParallelism(4);
    // creates the input data and distributes them evenly among the available downstream tasks
    DataSet<Tuple3<String, Integer, Boolean>> input = createKeyedInput(env);
    List<Tuple3<String, Integer, Boolean>> actual = input.groupBy(0).reduceGroup(new KeyedCombReducer()).collect();
    String expected = "k1,6,true\nk2,4,true\n";
    compareResultAsTuples(actual, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Aggregations

Tuple3 (org.apache.flink.api.java.tuple.Tuple3)345 Test (org.junit.Test)317 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)264 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)130 Plan (org.apache.flink.api.common.Plan)85 OptimizedPlan (org.apache.flink.optimizer.plan.OptimizedPlan)54 SinkPlanNode (org.apache.flink.optimizer.plan.SinkPlanNode)52 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)49 Tuple5 (org.apache.flink.api.java.tuple.Tuple5)40 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)38 TimeWindow (org.apache.flink.streaming.api.windowing.windows.TimeWindow)38 DualInputPlanNode (org.apache.flink.optimizer.plan.DualInputPlanNode)37 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)26 FieldSet (org.apache.flink.api.common.operators.util.FieldSet)24 EventTimeTrigger (org.apache.flink.streaming.api.windowing.triggers.EventTimeTrigger)23 ReducingStateDescriptor (org.apache.flink.api.common.state.ReducingStateDescriptor)19 InvalidProgramException (org.apache.flink.api.common.InvalidProgramException)18 TumblingEventTimeWindows (org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows)17 MapOperatorBase (org.apache.flink.api.common.operators.base.MapOperatorBase)16 FoldingStateDescriptor (org.apache.flink.api.common.state.FoldingStateDescriptor)16