Search in sources :

Example 71 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class UnionITCase method testUnionWithEmptyDataSet.

@Test
public void testUnionWithEmptyDataSet() throws Exception {
    /*
		 * Test on union with empty dataset
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    // Don't know how to make an empty result in an other way than filtering it
    DataSet<Tuple3<Integer, Long, String>> empty = CollectionDataSets.get3TupleDataSet(env).filter(new RichFilter1());
    DataSet<Tuple3<Integer, Long, String>> unionDs = CollectionDataSets.get3TupleDataSet(env).union(empty);
    List<Tuple3<Integer, Long, String>> result = unionDs.collect();
    String expected = FULL_TUPLE_3_STRING;
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 72 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class UnionITCase method testUnion2IdenticalDataSets.

@Test
public void testUnion2IdenticalDataSets() throws Exception {
    /*
		 * Union of 2 Same Data Sets
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> unionDs = ds.union(CollectionDataSets.get3TupleDataSet(env));
    List<Tuple3<Integer, Long, String>> result = unionDs.collect();
    String expected = FULL_TUPLE_3_STRING + FULL_TUPLE_3_STRING;
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 73 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ValueCollectionDataSets method getGroupSortedNestedTupleDataSet2.

public static DataSet<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> getGroupSortedNestedTupleDataSet2(ExecutionEnvironment env) {
    List<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> data = new ArrayList<>();
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(1), new IntValue(3)), new StringValue("a"), new IntValue(2)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(1), new IntValue(2)), new StringValue("a"), new IntValue(1)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(2), new IntValue(1)), new StringValue("a"), new IntValue(3)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(2), new IntValue(2)), new StringValue("b"), new IntValue(4)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(3), new IntValue(3)), new StringValue("c"), new IntValue(5)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(3), new IntValue(6)), new StringValue("c"), new IntValue(6)));
    data.add(new Tuple3<>(new Tuple2<IntValue, IntValue>(new IntValue(4), new IntValue(9)), new StringValue("c"), new IntValue(7)));
    TupleTypeInfo<Tuple3<Tuple2<IntValue, IntValue>, StringValue, IntValue>> type = new TupleTypeInfo<>(new TupleTypeInfo<Tuple2<IntValue, IntValue>>(ValueTypeInfo.INT_VALUE_TYPE_INFO, ValueTypeInfo.INT_VALUE_TYPE_INFO), ValueTypeInfo.STRING_VALUE_TYPE_INFO, ValueTypeInfo.INT_VALUE_TYPE_INFO);
    return env.fromCollection(data, type);
}
Also used : Tuple2(org.apache.flink.api.java.tuple.Tuple2) Tuple3(org.apache.flink.api.java.tuple.Tuple3) ArrayList(java.util.ArrayList) StringValue(org.apache.flink.types.StringValue) IntValue(org.apache.flink.types.IntValue) TupleTypeInfo(org.apache.flink.api.java.typeutils.TupleTypeInfo)

Example 74 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ReduceITCase method testAllReduceForTuple.

@Test
public void testAllReduceForTuple() throws Exception {
    /*
		 * All-reduce for tuple
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> reduceDs = ds.reduce(new AllAddingTuple3Reduce());
    List<Tuple3<Integer, Long, String>> result = reduceDs.collect();
    String expected = "231,91,Hello World\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 75 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class ReduceITCase method testReduceOnTuplesWithKeyFieldSelector.

@Test
public void testReduceOnTuplesWithKeyFieldSelector() throws Exception {
    /*
		 * Reduce on tuples with key field selector
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> reduceDs = ds.groupBy(1).reduce(new Tuple3Reduce("B-)"));
    List<Tuple3<Integer, Long, String>> result = reduceDs.collect();
    String expected = "1,1,Hi\n" + "5,2,B-)\n" + "15,3,B-)\n" + "34,4,B-)\n" + "65,5,B-)\n" + "111,6,B-)\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Aggregations

Tuple3 (org.apache.flink.api.java.tuple.Tuple3)559 Test (org.junit.Test)506 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)415 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)182 Plan (org.apache.flink.api.common.Plan)89 Tuple5 (org.apache.flink.api.java.tuple.Tuple5)74 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)63 OptimizedPlan (org.apache.flink.optimizer.plan.OptimizedPlan)55 SinkPlanNode (org.apache.flink.optimizer.plan.SinkPlanNode)53 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)43 TimeWindow (org.apache.flink.streaming.api.windowing.windows.TimeWindow)43 DualInputPlanNode (org.apache.flink.optimizer.plan.DualInputPlanNode)38 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)37 IOException (java.io.IOException)32 ArrayList (java.util.ArrayList)31 Configuration (org.apache.flink.configuration.Configuration)29 EventTimeTrigger (org.apache.flink.streaming.api.windowing.triggers.EventTimeTrigger)27 FieldSet (org.apache.flink.api.common.operators.util.FieldSet)24 TypeHint (org.apache.flink.api.common.typeinfo.TypeHint)24 Tuple1 (org.apache.flink.api.java.tuple.Tuple1)21