Search in sources :

Example 56 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class FilterITCase method testFilterOnIntegerTupleField.

@Test
public void testFilterOnIntegerTupleField() throws Exception {
    /*
		 * Test filter on Integer tuple field.
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> filterDs = ds.filter(new Filter4());
    List<Tuple3<Integer, Long, String>> result = filterDs.collect();
    String expected = "2,2,Hello\n" + "4,3,Hello world, how are you?\n" + "6,3,Luke Skywalker\n" + "8,4,Comment#2\n" + "10,4,Comment#4\n" + "12,5,Comment#6\n" + "14,5,Comment#8\n" + "16,6,Comment#10\n" + "18,6,Comment#12\n" + "20,6,Comment#14\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 57 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class FilterITCase method testAllPassingFilter.

@Test
public void testAllPassingFilter() throws Exception {
    /*
		 * Test all-passing filter.
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> filterDs = ds.filter(new Filter2());
    List<Tuple3<Integer, Long, String>> result = filterDs.collect();
    String expected = "1,1,Hi\n" + "2,2,Hello\n" + "3,2,Hello world\n" + "4,3,Hello world, how are you?\n" + "5,3,I am fine.\n" + "6,3,Luke Skywalker\n" + "7,4,Comment#1\n" + "8,4,Comment#2\n" + "9,4,Comment#3\n" + "10,4,Comment#4\n" + "11,5,Comment#5\n" + "12,5,Comment#6\n" + "13,5,Comment#7\n" + "14,5,Comment#8\n" + "15,5,Comment#9\n" + "16,6,Comment#10\n" + "17,6,Comment#11\n" + "18,6,Comment#12\n" + "19,6,Comment#13\n" + "20,6,Comment#14\n" + "21,6,Comment#15\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 58 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class FilterITCase method testFilterWithBroadcastVariables.

@Test
public void testFilterWithBroadcastVariables() throws Exception {
    /*
		 * Test filter with broadcast variables
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Integer> intDs = CollectionDataSets.getIntegerDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> filterDs = ds.filter(new RichFilter2()).withBroadcastSet(intDs, "ints");
    List<Tuple3<Integer, Long, String>> result = filterDs.collect();
    String expected = "11,5,Comment#5\n" + "12,5,Comment#6\n" + "13,5,Comment#7\n" + "14,5,Comment#8\n" + "15,5,Comment#9\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 59 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class FilterITCase method testRichFilterOnStringTupleField.

@Test
public void testRichFilterOnStringTupleField() throws Exception {
    /*
		 * Test filter on String tuple field.
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Integer> ints = CollectionDataSets.getIntegerDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> filterDs = ds.filter(new RichFilter1()).withBroadcastSet(ints, "ints");
    List<Tuple3<Integer, Long, String>> result = filterDs.collect();
    String expected = "1,1,Hi\n" + "2,2,Hello\n" + "3,2,Hello world\n" + "4,3,Hello world, how are you?\n";
    compareResultAsTuples(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 60 with Tuple3

use of org.apache.flink.api.java.tuple.Tuple3 in project flink by apache.

the class FirstNITCase method testFirstNOnGroupedAndSortedDS.

@Test
public void testFirstNOnGroupedAndSortedDS() throws Exception {
    /*
		 * First-n on grouped and sorted data set
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);
    DataSet<Tuple2<Long, Integer>> first = ds.groupBy(1).sortGroup(0, Order.DESCENDING).first(3).project(1, 0);
    List<Tuple2<Long, Integer>> result = first.collect();
    String expected = "(1,1)\n" + "(2,3)\n(2,2)\n" + "(3,6)\n(3,5)\n(3,4)\n" + "(4,10)\n(4,9)\n(4,8)\n" + "(5,15)\n(5,14)\n(5,13)\n" + "(6,21)\n(6,20)\n(6,19)\n";
    compareResultAsText(result, expected);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple2(org.apache.flink.api.java.tuple.Tuple2) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Aggregations

Tuple3 (org.apache.flink.api.java.tuple.Tuple3)559 Test (org.junit.Test)506 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)415 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)182 Plan (org.apache.flink.api.common.Plan)89 Tuple5 (org.apache.flink.api.java.tuple.Tuple5)74 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)63 OptimizedPlan (org.apache.flink.optimizer.plan.OptimizedPlan)55 SinkPlanNode (org.apache.flink.optimizer.plan.SinkPlanNode)53 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)43 TimeWindow (org.apache.flink.streaming.api.windowing.windows.TimeWindow)43 DualInputPlanNode (org.apache.flink.optimizer.plan.DualInputPlanNode)38 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)37 IOException (java.io.IOException)32 ArrayList (java.util.ArrayList)31 Configuration (org.apache.flink.configuration.Configuration)29 EventTimeTrigger (org.apache.flink.streaming.api.windowing.triggers.EventTimeTrigger)27 FieldSet (org.apache.flink.api.common.operators.util.FieldSet)24 TypeHint (org.apache.flink.api.common.typeinfo.TypeHint)24 Tuple1 (org.apache.flink.api.java.tuple.Tuple1)21