Search in sources :

Example 1 with Tuple5

use of org.apache.flink.api.java.tuple.Tuple5 in project flink by apache.

the class CrossITCase method testCorrectnessOfCrossIfUDFReturnsLeftInputObject.

@Test
public void testCorrectnessOfCrossIfUDFReturnsLeftInputObject() throws Exception {
    /*
		 * check correctness of cross if UDF returns left input object
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.getSmall3TupleDataSet(env);
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds2 = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> crossDs = ds.cross(ds2).with(new Tuple3ReturnLeft());
    List<Tuple3<Integer, Long, String>> result = crossDs.collect();
    String expected = "1,1,Hi\n" + "1,1,Hi\n" + "1,1,Hi\n" + "2,2,Hello\n" + "2,2,Hello\n" + "2,2,Hello\n" + "3,2,Hello world\n" + "3,2,Hello world\n" + "3,2,Hello world\n";
    compareResultAsTuples(result, expected);
}
Also used : Tuple5(org.apache.flink.api.java.tuple.Tuple5) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 2 with Tuple5

use of org.apache.flink.api.java.tuple.Tuple5 in project flink by apache.

the class CrossITCase method testCorrectnessOfCrossIfUDFReturnsRightInputObject.

@Test
public void testCorrectnessOfCrossIfUDFReturnsRightInputObject() throws Exception {
    /*
		 * check correctness of cross if UDF returns right input object
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.getSmall3TupleDataSet(env);
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds2 = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> crossDs = ds.cross(ds2).with(new Tuple5ReturnRight());
    List<Tuple5<Integer, Long, Integer, String, Long>> result = crossDs.collect();
    String expected = "1,1,0,Hallo,1\n" + "1,1,0,Hallo,1\n" + "1,1,0,Hallo,1\n" + "2,2,1,Hallo Welt,2\n" + "2,2,1,Hallo Welt,2\n" + "2,2,1,Hallo Welt,2\n" + "2,3,2,Hallo Welt wie,1\n" + "2,3,2,Hallo Welt wie,1\n" + "2,3,2,Hallo Welt wie,1\n";
    compareResultAsTuples(result, expected);
}
Also used : Tuple5(org.apache.flink.api.java.tuple.Tuple5) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 3 with Tuple5

use of org.apache.flink.api.java.tuple.Tuple5 in project flink by apache.

the class CrossITCase method testCorrectnessOfCrossWithHuge.

@Test
public void testCorrectnessOfCrossWithHuge() throws Exception {
    /*
		 * check correctness of crossWithHuge (only correctness of result -> should be the same as with normal cross)
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds2 = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<Tuple2<Integer, String>> crossDs = ds.crossWithHuge(ds2).with(new Tuple5Cross());
    List<Tuple2<Integer, String>> result = crossDs.collect();
    String expected = "0,HalloHallo\n" + "1,HalloHallo Welt\n" + "2,HalloHallo Welt wie\n" + "1,Hallo WeltHallo\n" + "2,Hallo WeltHallo Welt\n" + "3,Hallo WeltHallo Welt wie\n" + "2,Hallo Welt wieHallo\n" + "3,Hallo Welt wieHallo Welt\n" + "4,Hallo Welt wieHallo Welt wie\n";
    compareResultAsTuples(result, expected);
}
Also used : Tuple5(org.apache.flink.api.java.tuple.Tuple5) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple2(org.apache.flink.api.java.tuple.Tuple2) Test(org.junit.Test)

Example 4 with Tuple5

use of org.apache.flink.api.java.tuple.Tuple5 in project flink by apache.

the class CrossITCase method testCorrectnessOfCrossATupleInputAndACustomTypeInput.

@Test
public void testCorrectnessOfCrossATupleInputAndACustomTypeInput() throws Exception {
    /*
		 * check correctness of cross a tuple input and a custom type input
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<CustomType> ds2 = CollectionDataSets.getSmallCustomTypeDataSet(env);
    DataSet<Tuple3<Integer, Long, String>> crossDs = ds.cross(ds2).with(new MixedCross());
    List<Tuple3<Integer, Long, String>> result = crossDs.collect();
    String expected = "2,0,HalloHi\n" + "3,0,HalloHello\n" + "3,0,HalloHello world\n" + "3,0,Hallo WeltHi\n" + "4,1,Hallo WeltHello\n" + "4,2,Hallo WeltHello world\n" + "3,0,Hallo Welt wieHi\n" + "4,2,Hallo Welt wieHello\n" + "4,4,Hallo Welt wieHello world\n";
    compareResultAsTuples(result, expected);
}
Also used : CustomType(org.apache.flink.test.javaApiOperators.util.CollectionDataSets.CustomType) Tuple5(org.apache.flink.api.java.tuple.Tuple5) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Example 5 with Tuple5

use of org.apache.flink.api.java.tuple.Tuple5 in project flink by apache.

the class CrossITCase method testProjectCrossOnATupleInput2.

@Test
public void testProjectCrossOnATupleInput2() throws Exception {
    /*
		 * project cross on a tuple input 2
		 */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.getSmall3TupleDataSet(env);
    DataSet<Tuple5<Integer, Long, Integer, String, Long>> ds2 = CollectionDataSets.getSmall5TupleDataSet(env);
    DataSet<Tuple6<String, String, Long, Long, Long, Integer>> crossDs = ds.cross(ds2).projectSecond(3).projectFirst(2, 1).projectSecond(4, 1).projectFirst(0);
    List<Tuple6<String, String, Long, Long, Long, Integer>> result = crossDs.collect();
    String expected = "Hallo,Hi,1,1,1,1\n" + "Hallo Welt,Hi,1,2,2,1\n" + "Hallo Welt wie,Hi,1,1,3,1\n" + "Hallo,Hello,2,1,1,2\n" + "Hallo Welt,Hello,2,2,2,2\n" + "Hallo Welt wie,Hello,2,1,3,2\n" + "Hallo,Hello world,2,1,1,3\n" + "Hallo Welt,Hello world,2,2,2,3\n" + "Hallo Welt wie,Hello world,2,1,3,3\n";
    compareResultAsTuples(result, expected);
}
Also used : Tuple5(org.apache.flink.api.java.tuple.Tuple5) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Tuple6(org.apache.flink.api.java.tuple.Tuple6) Tuple3(org.apache.flink.api.java.tuple.Tuple3) Test(org.junit.Test)

Aggregations

Tuple5 (org.apache.flink.api.java.tuple.Tuple5)311 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)293 Test (org.junit.Test)286 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)74 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)47 InvalidProgramException (org.apache.flink.api.common.InvalidProgramException)44 SemanticProperties (org.apache.flink.api.common.operators.SemanticProperties)23 Tuple (org.apache.flink.api.java.tuple.Tuple)13 Tuple1 (org.apache.flink.api.java.tuple.Tuple1)10 Tuple6 (org.apache.flink.api.java.tuple.Tuple6)10 AbstractTest (org.apache.flink.storm.util.AbstractTest)10 IOException (java.io.IOException)7 ArrayList (java.util.ArrayList)7 KeySelector (org.apache.flink.api.java.functions.KeySelector)6 CustomType (org.apache.flink.api.java.operator.JoinOperatorTest.CustomType)5 List (java.util.List)4 Plan (org.apache.flink.api.common.Plan)3 ParseException (org.apache.flink.api.common.io.ParseException)3 Configuration (org.apache.flink.configuration.Configuration)3 FileInputSplit (org.apache.flink.core.fs.FileInputSplit)3