Search in sources :

Example 1 with Triplet

use of org.apache.flink.graph.Triplet in project flink by apache.

the class GraphCreationWithCsvITCase method testCsvWithConstantValueMapper.

@Test
public void testCsvWithConstantValueMapper() throws Exception {
    /*
         *Test fromCsvReader with edge path and a mapper that assigns a Double constant as value
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    final String fileContent = "1,2,ot\n" + "3,2,tt\n" + "3,1,to\n";
    final FileInputSplit split = createTempFile(fileContent);
    Graph<Long, Double, String> graph = Graph.fromCsvReader(split.getPath().toString(), new AssignDoubleValueMapper(), env).types(Long.class, Double.class, String.class);
    List<Triplet<Long, Double, String>> result = graph.getTriplets().collect();
    // graph.getTriplets().writeAsCsv(resultPath);
    expectedResult = "1,2,0.1,0.1,ot\n" + "3,1,0.1,0.1,to\n" + "3,2,0.1,0.1,tt\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Triplet(org.apache.flink.graph.Triplet) Test(org.junit.Test)

Example 2 with Triplet

use of org.apache.flink.graph.Triplet in project flink by apache.

the class GraphCreationWithCsvITCase method testCreateCsvFileDelimiterConfiguration.

@Test
public void testCreateCsvFileDelimiterConfiguration() throws Exception {
    /*
         * Test with an Edge and Vertex csv file. Tests the configuration methods FieldDelimiterEdges and
         * FieldDelimiterVertices
         * Also tests the configuration methods LineDelimiterEdges and LineDelimiterVertices
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    final String fileContent = "header\n1;1\n" + "2;2\n" + "3;3\n";
    final FileInputSplit split = createTempFile(fileContent);
    final String fileContent2 = "header|1:2:ot|" + "3:2:tt|" + "3:1:to|";
    final FileInputSplit split2 = createTempFile(fileContent2);
    Graph<Long, Long, String> graph = Graph.fromCsvReader(split.getPath().toString(), split2.getPath().toString(), env).ignoreFirstLineEdges().ignoreFirstLineVertices().fieldDelimiterEdges(":").fieldDelimiterVertices(";").lineDelimiterEdges("|").types(Long.class, Long.class, String.class);
    List<Triplet<Long, Long, String>> result = graph.getTriplets().collect();
    expectedResult = "1,2,1,2,ot\n" + "3,2,3,2,tt\n" + "3,1,3,1,to\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Triplet(org.apache.flink.graph.Triplet) Test(org.junit.Test)

Example 3 with Triplet

use of org.apache.flink.graph.Triplet in project flink by apache.

the class GraphCreationWithCsvITCase method testCreateWithCsvFile.

@Test
public void testCreateWithCsvFile() throws Exception {
    /*
         * Test with two Csv files one with Vertex Data and one with Edges data
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    final String fileContent = "1,1\n" + "2,2\n" + "3,3\n";
    final FileInputSplit split = createTempFile(fileContent);
    final String fileContent2 = "1,2,ot\n" + "3,2,tt\n" + "3,1,to\n";
    final FileInputSplit split2 = createTempFile(fileContent2);
    Graph<Long, Long, String> graph = Graph.fromCsvReader(split.getPath().toString(), split2.getPath().toString(), env).types(Long.class, Long.class, String.class);
    List<Triplet<Long, Long, String>> result = graph.getTriplets().collect();
    expectedResult = "1,2,1,2,ot\n" + "3,2,3,2,tt\n" + "3,1,3,1,to\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Triplet(org.apache.flink.graph.Triplet) Test(org.junit.Test)

Example 4 with Triplet

use of org.apache.flink.graph.Triplet in project flink by apache.

the class GraphOperationsITCase method testTriplets.

@Test
public void testTriplets() throws Exception {
    /*
         * Test getTriplets()
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    Graph<Long, Long, Long> graph = Graph.fromDataSet(TestGraphUtils.getLongLongVertexData(env), TestGraphUtils.getLongLongEdgeData(env), env);
    DataSet<Triplet<Long, Long, Long>> data = graph.getTriplets();
    List<Triplet<Long, Long, Long>> result = data.collect();
    expectedResult = "1,2,1,2,12\n" + "1,3,1,3,13\n" + "2,3,2,3,23\n" + "3,4,3,4,34\n" + "3,5,3,5,35\n" + "4,5,4,5,45\n" + "5,1,5,1,51\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Triplet(org.apache.flink.graph.Triplet) Test(org.junit.Test)

Example 5 with Triplet

use of org.apache.flink.graph.Triplet in project flink by apache.

the class GraphCreationWithCsvITCase method testCsvWithNullEdge.

@Test
public void testCsvWithNullEdge() throws Exception {
    /*
        Test fromCsvReader with edge and vertex path and nullvalue for edge
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    final String vertexFileContent = "1,one\n" + "2,two\n" + "3,three\n";
    final String edgeFileContent = "1,2\n" + "3,2\n" + "3,1\n";
    final FileInputSplit split = createTempFile(vertexFileContent);
    final FileInputSplit edgeSplit = createTempFile(edgeFileContent);
    Graph<Long, String, NullValue> graph = Graph.fromCsvReader(split.getPath().toString(), edgeSplit.getPath().toString(), env).vertexTypes(Long.class, String.class);
    List<Triplet<Long, String, NullValue>> result = graph.getTriplets().collect();
    expectedResult = "1,2,one,two,(null)\n" + "3,2,three,two,(null)\n" + "3,1,three,one,(null)\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) NullValue(org.apache.flink.types.NullValue) Triplet(org.apache.flink.graph.Triplet) Test(org.junit.Test)

Aggregations

ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)7 Triplet (org.apache.flink.graph.Triplet)7 Test (org.junit.Test)6 FileInputSplit (org.apache.flink.core.fs.FileInputSplit)5 NullValue (org.apache.flink.types.NullValue)2 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)1 Edge (org.apache.flink.graph.Edge)1 Vertex (org.apache.flink.graph.Vertex)1