Search in sources :

Example 16 with Edge

use of org.apache.flink.graph.Edge in project flink by apache.

the class CirculantGraph method generate.

@Override
public Graph<LongValue, NullValue, NullValue> generate() {
    // Vertices
    DataSet<Vertex<LongValue, NullValue>> vertices = GraphGeneratorUtils.vertexSequence(env, parallelism, vertexCount);
    // Edges
    LongValueSequenceIterator iterator = new LongValueSequenceIterator(0, this.vertexCount - 1);
    // Validate ranges
    Collections.sort(offsetRanges);
    Iterator<OffsetRange> iter = offsetRanges.iterator();
    OffsetRange lastRange = iter.next();
    while (iter.hasNext()) {
        OffsetRange nextRange = iter.next();
        if (lastRange.overlaps(nextRange)) {
            throw new IllegalArgumentException("Overlapping ranges " + lastRange + " and " + nextRange);
        }
        lastRange = nextRange;
    }
    DataSet<Edge<LongValue, NullValue>> edges = env.fromParallelCollection(iterator, LongValue.class).setParallelism(parallelism).name("Edge iterators").flatMap(new LinkVertexToOffsets(vertexCount, offsetRanges)).setParallelism(parallelism).name("Circulant graph edges");
    // Graph
    return Graph.fromDataSet(vertices, edges, env);
}
Also used : LongValueSequenceIterator(org.apache.flink.util.LongValueSequenceIterator) Vertex(org.apache.flink.graph.Vertex) LongValue(org.apache.flink.types.LongValue) Edge(org.apache.flink.graph.Edge)

Example 17 with Edge

use of org.apache.flink.graph.Edge in project flink by apache.

the class GridGraph method generate.

@Override
public Graph<LongValue, NullValue, NullValue> generate() {
    Preconditions.checkState(!dimensions.isEmpty(), "No dimensions added to GridGraph");
    // Vertices
    DataSet<Vertex<LongValue, NullValue>> vertices = GraphGeneratorUtils.vertexSequence(env, parallelism, vertexCount);
    // Edges
    LongValueSequenceIterator iterator = new LongValueSequenceIterator(0, this.vertexCount - 1);
    DataSet<Edge<LongValue, NullValue>> edges = env.fromParallelCollection(iterator, LongValue.class).setParallelism(parallelism).name("Edge iterators").flatMap(new LinkVertexToNeighbors(vertexCount, dimensions)).setParallelism(parallelism).name("Grid graph edges");
    // Graph
    return Graph.fromDataSet(vertices, edges, env);
}
Also used : LongValueSequenceIterator(org.apache.flink.util.LongValueSequenceIterator) Vertex(org.apache.flink.graph.Vertex) LongValue(org.apache.flink.types.LongValue) Edge(org.apache.flink.graph.Edge)

Example 18 with Edge

use of org.apache.flink.graph.Edge in project flink by apache.

the class SingletonEdgeGraph method generate.

@Override
public Graph<LongValue, NullValue, NullValue> generate() {
    Preconditions.checkState(vertexPairCount > 0);
    // Vertices
    long vertexCount = 2 * vertexPairCount;
    DataSet<Vertex<LongValue, NullValue>> vertices = GraphGeneratorUtils.vertexSequence(env, parallelism, vertexCount);
    // Edges
    LongValueSequenceIterator iterator = new LongValueSequenceIterator(0, vertexCount - 1);
    DataSet<Edge<LongValue, NullValue>> edges = env.fromParallelCollection(iterator, LongValue.class).setParallelism(parallelism).name("Edge iterators").map(new LinkVertexToSingletonNeighbor()).setParallelism(parallelism).name("Complete graph edges");
    // Graph
    return Graph.fromDataSet(vertices, edges, env);
}
Also used : LongValueSequenceIterator(org.apache.flink.util.LongValueSequenceIterator) Vertex(org.apache.flink.graph.Vertex) LongValue(org.apache.flink.types.LongValue) Edge(org.apache.flink.graph.Edge)

Example 19 with Edge

use of org.apache.flink.graph.Edge in project flink by apache.

the class StarGraph method generate.

@Override
public Graph<LongValue, NullValue, NullValue> generate() {
    Preconditions.checkState(vertexCount >= 2);
    // Vertices
    DataSet<Vertex<LongValue, NullValue>> vertices = GraphGeneratorUtils.vertexSequence(env, parallelism, vertexCount);
    // Edges
    LongValueSequenceIterator iterator = new LongValueSequenceIterator(1, this.vertexCount - 1);
    DataSet<Edge<LongValue, NullValue>> edges = env.fromParallelCollection(iterator, LongValue.class).setParallelism(parallelism).name("Edge iterators").flatMap(new LinkVertexToCenter()).setParallelism(parallelism).name("Star graph edges");
    // Graph
    return Graph.fromDataSet(vertices, edges, env);
}
Also used : LongValueSequenceIterator(org.apache.flink.util.LongValueSequenceIterator) Vertex(org.apache.flink.graph.Vertex) LongValue(org.apache.flink.types.LongValue) Edge(org.apache.flink.graph.Edge)

Example 20 with Edge

use of org.apache.flink.graph.Edge in project flink by apache.

the class GatherSumApplyConfigurationITCase method testIterationDefaultDirection.

@Test
public void testIterationDefaultDirection() throws Exception {
    /*
         * Test that if no direction parameter is given, the iteration works as before
         * (i.e. it gathers information from the IN edges and neighbors and the information is calculated for an OUT edge
         * Default direction parameter is OUT for the GatherSumApplyIterations)
         * When data is gathered from the IN edges the Gather Sum and Apply functions
         * set the set of vertices which have path to a vertex as the value of that vertex
         */
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    List<Edge<Long, Long>> edges = TestGraphUtils.getLongLongEdges();
    edges.remove(0);
    Graph<Long, HashSet<Long>, Long> graph = Graph.fromCollection(TestGraphUtils.getLongLongVertices(), edges, env).mapVertices(new GatherSumApplyConfigurationITCase.InitialiseHashSetMapper());
    DataSet<Vertex<Long, HashSet<Long>>> resultedVertices = graph.runGatherSumApplyIteration(new GetReachableVertices(), new FindAllReachableVertices(), new UpdateReachableVertices(), 4).getVertices();
    List<Vertex<Long, HashSet<Long>>> result = resultedVertices.collect();
    expectedResult = "1,[1, 2, 3, 4, 5]\n" + "2,[2]\n" + "3,[1, 2, 3, 4, 5]\n" + "4,[1, 2, 3, 4, 5]\n" + "5,[1, 2, 3, 4, 5]\n";
    compareResultAsTuples(result, expectedResult);
}
Also used : Vertex(org.apache.flink.graph.Vertex) ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) Edge(org.apache.flink.graph.Edge) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

Edge (org.apache.flink.graph.Edge)87 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)72 Test (org.junit.Test)69 Vertex (org.apache.flink.graph.Vertex)46 NullValue (org.apache.flink.types.NullValue)20 LongValue (org.apache.flink.types.LongValue)16 ArrayList (java.util.ArrayList)14 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)12 ChecksumHashCode (org.apache.flink.graph.asm.dataset.ChecksumHashCode)7 Checksum (org.apache.flink.graph.asm.dataset.ChecksumHashCode.Checksum)7 Plan (org.apache.flink.api.common.Plan)6 FieldList (org.apache.flink.api.common.operators.util.FieldList)6 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)6 DualInputPlanNode (org.apache.flink.optimizer.plan.DualInputPlanNode)6 OptimizedPlan (org.apache.flink.optimizer.plan.OptimizedPlan)6 PlanNode (org.apache.flink.optimizer.plan.PlanNode)6 SinkPlanNode (org.apache.flink.optimizer.plan.SinkPlanNode)6 WorksetIterationPlanNode (org.apache.flink.optimizer.plan.WorksetIterationPlanNode)6 LinkedList (java.util.LinkedList)5 LongValueSequenceIterator (org.apache.flink.util.LongValueSequenceIterator)5