Search in sources :

Example 11 with Shard

use of com.baidu.hugegraph.structure.graph.Shard in project hugegraph-computer by hugegraph.

the class HugeInputSplitFetcher method fetchEdgeInputSplits.

@Override
public List<InputSplit> fetchEdgeInputSplits() {
    long splitSize = this.config.get(ComputerOptions.INPUT_SPLITS_SIZE);
    int maxSplits = this.config.get(ComputerOptions.INPUT_MAX_SPLITS);
    List<Shard> shards = this.client.traverser().edgeShards(splitSize);
    E.checkArgument(shards.size() <= maxSplits, "Too many shards due to too small splitSize");
    List<InputSplit> splits = new ArrayList<>();
    for (Shard shard : shards) {
        InputSplit split = new InputSplit(shard.start(), shard.end());
        splits.add(split);
    }
    return splits;
}
Also used : ArrayList(java.util.ArrayList) Shard(com.baidu.hugegraph.structure.graph.Shard) InputSplit(com.baidu.hugegraph.computer.core.input.InputSplit)

Example 12 with Shard

use of com.baidu.hugegraph.structure.graph.Shard in project incubator-hugegraph-toolchain by apache.

the class CommonTraverserApiTest method testScanVertexInPaging.

@Test
public void testScanVertexInPaging() {
    List<Shard> shards = verticesAPI.shards(1 * 1024 * 1024);
    List<Vertex> vertices = new LinkedList<>();
    for (Shard shard : shards) {
        String page = "";
        while (page != null) {
            Vertices results = verticesAPI.scan(shard, page, DEFAULT_PAGE_LIMIT);
            vertices.addAll(ImmutableList.copyOf(results.results()));
            page = results.page();
        }
    }
    Assert.assertEquals(6, vertices.size());
}
Also used : Vertex(com.baidu.hugegraph.structure.graph.Vertex) Shard(com.baidu.hugegraph.structure.graph.Shard) LinkedList(java.util.LinkedList) Vertices(com.baidu.hugegraph.structure.graph.Vertices) Test(org.junit.Test) BaseApiTest(com.baidu.hugegraph.api.BaseApiTest)

Example 13 with Shard

use of com.baidu.hugegraph.structure.graph.Shard in project incubator-hugegraph-toolchain by apache.

the class CommonTraverserApiTest method testScanVertex.

@Test
public void testScanVertex() {
    List<Shard> shards = verticesAPI.shards(1 * 1024 * 1024);
    List<Vertex> vertices = new LinkedList<>();
    for (Shard shard : shards) {
        Vertices results = verticesAPI.scan(shard, null, 0L);
        vertices.addAll(ImmutableList.copyOf(results.results()));
        Assert.assertNull(results.page());
    }
    Assert.assertEquals(6, vertices.size());
}
Also used : Vertex(com.baidu.hugegraph.structure.graph.Vertex) Shard(com.baidu.hugegraph.structure.graph.Shard) LinkedList(java.util.LinkedList) Vertices(com.baidu.hugegraph.structure.graph.Vertices) Test(org.junit.Test) BaseApiTest(com.baidu.hugegraph.api.BaseApiTest)

Aggregations

Shard (com.baidu.hugegraph.structure.graph.Shard)13 Test (org.junit.Test)8 BaseApiTest (com.baidu.hugegraph.api.BaseApiTest)6 LinkedList (java.util.LinkedList)6 Edge (com.baidu.hugegraph.structure.graph.Edge)3 Vertex (com.baidu.hugegraph.structure.graph.Vertex)3 ArrayList (java.util.ArrayList)3 BaseClientTest (com.baidu.hugegraph.BaseClientTest)2 InputSplit (com.baidu.hugegraph.computer.core.input.InputSplit)2 Edges (com.baidu.hugegraph.structure.graph.Edges)2 Vertices (com.baidu.hugegraph.structure.graph.Vertices)2 ToolsException (com.baidu.hugegraph.exception.ToolsException)1 BufferedReader (java.io.BufferedReader)1 FileInputStream (java.io.FileInputStream)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 InputStreamReader (java.io.InputStreamReader)1