Examples with FileSource com.baidu.hugegraph.loader.source.file.FileSource used on opensource projects

Example 6 with FileSource

use of com.baidu.hugegraph.loader.source.file.FileSource in project hugegraph-computer by hugegraph.

the class LoaderFileInputSplitFetcher method fetchEdgeInputSplits.

@Override
public List<InputSplit> fetchEdgeInputSplits() {
    List<InputSplit> splits = new ArrayList<>();
    for (InputStruct edgeInputStruct : this.edgeInputStructs) {
        FileSource source = (FileSource) edgeInputStruct.input();
        List<String> paths = this.scanPaths(source);
        if (CollectionUtils.isNotEmpty(paths)) {
            for (String path : paths) {
                FileInputSplit split = new FileInputSplit(ElemType.EDGE, edgeInputStruct, path);
                splits.add(split);
            }
        }
    }
    return splits;
}

Example 7 with FileSource

use of com.baidu.hugegraph.loader.source.file.FileSource in project hugegraph-computer by hugegraph.

the class LoaderFileInputSplitFetcher method fetchVertexInputSplits.

@Override
public List<InputSplit> fetchVertexInputSplits() {
    List<InputSplit> splits = new ArrayList<>();
    for (InputStruct vertexInputStruct : this.vertexInputStructs) {
        FileSource source = (FileSource) vertexInputStruct.input();
        List<String> paths = this.scanPaths(source);
        if (CollectionUtils.isNotEmpty(paths)) {
            for (String path : paths) {
                FileInputSplit split = new FileInputSplit(ElemType.VERTEX, vertexInputStruct, path);
                splits.add(split);
            }
        }
    }
    return splits;
}

Example 8 with FileSource

use of com.baidu.hugegraph.loader.source.file.FileSource in project hugegraph-computer by hugegraph.

the class FileElementFetcher method createReader.

private InputReader createReader(FileInputSplit split) {
    InputStruct struct = split.struct();
    FileSource source = (FileSource) struct.input();
    source.path(split.path());
    FileReader reader = (FileReader) InputReader.create(struct.input());
    reader.init(this.context, struct);
    return reader;
}

Example 9 with FileSource

use of com.baidu.hugegraph.loader.source.file.FileSource in project incubator-hugegraph-toolchain by apache.

the class LoadTaskService method buildLoadMapping.

private LoadMapping buildLoadMapping(GraphConnection connection, FileMapping fileMapping) {
    FileSource source = this.buildFileSource(fileMapping);
    List<com.baidu.hugegraph.loader.mapping.VertexMapping> vMappings;
    vMappings = this.buildVertexMappings(connection, fileMapping);
    List<com.baidu.hugegraph.loader.mapping.EdgeMapping> eMappings;
    eMappings = this.buildEdgeMappings(connection, fileMapping);
    InputStruct inputStruct = new InputStruct(vMappings, eMappings);
    inputStruct.id("1");
    inputStruct.input(source);
    return new LoadMapping(ImmutableList.of(inputStruct));
}