Examples with InputStruct com.baidu.hugegraph.loader.mapping.InputStruct used on opensource projects

Example 6 with InputStruct

use of com.baidu.hugegraph.loader.mapping.InputStruct in project hugegraph-computer by hugegraph.

the class LoaderFileInputSplitFetcher method fetchVertexInputSplits.

@Override
public List<InputSplit> fetchVertexInputSplits() {
    List<InputSplit> splits = new ArrayList<>();
    for (InputStruct vertexInputStruct : this.vertexInputStructs) {
        FileSource source = (FileSource) vertexInputStruct.input();
        List<String> paths = this.scanPaths(source);
        if (CollectionUtils.isNotEmpty(paths)) {
            for (String path : paths) {
                FileInputSplit split = new FileInputSplit(ElemType.VERTEX, vertexInputStruct, path);
                splits.add(split);
            }
        }
    }
    return splits;
}

Example 7 with InputStruct

use of com.baidu.hugegraph.loader.mapping.InputStruct in project hugegraph-computer by hugegraph.

the class FileElementFetcher method createReader.

private InputReader createReader(FileInputSplit split) {
    InputStruct struct = split.struct();
    FileSource source = (FileSource) struct.input();
    source.path(split.path());
    FileReader reader = (FileReader) InputReader.create(struct.input());
    reader.init(this.context, struct);
    return reader;
}

Example 8 with InputStruct

use of com.baidu.hugegraph.loader.mapping.InputStruct in project incubator-hugegraph-toolchain by apache.

the class LoadTaskService method buildLoadMapping.

private LoadMapping buildLoadMapping(GraphConnection connection, FileMapping fileMapping) {
    FileSource source = this.buildFileSource(fileMapping);
    List<com.baidu.hugegraph.loader.mapping.VertexMapping> vMappings;
    vMappings = this.buildVertexMappings(connection, fileMapping);
    List<com.baidu.hugegraph.loader.mapping.EdgeMapping> eMappings;
    eMappings = this.buildEdgeMappings(connection, fileMapping);
    InputStruct inputStruct = new InputStruct(vMappings, eMappings);
    inputStruct.id("1");
    inputStruct.input(source);
    return new LoadMapping(ImmutableList.of(inputStruct));
}

Example 9 with InputStruct

use of com.baidu.hugegraph.loader.mapping.InputStruct in project incubator-hugegraph-toolchain by apache.

the class HugeGraphLoader method loadStructs.

private void loadStructs(List<InputStruct> structs) {
    // Load input structs one by one
    for (InputStruct struct : structs) {
        if (this.context.stopped()) {
            break;
        }
        if (struct.skip()) {
            continue;
        }
        // Create and init InputReader, fetch next batch lines
        try (InputReader reader = InputReader.create(struct.input())) {
            // Init reader
            reader.init(this.context, struct);
            // Load data from current input mapping
            this.loadStruct(struct, reader);
        } catch (InitException e) {
            throw new LoadException("Failed to init input reader", e);
        }
    }
}