Examples with EntryIterator com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.EntryIterator used on opensource projects

Example 1 with EntryIterator

use of com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.EntryIterator in project hugegraph-computer by hugegraph.

the class SorterImpl method sortBuffers.

private void sortBuffers(List<EntryIterator> entries, OuterSortFlusher flusher, String output) throws IOException {
    InputsSorter sorter = new InputsSorterImpl();
    try (HgkvDirBuilder builder = new HgkvDirBuilderImpl(this.config, output)) {
        EntryIterator result = sorter.sort(entries);
        flusher.flush(result, builder);
    }
}

Example 2 with EntryIterator

use of com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.EntryIterator in project hugegraph-computer by hugegraph.

the class SorterImpl method sortBuffer.

@Override
public void sortBuffer(RandomAccessInput input, InnerSortFlusher flusher, boolean withSubKv) throws Exception {
    try (EntryIterator entries = new KvEntriesInput(input, withSubKv)) {
        InputSorter sorter = new JavaInputSorter();
        flusher.flush(sorter.sort(entries));
    }
}

Example 3 with EntryIterator

use of com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.EntryIterator in project hugegraph-computer by hugegraph.

the class HgkvDirMergerImpl method mergeInputs.

private void mergeInputs(List<String> inputs, InputToEntries inputToEntries, OuterSortFlusher flusher, String output) throws Exception {
    /*
         * File value format is different, upper layer is required to
         * provide the file reading mode
         */
    List<EntryIterator> entries = inputs.stream().map(inputToEntries::inputToEntries).collect(Collectors.toList());
    InputsSorter sorter = new InputsSorterImpl();
    // Merge inputs and write to output
    try (EntryIterator sortedKv = sorter.sort(entries);
        HgkvDirBuilder builder = new HgkvDirBuilderImpl(this.config, output)) {
        flusher.flush(sortedKv, builder);
    }
}

Example 4 with EntryIterator

use of com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.EntryIterator in project hugegraph-computer by hugegraph.

the class SorterImpl method iterator.

@Override
public PeekableIterator<KvEntry> iterator(List<String> inputs, boolean withSubKv) throws IOException {
    InputsSorterImpl sorter = new InputsSorterImpl();
    List<EntryIterator> entries = new ArrayList<>();
    for (String input : inputs) {
        HgkvDirReader reader = new HgkvDirReaderImpl(input, false, withSubKv);
        entries.add(reader.iterator());
    }
    return PeekableIteratorAdaptor.of(sorter.sort(entries));
}