use of com.baidu.hugegraph.computer.core.sort.sorter.InputsSorterImpl in project hugegraph-computer by hugegraph.
the class SorterImpl method sortBuffers.
private void sortBuffers(List<EntryIterator> entries, OuterSortFlusher flusher, String output) throws IOException {
InputsSorter sorter = new InputsSorterImpl();
try (HgkvDirBuilder builder = new HgkvDirBuilderImpl(this.config, output)) {
EntryIterator result = sorter.sort(entries);
flusher.flush(result, builder);
}
}
use of com.baidu.hugegraph.computer.core.sort.sorter.InputsSorterImpl in project hugegraph-computer by hugegraph.
the class FileMergerImpl method mergeInputs.
private void mergeInputs(List<String> inputs, Function<String, EntryIterator> inputToIter, OuterSortFlusher flusher, String output, Function<String, KvEntryFileWriter> fileToWriter) throws Exception {
/*
* File value format is different, upper layer is required to
* provide the file reading mode
*/
List<EntryIterator> entries = inputs.stream().map(inputToIter).collect(Collectors.toList());
InputsSorter sorter = new InputsSorterImpl();
// Merge inputs and write to output
try (EntryIterator sortedKv = sorter.sort(entries);
KvEntryFileWriter builder = fileToWriter.apply(output)) {
flusher.flush(sortedKv, builder);
}
}
use of com.baidu.hugegraph.computer.core.sort.sorter.InputsSorterImpl in project hugegraph-computer by hugegraph.
the class DefaultSorter method iterator.
public PeekableIterator<KvEntry> iterator(List<String> inputs, Function<String, EntryIterator> fileToEntries) throws IOException {
List<EntryIterator> entries = inputs.stream().map(fileToEntries).collect(Collectors.toList());
InputsSorterImpl sorter = new InputsSorterImpl();
return PeekableIteratorAdaptor.of(sorter.sort(entries));
}
use of com.baidu.hugegraph.computer.core.sort.sorter.InputsSorterImpl in project hugegraph-computer by hugegraph.
the class DefaultSorter method mergeBuffers.
public void mergeBuffers(List<EntryIterator> entries, KvEntryFileWriter writer, OuterSortFlusher flusher) throws IOException {
InputsSorter sorter = new InputsSorterImpl();
EntryIterator result = sorter.sort(entries);
flusher.flush(result, writer);
}
use of com.baidu.hugegraph.computer.core.sort.sorter.InputsSorterImpl in project hugegraph-computer by hugegraph.
the class HgkvDirMergerImpl method mergeInputs.
private void mergeInputs(List<String> inputs, InputToEntries inputToEntries, OuterSortFlusher flusher, String output) throws Exception {
/*
* File value format is different, upper layer is required to
* provide the file reading mode
*/
List<EntryIterator> entries = inputs.stream().map(inputToEntries::inputToEntries).collect(Collectors.toList());
InputsSorter sorter = new InputsSorterImpl();
// Merge inputs and write to output
try (EntryIterator sortedKv = sorter.sort(entries);
HgkvDirBuilder builder = new HgkvDirBuilderImpl(this.config, output)) {
flusher.flush(sortedKv, builder);
}
}
Aggregations