Search in sources :

Example 1 with KvEntryFileReader

use of com.baidu.hugegraph.computer.core.store.KvEntryFileReader in project hugegraph-computer by hugegraph.

the class SorterTest method testSortKvBuffers.

@Test
public void testSortKvBuffers() throws Exception {
    Config config = UnitTestBase.updateWithRequiredOptions(ComputerOptions.HGKV_MAX_FILE_SIZE, "32", ComputerOptions.HGKV_DATABLOCK_SIZE, "16", ComputerOptions.HGKV_MERGE_FILES_NUM, "3", ComputerOptions.TRANSPORT_RECV_FILE_MODE, "false");
    List<Integer> map1 = ImmutableList.of(2, 3, 2, 1, 5, 2, 6, 9, 6, 2);
    List<Integer> map2 = ImmutableList.of(1, 3, 1, 1, 3, 2, 6, 9, 8, 2);
    String path = StoreTestUtil.availablePathById("1");
    // Merge 4 sorted input
    List<RandomAccessInput> inputs = ImmutableList.of(SorterTestUtil.inputFromKvMap(map1), SorterTestUtil.inputFromKvMap(map2), SorterTestUtil.inputFromKvMap(map1), SorterTestUtil.inputFromKvMap(map2));
    Sorter sorter = SorterTestUtil.createSorter(config);
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(IntValue::new, new IntValueSumCombiner());
    sorter.mergeBuffers(inputs, new CombineKvOuterSortFlusher(combiner), path, false);
    // Assert merge result from target hgkvDir
    KvEntryFileReader reader = new HgkvDirReaderImpl(path, false);
    EntryIterator iter = reader.iterator();
    SorterTestUtil.assertKvEntry(iter.next(), 1, 8);
    SorterTestUtil.assertKvEntry(iter.next(), 2, 8);
    SorterTestUtil.assertKvEntry(iter.next(), 3, 4);
    SorterTestUtil.assertKvEntry(iter.next(), 5, 4);
    SorterTestUtil.assertKvEntry(iter.next(), 6, 40);
    SorterTestUtil.assertKvEntry(iter.next(), 8, 4);
    Assert.assertFalse(iter.hasNext());
}
Also used : IntValueSumCombiner(com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner) CombineKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher) Config(com.baidu.hugegraph.computer.core.config.Config) EntryIterator(com.baidu.hugegraph.computer.core.store.EntryIterator) PointerCombiner(com.baidu.hugegraph.computer.core.combiner.PointerCombiner) KvEntryFileReader(com.baidu.hugegraph.computer.core.store.KvEntryFileReader) RandomAccessInput(com.baidu.hugegraph.computer.core.io.RandomAccessInput) HgkvDirReaderImpl(com.baidu.hugegraph.computer.core.store.file.hgkvfile.reader.HgkvDirReaderImpl) Sorter(com.baidu.hugegraph.computer.core.sort.Sorter) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue) Test(org.junit.Test)

Aggregations

IntValueSumCombiner (com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner)1 PointerCombiner (com.baidu.hugegraph.computer.core.combiner.PointerCombiner)1 Config (com.baidu.hugegraph.computer.core.config.Config)1 IntValue (com.baidu.hugegraph.computer.core.graph.value.IntValue)1 RandomAccessInput (com.baidu.hugegraph.computer.core.io.RandomAccessInput)1 Sorter (com.baidu.hugegraph.computer.core.sort.Sorter)1 CombineKvOuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher)1 EntryIterator (com.baidu.hugegraph.computer.core.store.EntryIterator)1 KvEntryFileReader (com.baidu.hugegraph.computer.core.store.KvEntryFileReader)1 HgkvDirReaderImpl (com.baidu.hugegraph.computer.core.store.file.hgkvfile.reader.HgkvDirReaderImpl)1 Test (org.junit.Test)1