Search in sources :

Example 11 with PointerCombiner

use of com.baidu.hugegraph.computer.core.combiner.PointerCombiner in project hugegraph-computer by hugegraph.

the class SorterTest method sortedSubKvBuffer.

private BytesInput sortedSubKvBuffer(Config config) throws Exception {
    List<Integer> kv1 = ImmutableList.of(3, 2, 1, 4, 1);
    List<Integer> kv2 = ImmutableList.of(1, 3, 1, 5, 1);
    List<Integer> kv3 = ImmutableList.of(2, 8, 1, 9, 1);
    List<Integer> kv4 = ImmutableList.of(3, 2, 1, 3, 1);
    List<Integer> kv5 = ImmutableList.of(2, 5, 1, 8, 1);
    List<List<Integer>> data = ImmutableList.of(kv1, kv2, kv3, kv4, kv5);
    BytesInput input = SorterTestUtil.inputFromSubKvMap(data);
    BytesOutput output = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(IntValue::new, new IntValueSumCombiner());
    int flushThreshold = config.get(ComputerOptions.INPUT_MAX_EDGES_IN_ONE_VERTEX);
    InnerSortFlusher flusher = new CombineSubKvInnerSortFlusher(output, combiner, flushThreshold);
    Sorter sorter = SorterTestUtil.createSorter(config);
    sorter.sortBuffer(input, flusher, true);
    return EntriesUtil.inputFromOutput(output);
}
Also used : IntValueSumCombiner(com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner) CombineSubKvInnerSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvInnerSortFlusher) CombineSubKvInnerSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvInnerSortFlusher) CombineKvInnerSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineKvInnerSortFlusher) InnerSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.InnerSortFlusher) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) PointerCombiner(com.baidu.hugegraph.computer.core.combiner.PointerCombiner) Sorter(com.baidu.hugegraph.computer.core.sort.Sorter) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Aggregations

PointerCombiner (com.baidu.hugegraph.computer.core.combiner.PointerCombiner)11 IntValue (com.baidu.hugegraph.computer.core.graph.value.IntValue)10 IntValueSumCombiner (com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner)9 Sorter (com.baidu.hugegraph.computer.core.sort.Sorter)7 CombineKvOuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher)6 CombineKvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvInnerSortFlusher)5 BytesInput (com.baidu.hugegraph.computer.core.io.BytesInput)4 BytesOutput (com.baidu.hugegraph.computer.core.io.BytesOutput)4 InnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.InnerSortFlusher)4 OuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.OuterSortFlusher)4 Test (org.junit.Test)4 Config (com.baidu.hugegraph.computer.core.config.Config)3 KvEntry (com.baidu.hugegraph.computer.core.store.entry.KvEntry)3 RandomAccessInput (com.baidu.hugegraph.computer.core.io.RandomAccessInput)2 CombineSubKvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvInnerSortFlusher)2 CombineSubKvOuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvOuterSortFlusher)2 KvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.KvInnerSortFlusher)2 KvOuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.KvOuterSortFlusher)2 KvEntriesInput (com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput)2 ImmutableList (com.google.common.collect.ImmutableList)2