Search in sources :

Example 1 with BytesInput

use of com.baidu.hugegraph.computer.core.io.BytesInput in project hugegraph-computer by hugegraph.

the class KvEntryWriterImpl method sortAndWriteSubKvs.

private void sortAndWriteSubKvs() throws IOException {
    BytesInput input = EntriesUtil.inputFromOutput(this.subKvBuffer);
    InputSorter sorter = new JavaInputSorter();
    Iterator<KvEntry> subKvs = sorter.sort(new KvEntriesInput(input));
    while (subKvs.hasNext()) {
        KvEntry subKv = subKvs.next();
        subKv.key().write(this.output);
        subKv.value().write(this.output);
    }
}
Also used : JavaInputSorter(com.baidu.hugegraph.computer.core.sort.sorter.JavaInputSorter) InputSorter(com.baidu.hugegraph.computer.core.sort.sorter.InputSorter) JavaInputSorter(com.baidu.hugegraph.computer.core.sort.sorter.JavaInputSorter) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) KvEntriesInput(com.baidu.hugegraph.computer.core.store.hgkvfile.buffer.KvEntriesInput)

Example 2 with BytesInput

use of com.baidu.hugegraph.computer.core.io.BytesInput in project hugegraph-computer by hugegraph.

the class StoreTestUtil method bufferFileFromKvMap.

public static void bufferFileFromKvMap(List<Integer> map, String path) throws IOException {
    RandomAccessOutput output = IOFactory.createFileOutput(new File(path));
    BytesInput buffer = SorterTestUtil.inputFromKvMap(map);
    output.write(buffer.readBytes((int) buffer.available()));
    output.close();
}
Also used : RandomAccessOutput(com.baidu.hugegraph.computer.core.io.RandomAccessOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) File(java.io.File)

Example 3 with BytesInput

use of com.baidu.hugegraph.computer.core.io.BytesInput in project hugegraph-computer by hugegraph.

the class StoreTestUtil method kvEntriesFromMap.

public static List<KvEntry> kvEntriesFromMap(List<Integer> map) throws IOException {
    BytesOutput data = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    Iterator<Integer> iterator = map.iterator();
    while (iterator.hasNext()) {
        // Write key length
        writeData(data, iterator.next());
        // Write value length
        writeData(data, iterator.next());
    }
    BytesInput input = IOFactory.createBytesInput(data.buffer(), (int) data.position());
    KvEntriesInput iter = new KvEntriesInput(input);
    List<KvEntry> entries = new ArrayList<>();
    while (iter.hasNext()) {
        entries.add(iter.next());
    }
    iter.close();
    return entries;
}
Also used : BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) KvEntry(com.baidu.hugegraph.computer.core.store.entry.KvEntry) ArrayList(java.util.ArrayList) KvEntriesInput(com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput)

Example 4 with BytesInput

use of com.baidu.hugegraph.computer.core.io.BytesInput in project hugegraph-computer by hugegraph.

the class StoreTestUtil method idFromPointer.

public static Id idFromPointer(Pointer pointer) throws IOException {
    BytesInput input = IOFactory.createBytesInput(pointer.bytes());
    Id id = new BytesId();
    id.read(input);
    return id;
}
Also used : BytesId(com.baidu.hugegraph.computer.core.graph.id.BytesId) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) Id(com.baidu.hugegraph.computer.core.graph.id.Id) BytesId(com.baidu.hugegraph.computer.core.graph.id.BytesId)

Example 5 with BytesInput

use of com.baidu.hugegraph.computer.core.io.BytesInput in project hugegraph-computer by hugegraph.

the class StoreTestUtil method bufferFileFromSubKvMap.

public static void bufferFileFromSubKvMap(List<List<Integer>> map, String path) throws IOException {
    RandomAccessOutput output = IOFactory.createFileOutput(new File(path));
    BytesInput buffer = SorterTestUtil.inputFromSubKvMap(map);
    output.write(buffer.readBytes((int) buffer.available()));
    output.close();
}
Also used : RandomAccessOutput(com.baidu.hugegraph.computer.core.io.RandomAccessOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) File(java.io.File)

Aggregations

BytesInput (com.baidu.hugegraph.computer.core.io.BytesInput)32 BytesOutput (com.baidu.hugegraph.computer.core.io.BytesOutput)16 Test (org.junit.Test)16 KvEntriesInput (com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput)10 KvEntry (com.baidu.hugegraph.computer.core.store.entry.KvEntry)8 Sorter (com.baidu.hugegraph.computer.core.sort.Sorter)7 Config (com.baidu.hugegraph.computer.core.config.Config)5 Id (com.baidu.hugegraph.computer.core.graph.id.Id)5 PointerCombiner (com.baidu.hugegraph.computer.core.combiner.PointerCombiner)4 ComputerException (com.baidu.hugegraph.computer.core.common.exception.ComputerException)4 BytesId (com.baidu.hugegraph.computer.core.graph.id.BytesId)4 IntValue (com.baidu.hugegraph.computer.core.graph.value.IntValue)4 CombineKvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvInnerSortFlusher)4 IOException (java.io.IOException)4 ArrayList (java.util.ArrayList)4 IntValueSumCombiner (com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner)3 RandomAccessInput (com.baidu.hugegraph.computer.core.io.RandomAccessInput)3 EntryOutput (com.baidu.hugegraph.computer.core.store.entry.EntryOutput)3 EntryOutputImpl (com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl)3 Pointer (com.baidu.hugegraph.computer.core.store.entry.Pointer)3