Search in sources :

Example 1 with EntryOutputImpl

use of com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl in project hugegraph-computer by hugegraph.

the class EntryOutputTest method inputFromEntries.

private static BytesInput inputFromEntries(List<Integer> entries, boolean needSort) throws IOException {
    /*
         * All integer data will convert to Id type, so upper layer also
         * needs to use the Id type to make a judgment
         */
    List<Id> data = intListToLongIds(entries);
    BytesOutput output = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(output, needSort);
    int index = 0;
    KvEntryWriter entry1 = entryOutput.writeEntry(data.get(index++));
    entry1.writeSubKv(data.get(index++), data.get(index++));
    entry1.writeSubKv(data.get(index++), data.get(index++));
    entry1.writeSubKv(data.get(index++), data.get(index++));
    entry1.writeFinish();
    KvEntryWriter entry2 = entryOutput.writeEntry(data.get(index++));
    entry2.writeSubKv(data.get(index++), data.get(index++));
    entry2.writeSubKv(data.get(index++), data.get(index));
    entry2.writeFinish();
    return EntriesUtil.inputFromOutput(output);
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) KvEntryWriter(com.baidu.hugegraph.computer.core.store.entry.KvEntryWriter) Id(com.baidu.hugegraph.computer.core.graph.id.Id) BytesId(com.baidu.hugegraph.computer.core.graph.id.BytesId)

Example 2 with EntryOutputImpl

use of com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl in project hugegraph-computer by hugegraph.

the class EntryOutputTest method testWriteKvEntry.

@Test
public void testWriteKvEntry() throws Exception {
    List<Integer> entries = ImmutableList.of(1, 5, 6, 6, 2, 1, 4, 8);
    List<Id> data = intListToLongIds(entries);
    BytesOutput output = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(output);
    for (int i = 0; i < data.size(); ) {
        Id id = data.get(i++);
        Writable value = data.get(i++);
        entryOutput.writeEntry(id, value);
    }
    // Assert result
    BytesInput input = EntriesUtil.inputFromOutput(output);
    EntryIterator iter = new KvEntriesInput(input);
    SorterTestUtil.assertKvEntry(iter.next(), BytesId.of(1), BytesId.of(5));
    SorterTestUtil.assertKvEntry(iter.next(), BytesId.of(6), BytesId.of(6));
    SorterTestUtil.assertKvEntry(iter.next(), BytesId.of(2), BytesId.of(1));
    SorterTestUtil.assertKvEntry(iter.next(), BytesId.of(4), BytesId.of(8));
    iter.close();
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) Writable(com.baidu.hugegraph.computer.core.io.Writable) KvEntriesInput(com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput) Id(com.baidu.hugegraph.computer.core.graph.id.Id) BytesId(com.baidu.hugegraph.computer.core.graph.id.BytesId) Test(org.junit.Test)

Example 3 with EntryOutputImpl

use of com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl in project hugegraph-computer by hugegraph.

the class IOFactory method createGraphOutput.

public static GraphOutput createGraphOutput(ComputerContext context, OutputFormat format, RandomAccessOutput out) {
    switch(format) {
        case BIN:
            EntryOutput entryOutput = new EntryOutputImpl(out);
            return new StreamGraphOutput(context, entryOutput);
        case CSV:
            StructRandomAccessOutput srao;
            srao = new StructRandomAccessOutput(out);
            return new CsvStructGraphOutput(context, srao);
        case JSON:
            srao = new StructRandomAccessOutput(out);
            return new JsonStructGraphOutput(context, srao);
        default:
            throw new ComputerException("Can't create GraphOutput for %s", format);
    }
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) ComputerException(com.baidu.hugegraph.computer.core.common.exception.ComputerException)

Example 4 with EntryOutputImpl

use of com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl in project hugegraph-computer by hugegraph.

the class ComputeManagerTest method writeEdges.

private static byte[] writeEdges(Vertex vertex, EdgeFrequency freq) throws IOException {
    BytesOutput bytesOutput = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(bytesOutput);
    GraphComputeOutput output = new StreamGraphOutput(context(), entryOutput);
    Whitebox.setInternalState(output, "frequency", freq);
    output.writeEdges(vertex);
    return bytesOutput.toByteArray();
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) StreamGraphOutput(com.baidu.hugegraph.computer.core.io.StreamGraphOutput) GraphComputeOutput(com.baidu.hugegraph.computer.core.io.GraphComputeOutput)

Example 5 with EntryOutputImpl

use of com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl in project hugegraph-computer by hugegraph.

the class EdgesInputTest method writeEdges.

private static byte[] writeEdges(Vertex vertex, EdgeFrequency freq) throws IOException {
    BytesOutput bytesOutput = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(bytesOutput);
    GraphComputeOutput output = new StreamGraphOutput(context(), entryOutput);
    Whitebox.setInternalState(output, "frequency", freq);
    output.writeEdges(vertex);
    return bytesOutput.toByteArray();
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) StreamGraphOutput(com.baidu.hugegraph.computer.core.io.StreamGraphOutput) GraphComputeOutput(com.baidu.hugegraph.computer.core.io.GraphComputeOutput)

Aggregations

EntryOutput (com.baidu.hugegraph.computer.core.store.entry.EntryOutput)12 EntryOutputImpl (com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl)12 BytesOutput (com.baidu.hugegraph.computer.core.io.BytesOutput)11 GraphComputeOutput (com.baidu.hugegraph.computer.core.io.GraphComputeOutput)4 StreamGraphOutput (com.baidu.hugegraph.computer.core.io.StreamGraphOutput)4 KvEntryWriter (com.baidu.hugegraph.computer.core.store.entry.KvEntryWriter)4 BytesId (com.baidu.hugegraph.computer.core.graph.id.BytesId)3 Id (com.baidu.hugegraph.computer.core.graph.id.Id)3 BytesInput (com.baidu.hugegraph.computer.core.io.BytesInput)3 Test (org.junit.Test)3 KvEntry (com.baidu.hugegraph.computer.core.store.entry.KvEntry)2 ComputerException (com.baidu.hugegraph.computer.core.common.exception.ComputerException)1 Edge (com.baidu.hugegraph.computer.core.graph.edge.Edge)1 Writable (com.baidu.hugegraph.computer.core.io.Writable)1 KvEntriesInput (com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput)1 SubKvEntriesInput (com.baidu.hugegraph.computer.core.store.buffer.SubKvEntriesInput)1