Search in sources :

Example 21 with BytesOutput

use of com.baidu.hugegraph.computer.core.io.BytesOutput in project hugegraph-computer by hugegraph.

the class PointerCombinerTest method testVertexPropertiesCombiner.

@Test
public void testVertexPropertiesCombiner() throws IOException {
    Config config = UnitTestBase.updateWithRequiredOptions(ComputerOptions.WORKER_COMBINER_CLASS, DoubleValueSumCombiner.class.getName(), ComputerOptions.WORKER_VERTEX_PROPERTIES_COMBINER_CLASS, MergeOldPropertiesCombiner.class.getName());
    Combiner<Properties> valueCombiner = config.createObject(ComputerOptions.WORKER_VERTEX_PROPERTIES_COMBINER_CLASS);
    GraphFactory graphFactory = graphFactory();
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(graphFactory::createProperties, valueCombiner);
    try (BytesOutput bytesOutput1 = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
        BytesOutput bytesOutput2 = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE)) {
        Properties value1 = graphFactory.createProperties();
        value1.put("p1", new LongValue(1L));
        Properties value2 = graphFactory.createProperties();
        value2.put("p2", new LongValue(2L));
        value1.write(bytesOutput1);
        value2.write(bytesOutput2);
        Pointer pointer1 = new InlinePointer(bytesOutput1.buffer(), bytesOutput1.position());
        Pointer pointer2 = new InlinePointer(bytesOutput2.buffer(), bytesOutput2.position());
        Pointer pointer = combiner.combine(pointer1, pointer2);
        BytesInput input = IOFactory.createBytesInput(pointer.bytes());
        Properties combinedValue = graphFactory.createProperties();
        combinedValue.read(input);
        Map<String, Value> map = combinedValue.get();
        Assert.assertEquals(2, map.size());
        Assert.assertEquals(new LongValue(1L), map.get("p1"));
        Assert.assertEquals(new LongValue(2L), map.get("p2"));
    }
}
Also used : GraphFactory(com.baidu.hugegraph.computer.core.graph.GraphFactory) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) Config(com.baidu.hugegraph.computer.core.config.Config) InlinePointer(com.baidu.hugegraph.computer.core.store.entry.InlinePointer) Pointer(com.baidu.hugegraph.computer.core.store.entry.Pointer) InlinePointer(com.baidu.hugegraph.computer.core.store.entry.InlinePointer) Properties(com.baidu.hugegraph.computer.core.graph.properties.Properties) LongValue(com.baidu.hugegraph.computer.core.graph.value.LongValue) DoubleValue(com.baidu.hugegraph.computer.core.graph.value.DoubleValue) Value(com.baidu.hugegraph.computer.core.graph.value.Value) LongValue(com.baidu.hugegraph.computer.core.graph.value.LongValue) Test(org.junit.Test)

Example 22 with BytesOutput

use of com.baidu.hugegraph.computer.core.io.BytesOutput in project hugegraph-computer by hugegraph.

the class PointerCombinerTest method testCombineEdgePropertiesFail.

@Test
public void testCombineEdgePropertiesFail() throws IOException {
    Config config = UnitTestBase.updateWithRequiredOptions(ComputerOptions.WORKER_COMBINER_CLASS, DoubleValueSumCombiner.class.getName(), ComputerOptions.WORKER_EDGE_PROPERTIES_COMBINER_CLASS, MergeOldPropertiesCombiner.class.getName());
    Combiner<Properties> valueCombiner = config.createObject(ComputerOptions.WORKER_EDGE_PROPERTIES_COMBINER_CLASS);
    GraphFactory graphFactory = graphFactory();
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(graphFactory::createProperties, valueCombiner);
    try (BytesOutput bytesOutput1 = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
        BytesOutput bytesOutput2 = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE)) {
        Properties value1 = graphFactory.createProperties();
        value1.put("p1", new LongValue(1L));
        Properties value2 = graphFactory.createProperties();
        value2.put("p2", new LongValue(2L));
        // Only write count.
        bytesOutput1.writeInt(1);
        value2.write(bytesOutput2);
        Pointer pointer1 = new InlinePointer(bytesOutput1.buffer(), bytesOutput1.position());
        Pointer pointer2 = new InlinePointer(bytesOutput2.buffer(), bytesOutput2.position());
        Assert.assertThrows(ComputerException.class, () -> {
            combiner.combine(pointer1, pointer2);
        }, e -> {
            Assert.assertContains("Failed to combine pointer", e.getMessage());
        });
    }
}
Also used : GraphFactory(com.baidu.hugegraph.computer.core.graph.GraphFactory) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) Config(com.baidu.hugegraph.computer.core.config.Config) InlinePointer(com.baidu.hugegraph.computer.core.store.entry.InlinePointer) LongValue(com.baidu.hugegraph.computer.core.graph.value.LongValue) Pointer(com.baidu.hugegraph.computer.core.store.entry.Pointer) InlinePointer(com.baidu.hugegraph.computer.core.store.entry.InlinePointer) Properties(com.baidu.hugegraph.computer.core.graph.properties.Properties) Test(org.junit.Test)

Example 23 with BytesOutput

use of com.baidu.hugegraph.computer.core.io.BytesOutput in project hugegraph-computer by hugegraph.

the class ComputeManagerTest method writeVertex.

private static byte[] writeVertex(Vertex vertex) throws IOException {
    BytesOutput bytesOutput = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(bytesOutput);
    GraphComputeOutput output = new StreamGraphOutput(context(), entryOutput);
    output.writeVertex(vertex);
    return bytesOutput.toByteArray();
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) StreamGraphOutput(com.baidu.hugegraph.computer.core.io.StreamGraphOutput) GraphComputeOutput(com.baidu.hugegraph.computer.core.io.GraphComputeOutput)

Example 24 with BytesOutput

use of com.baidu.hugegraph.computer.core.io.BytesOutput in project hugegraph-computer by hugegraph.

the class UnitTestBase method assertIdEqualAfterWriteAndRead.

public static void assertIdEqualAfterWriteAndRead(Id oldId) throws IOException {
    byte[] bytes;
    try (BytesOutput bao = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE)) {
        oldId.write(bao);
        bytes = bao.toByteArray();
    }
    Id newId = IdFactory.createId(oldId.idType());
    try (BytesInput bai = IOFactory.createBytesInput(bytes)) {
        newId.read(bai);
        Assert.assertEquals(oldId, newId);
    }
}
Also used : BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput) BytesInput(com.baidu.hugegraph.computer.core.io.BytesInput) Id(com.baidu.hugegraph.computer.core.graph.id.Id)

Example 25 with BytesOutput

use of com.baidu.hugegraph.computer.core.io.BytesOutput in project hugegraph-computer by hugegraph.

the class ReceiverUtil method writeMessage.

public static byte[] writeMessage(Id id, Writable message) throws IOException {
    BytesOutput bytesOutput = IOFactory.createBytesOutput(Constants.SMALL_BUF_SIZE);
    EntryOutput entryOutput = new EntryOutputImpl(bytesOutput);
    entryOutput.writeEntry(out -> {
        id.write(out);
    }, message);
    return bytesOutput.toByteArray();
}
Also used : EntryOutput(com.baidu.hugegraph.computer.core.store.entry.EntryOutput) EntryOutputImpl(com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl) BytesOutput(com.baidu.hugegraph.computer.core.io.BytesOutput)

Aggregations

BytesOutput (com.baidu.hugegraph.computer.core.io.BytesOutput)36 BytesInput (com.baidu.hugegraph.computer.core.io.BytesInput)16 Test (org.junit.Test)16 EntryOutput (com.baidu.hugegraph.computer.core.store.entry.EntryOutput)11 EntryOutputImpl (com.baidu.hugegraph.computer.core.store.entry.EntryOutputImpl)11 Sorter (com.baidu.hugegraph.computer.core.sort.Sorter)7 CombineKvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvInnerSortFlusher)7 InnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.InnerSortFlusher)5 KvEntriesInput (com.baidu.hugegraph.computer.core.store.buffer.KvEntriesInput)5 KvEntry (com.baidu.hugegraph.computer.core.store.entry.KvEntry)5 ArrayList (java.util.ArrayList)5 PointerCombiner (com.baidu.hugegraph.computer.core.combiner.PointerCombiner)4 Config (com.baidu.hugegraph.computer.core.config.Config)4 Id (com.baidu.hugegraph.computer.core.graph.id.Id)4 IntValue (com.baidu.hugegraph.computer.core.graph.value.IntValue)4 GraphComputeOutput (com.baidu.hugegraph.computer.core.io.GraphComputeOutput)4 RandomAccessInput (com.baidu.hugegraph.computer.core.io.RandomAccessInput)4 StreamGraphOutput (com.baidu.hugegraph.computer.core.io.StreamGraphOutput)4 KvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.KvInnerSortFlusher)4 KvEntryWriter (com.baidu.hugegraph.computer.core.store.entry.KvEntryWriter)4