Search in sources :

Example 26 with IntValue

use of com.baidu.hugegraph.computer.core.graph.value.IntValue in project hugegraph-computer by hugegraph.

the class MockMasterComputation2 method assertStep1Aggregators.

@Override
protected void assertStep1Aggregators(MasterComputationContext context) {
    Assert.assertEquals(new IntValue(10), context.aggregatedValue(MockMasterComputation.AGGR_CUSTOM_INT));
    Assert.assertEquals(new FloatValue(10.4f), context.aggregatedValue(MockMasterComputation.AGGR_CUSTOM_FLOAT));
    Assert.assertEquals(new FloatValue(6.28f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_UNSTABLE));
    Assert.assertEquals(new IntValue(9), context.aggregatedValue(MockMasterComputation.AGGR_INT_UNSTABLE));
    Assert.assertEquals(new IntValue(10), context.aggregatedValue(MockMasterComputation.AGGR_INT_SUM));
    Assert.assertEquals(new IntValue(8), context.aggregatedValue(MockMasterComputation.AGGR_INT_MAX));
    Assert.assertEquals(new LongValue(10L), context.aggregatedValue(MockMasterComputation.AGGR_LONG_SUM));
    Assert.assertEquals(new LongValue(8L), context.aggregatedValue(MockMasterComputation.AGGR_LONG_MAX));
    Assert.assertEquals(new FloatValue(20.8f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_SUM));
    Assert.assertEquals(new FloatValue(-10.0f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_MIN));
    Assert.assertEquals(new DoubleValue(20.8), context.aggregatedValue(MockMasterComputation.AGGR_DOUBLE_SUM));
    Assert.assertEquals(new DoubleValue(-10.0), context.aggregatedValue(MockMasterComputation.AGGR_DOUBLE_MIN));
}
Also used : DoubleValue(com.baidu.hugegraph.computer.core.graph.value.DoubleValue) LongValue(com.baidu.hugegraph.computer.core.graph.value.LongValue) FloatValue(com.baidu.hugegraph.computer.core.graph.value.FloatValue) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Example 27 with IntValue

use of com.baidu.hugegraph.computer.core.graph.value.IntValue in project hugegraph-computer by hugegraph.

the class MockComputation method afterSuperstep.

@Override
public void afterSuperstep(WorkerContext context) {
    context.aggregateValue(MockMasterComputation.AGGR_CUSTOM_INT, this.aggrCustomInt.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_CUSTOM_FLOAT, this.aggrCustomFloat.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_FLOAT_UNSTABLE, new FloatValue(3.14f));
    context.aggregateValue(MockMasterComputation.AGGR_INT_UNSTABLE, new IntValue(10 - context.superstep()));
    context.aggregateValue(MockMasterComputation.AGGR_INT_SUM, this.aggrIntSum.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_INT_MAX, this.aggrIntMax.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_LONG_SUM, this.aggrLongSum.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_LONG_MAX, this.aggrLongMax.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_FLOAT_SUM, this.aggrFloatSum.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_FLOAT_MIN, this.aggrFloatMin.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_DOUBLE_SUM, this.aggrDoubleSum.aggregatedValue());
    context.aggregateValue(MockMasterComputation.AGGR_DOUBLE_MIN, this.aggrDoubleMin.aggregatedValue());
    this.assertAggregateValueWithError(context);
}
Also used : FloatValue(com.baidu.hugegraph.computer.core.graph.value.FloatValue) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Example 28 with IntValue

use of com.baidu.hugegraph.computer.core.graph.value.IntValue in project hugegraph-computer by hugegraph.

the class MockComputation method assertStep1Aggregators.

protected void assertStep1Aggregators(WorkerContext context) {
    Assert.assertEquals(new IntValue(5), context.aggregatedValue(MockMasterComputation.AGGR_CUSTOM_INT));
    Assert.assertEquals(new FloatValue(5.2f), context.aggregatedValue(MockMasterComputation.AGGR_CUSTOM_FLOAT));
    Assert.assertEquals(new FloatValue(8.8f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_UNSTABLE));
    Assert.assertEquals(new IntValue(888), context.aggregatedValue(MockMasterComputation.AGGR_INT_UNSTABLE));
    Assert.assertEquals(new IntValue(5), context.aggregatedValue(MockMasterComputation.AGGR_INT_SUM));
    Assert.assertEquals(new IntValue(8), context.aggregatedValue(MockMasterComputation.AGGR_INT_MAX));
    Assert.assertEquals(new LongValue(5L), context.aggregatedValue(MockMasterComputation.AGGR_LONG_SUM));
    Assert.assertEquals(new LongValue(8L), context.aggregatedValue(MockMasterComputation.AGGR_LONG_MAX));
    Assert.assertEquals(new FloatValue(10.4f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_SUM));
    Assert.assertEquals(new FloatValue(-10.0f), context.aggregatedValue(MockMasterComputation.AGGR_FLOAT_MIN));
    Assert.assertEquals(new DoubleValue(10.4), context.aggregatedValue(MockMasterComputation.AGGR_DOUBLE_SUM));
    Assert.assertEquals(new DoubleValue(-10.0), context.aggregatedValue(MockMasterComputation.AGGR_DOUBLE_MIN));
}
Also used : DoubleValue(com.baidu.hugegraph.computer.core.graph.value.DoubleValue) LongValue(com.baidu.hugegraph.computer.core.graph.value.LongValue) FloatValue(com.baidu.hugegraph.computer.core.graph.value.FloatValue) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Example 29 with IntValue

use of com.baidu.hugegraph.computer.core.graph.value.IntValue in project hugegraph-computer by hugegraph.

the class SortLargeDataTest method mergeBuffers.

private static void mergeBuffers(Sorter sorter, List<RandomAccessInput> buffers, String output) throws Exception {
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(IntValue::new, new IntValueSumCombiner());
    OuterSortFlusher flusher = new CombineKvOuterSortFlusher(combiner);
    sorter.mergeBuffers(buffers, flusher, output, false);
}
Also used : IntValueSumCombiner(com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner) OuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.OuterSortFlusher) KvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.KvOuterSortFlusher) CombineKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher) CombineKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher) PointerCombiner(com.baidu.hugegraph.computer.core.combiner.PointerCombiner) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Example 30 with IntValue

use of com.baidu.hugegraph.computer.core.graph.value.IntValue in project hugegraph-computer by hugegraph.

the class SorterTest method testMergeSubKvFiles.

private void testMergeSubKvFiles(Config config) throws Exception {
    int flushThreshold = config.get(ComputerOptions.INPUT_MAX_EDGES_IN_ONE_VERTEX);
    List<Integer> kv1 = ImmutableList.of(1, 2, 1, 4, 1);
    List<Integer> kv2 = ImmutableList.of(4, 2, 1, 3, 1);
    List<Integer> kv3 = ImmutableList.of(4, 6, 1, 8, 1);
    List<Integer> kv4 = ImmutableList.of(1, 1, 1, 2, 1);
    List<Integer> kv5 = ImmutableList.of(1, 5, 1, 7, 1);
    List<Integer> kv6 = ImmutableList.of(2, 2, 1, 5, 1);
    List<List<Integer>> data1 = ImmutableList.of(kv1, kv2, kv3);
    List<List<Integer>> data2 = ImmutableList.of(kv4, kv5, kv6);
    List<List<Integer>> data3 = ImmutableList.of(kv4, kv1, kv3);
    List<List<List<Integer>>> datas = ImmutableList.of(data1, data2, data3);
    String input1 = StoreTestUtil.availablePathById(1);
    String input2 = StoreTestUtil.availablePathById(2);
    String input3 = StoreTestUtil.availablePathById(3);
    String output = StoreTestUtil.availablePathById(0);
    List<String> inputs = ImmutableList.of(input1, input2, input3);
    List<String> outputs = ImmutableList.of(output);
    boolean useBufferFile = config.get(ComputerOptions.TRANSPORT_RECV_FILE_MODE);
    for (int i = 0; i < inputs.size(); i++) {
        String input = inputs.get(i);
        List<List<Integer>> data = datas.get(i);
        if (useBufferFile) {
            StoreTestUtil.bufferFileFromSubKvMap(data, input);
        } else {
            StoreTestUtil.hgkvDirFromSubKvMap(config, data, input);
        }
    }
    Sorter sorter = SorterTestUtil.createSorter(config);
    PointerCombiner combiner = SorterTestUtil.createPointerCombiner(IntValue::new, new IntValueSumCombiner());
    OuterSortFlusher flusher = new CombineSubKvOuterSortFlusher(combiner, flushThreshold);
    flusher.sources(inputs.size());
    sorter.mergeInputs(inputs, flusher, outputs, true);
    /* Assert result
         * key 1 subKv 1 2 2 4
         * key 1 subKv 4 2 5 1
         * key 1 subKv 7 1
         * key 2 subKv 2 1 5 1
         * key 4 subKv 2 1 3 1
         * key 4 subKv 6 2 8 2
         */
    try (CIter<KvEntry> kvIter = sorter.iterator(outputs, true)) {
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 1, 1, 2, 2, 4);
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 1, 4, 2, 5, 1);
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 1, 7, 1);
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 2, 2, 1, 5, 1);
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 4, 2, 1, 3, 1);
        SorterTestUtil.assertSubKvByKv(kvIter.next(), 4, 6, 2, 8, 2);
    }
    FileUtil.deleteFilesQuietly(inputs);
    FileUtil.deleteFilesQuietly(outputs);
}
Also used : IntValueSumCombiner(com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner) CombineSubKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvOuterSortFlusher) OuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.OuterSortFlusher) CombineSubKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineSubKvOuterSortFlusher) CombineKvOuterSortFlusher(com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher) KvEntry(com.baidu.hugegraph.computer.core.store.entry.KvEntry) PointerCombiner(com.baidu.hugegraph.computer.core.combiner.PointerCombiner) Sorter(com.baidu.hugegraph.computer.core.sort.Sorter) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) IntValue(com.baidu.hugegraph.computer.core.graph.value.IntValue)

Aggregations

IntValue (com.baidu.hugegraph.computer.core.graph.value.IntValue)34 FloatValue (com.baidu.hugegraph.computer.core.graph.value.FloatValue)15 DoubleValue (com.baidu.hugegraph.computer.core.graph.value.DoubleValue)14 LongValue (com.baidu.hugegraph.computer.core.graph.value.LongValue)13 Test (org.junit.Test)12 PointerCombiner (com.baidu.hugegraph.computer.core.combiner.PointerCombiner)10 IntValueSumCombiner (com.baidu.hugegraph.computer.core.combiner.IntValueSumCombiner)9 Sorter (com.baidu.hugegraph.computer.core.sort.Sorter)7 CombineKvOuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvOuterSortFlusher)6 GraphFactory (com.baidu.hugegraph.computer.core.graph.GraphFactory)4 BooleanValue (com.baidu.hugegraph.computer.core.graph.value.BooleanValue)4 Vertex (com.baidu.hugegraph.computer.core.graph.vertex.Vertex)4 BytesInput (com.baidu.hugegraph.computer.core.io.BytesInput)4 BytesOutput (com.baidu.hugegraph.computer.core.io.BytesOutput)4 CombineKvInnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.CombineKvInnerSortFlusher)4 OuterSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.OuterSortFlusher)4 Config (com.baidu.hugegraph.computer.core.config.Config)3 Properties (com.baidu.hugegraph.computer.core.graph.properties.Properties)3 InnerSortFlusher (com.baidu.hugegraph.computer.core.sort.flusher.InnerSortFlusher)3 KvEntry (com.baidu.hugegraph.computer.core.store.entry.KvEntry)3