Search in sources :

Example 1 with BiConsumerEx

use of com.hazelcast.function.BiConsumerEx in project hazelcast by hazelcast.

the class AggregateOperation3Test method when_withCombiningAccumulateFn_then_accumulateFnCombines.

@Test
public void when_withCombiningAccumulateFn_then_accumulateFnCombines() {
    // Given
    AggregateOperation3<Object, Object, Object, LongAccumulator, Long> aggrOp = AggregateOperation.withCreate(LongAccumulator::new).andAccumulate0((acc, item) -> acc.addAllowingOverflow(1)).andAccumulate1((acc, item) -> acc.addAllowingOverflow(10)).andAccumulate2((acc, item) -> acc.addAllowingOverflow(100)).andCombine(LongAccumulator::addAllowingOverflow).andExportFinish(LongAccumulator::get);
    AggregateOperation1<LongAccumulator, LongAccumulator, Long> combiningAggrOp = aggrOp.withCombiningAccumulateFn(wholeItem());
    BiConsumerEx<? super LongAccumulator, ? super LongAccumulator> accFn = combiningAggrOp.accumulateFn();
    LongAccumulator partialAcc1 = combiningAggrOp.createFn().get();
    LongAccumulator partialAcc2 = combiningAggrOp.createFn().get();
    LongAccumulator combinedAcc = combiningAggrOp.createFn().get();
    // When
    partialAcc1.set(2);
    partialAcc2.set(3);
    accFn.accept(combinedAcc, partialAcc1);
    accFn.accept(combinedAcc, partialAcc2);
    // Then
    assertEquals(5, combinedAcc.get());
}
Also used : LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) FunctionEx(com.hazelcast.function.FunctionEx) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) RunWith(org.junit.runner.RunWith) BiConsumerEx(com.hazelcast.function.BiConsumerEx) Tag.tag2(com.hazelcast.jet.datamodel.Tag.tag2) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) SupplierEx(com.hazelcast.function.SupplierEx) Tag.tag1(com.hazelcast.jet.datamodel.Tag.tag1) Tag.tag0(com.hazelcast.jet.datamodel.Tag.tag0) Assert.assertSame(org.junit.Assert.assertSame) Tag.tag(com.hazelcast.jet.datamodel.Tag.tag) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) Functions.wholeItem(com.hazelcast.function.Functions.wholeItem) AggregateOperations.aggregateOperation3(com.hazelcast.jet.aggregate.AggregateOperations.aggregateOperation3) Assert.assertEquals(org.junit.Assert.assertEquals) Tuple3(com.hazelcast.jet.datamodel.Tuple3) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 2 with BiConsumerEx

use of com.hazelcast.function.BiConsumerEx in project hazelcast by hazelcast.

the class AggregateOperation2Test method when_withCombiningAccumulateFn_then_accumulateFnCombines.

@Test
public void when_withCombiningAccumulateFn_then_accumulateFnCombines() {
    // Given
    AggregateOperation2<Object, Object, LongAccumulator, Long> aggrOp = AggregateOperation.withCreate(LongAccumulator::new).andAccumulate0((acc, item) -> acc.addAllowingOverflow(1)).andAccumulate1((acc, item) -> acc.addAllowingOverflow(10)).andCombine(LongAccumulator::addAllowingOverflow).andExportFinish(LongAccumulator::get);
    AggregateOperation1<LongAccumulator, LongAccumulator, Long> combiningAggrOp = aggrOp.withCombiningAccumulateFn(wholeItem());
    BiConsumerEx<? super LongAccumulator, ? super LongAccumulator> accFn = combiningAggrOp.accumulateFn();
    LongAccumulator partialAcc1 = combiningAggrOp.createFn().get();
    LongAccumulator partialAcc2 = combiningAggrOp.createFn().get();
    LongAccumulator combinedAcc = combiningAggrOp.createFn().get();
    // When
    partialAcc1.set(2);
    partialAcc2.set(3);
    accFn.accept(combinedAcc, partialAcc1);
    accFn.accept(combinedAcc, partialAcc2);
    // Then
    assertEquals(5, combinedAcc.get());
}
Also used : LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) FunctionEx(com.hazelcast.function.FunctionEx) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) AggregateOperations.aggregateOperation2(com.hazelcast.jet.aggregate.AggregateOperations.aggregateOperation2) QuickTest(com.hazelcast.test.annotation.QuickTest) RunWith(org.junit.runner.RunWith) BiConsumerEx(com.hazelcast.function.BiConsumerEx) Tag.tag2(com.hazelcast.jet.datamodel.Tag.tag2) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) SupplierEx(com.hazelcast.function.SupplierEx) Tag.tag1(com.hazelcast.jet.datamodel.Tag.tag1) Tag.tag0(com.hazelcast.jet.datamodel.Tag.tag0) Assert.assertSame(org.junit.Assert.assertSame) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) Functions.wholeItem(com.hazelcast.function.Functions.wholeItem) Tuple2(com.hazelcast.jet.datamodel.Tuple2) Assert.assertEquals(org.junit.Assert.assertEquals) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 3 with BiConsumerEx

use of com.hazelcast.function.BiConsumerEx in project hazelcast by hazelcast.

the class AggregateOperationTest method when_withCombiningAccumulateFn_then_accumulateFnCombines.

@Test
public void when_withCombiningAccumulateFn_then_accumulateFnCombines() {
    // Given
    AggregateOperation<LongAccumulator, Long> aggrOp = AggregateOperation.withCreate(LongAccumulator::new).andAccumulate(tag0(), (acc, item) -> acc.add(1)).andAccumulate(tag1(), (acc, item) -> acc.add(10)).andCombine(LongAccumulator::add).andExportFinish(LongAccumulator::get);
    AggregateOperation1<LongAccumulator, LongAccumulator, Long> combiningAggrOp = aggrOp.withCombiningAccumulateFn(wholeItem());
    BiConsumerEx<? super LongAccumulator, ? super Object> accFn = combiningAggrOp.accumulateFn(tag0());
    LongAccumulator partialAcc1 = combiningAggrOp.createFn().get();
    LongAccumulator partialAcc2 = combiningAggrOp.createFn().get();
    LongAccumulator combinedAcc = combiningAggrOp.createFn().get();
    // When
    partialAcc1.set(2);
    partialAcc2.set(3);
    accFn.accept(combinedAcc, partialAcc1);
    accFn.accept(combinedAcc, partialAcc2);
    // Then
    assertEquals(5, combinedAcc.get());
}
Also used : LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) FunctionEx(com.hazelcast.function.FunctionEx) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Tag(com.hazelcast.jet.datamodel.Tag) RunWith(org.junit.runner.RunWith) BiConsumerEx(com.hazelcast.function.BiConsumerEx) Tag.tag2(com.hazelcast.jet.datamodel.Tag.tag2) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) AggregateOperations.coAggregateOperationBuilder(com.hazelcast.jet.aggregate.AggregateOperations.coAggregateOperationBuilder) SupplierEx(com.hazelcast.function.SupplierEx) Tag.tag1(com.hazelcast.jet.datamodel.Tag.tag1) Tag.tag0(com.hazelcast.jet.datamodel.Tag.tag0) Assert.assertSame(org.junit.Assert.assertSame) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) Functions.wholeItem(com.hazelcast.function.Functions.wholeItem) Assert.assertEquals(org.junit.Assert.assertEquals) AggregateOperations.summingLong(com.hazelcast.jet.aggregate.AggregateOperations.summingLong) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 4 with BiConsumerEx

use of com.hazelcast.function.BiConsumerEx in project hazelcast by hazelcast.

the class CoAggregateOperationBuilder method build.

/**
 * Builds and returns the multi-input {@link AggregateOperation}. It will
 * call the supplied {@code exportFinishFn} to transform the {@link ItemsByTag}
 * it creates to the result type it emits as the actual result.
 *
 * @param exportFinishFn function to convert {@link ItemsByTag} to the
 *     target result type. It must be stateless and {@linkplain
 *     Processor#isCooperative() cooperative}.
 */
@Nonnull
@SuppressWarnings({ "unchecked", "ConstantConditions" })
public <R> AggregateOperation<Object[], R> build(@Nonnull FunctionEx<? super ItemsByTag, ? extends R> exportFinishFn) {
    checkSerializable(exportFinishFn, "exportFinishFn");
    Tag[] tags = opsByTag.keySet().stream().sorted().toArray(Tag[]::new);
    for (int i = 0; i < tags.length; i++) {
        Preconditions.checkTrue(tags[i].index() == i, "Registered tags' indices are " + stream(tags).map(Tag::index).collect(toList()) + ", but should be " + range(0, tags.length).boxed().collect(toList()));
    }
    // Variable `sorted` extracted due to type inference failure
    Stream<Entry<Tag, AggregateOperation1>> sorted = opsByTag.entrySet().stream().sorted(comparing(Entry::getKey));
    List<AggregateOperation1> ops = sorted.map(Entry::getValue).collect(toList());
    BiConsumerEx[] combineFns = ops.stream().map(AggregateOperation::combineFn).toArray(BiConsumerEx[]::new);
    BiConsumerEx[] deductFns = ops.stream().map(AggregateOperation::deductFn).toArray(BiConsumerEx[]::new);
    FunctionEx[] exportFns = ops.stream().map(AggregateOperation::exportFn).toArray(FunctionEx[]::new);
    FunctionEx[] finishFns = ops.stream().map(AggregateOperation::finishFn).toArray(FunctionEx[]::new);
    AggregateOperationBuilder.VarArity<Object[], Void> b = AggregateOperation.withCreate(() -> ops.stream().map(op -> op.createFn().get()).toArray()).varArity();
    opsByTag.forEach((tag, op) -> {
        int index = tag.index();
        b.andAccumulate(tag, (acc, item) -> op.accumulateFn().accept(acc[index], item));
    });
    return b.andCombine(stream(combineFns).anyMatch(Objects::isNull) ? null : (acc1, acc2) -> {
        for (int i = 0; i < combineFns.length; i++) {
            combineFns[i].accept(acc1[i], acc2[i]);
        }
    }).andDeduct(stream(deductFns).anyMatch(Objects::isNull) ? null : (acc1, acc2) -> {
        for (int i = 0; i < deductFns.length; i++) {
            deductFns[i].accept(acc1[i], acc2[i]);
        }
    }).<R>andExport(acc -> {
        ItemsByTag result = new ItemsByTag();
        for (int i = 0; i < exportFns.length; i++) {
            result.put(tags[i], exportFns[i].apply(acc[i]));
        }
        return exportFinishFn.apply(result);
    }).andFinish(acc -> {
        ItemsByTag result = new ItemsByTag();
        for (int i = 0; i < finishFns.length; i++) {
            result.put(tags[i], finishFns[i].apply(acc[i]));
        }
        return exportFinishFn.apply(result);
    });
}
Also used : FunctionEx(com.hazelcast.function.FunctionEx) Preconditions(com.hazelcast.internal.util.Preconditions) IntStream.range(java.util.stream.IntStream.range) SerializationUtil.checkSerializable(com.hazelcast.internal.serialization.impl.SerializationUtil.checkSerializable) Tag(com.hazelcast.jet.datamodel.Tag) BiConsumerEx(com.hazelcast.function.BiConsumerEx) Processor(com.hazelcast.jet.core.Processor) HashMap(java.util.HashMap) Objects(java.util.Objects) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) Stream(java.util.stream.Stream) StageWithKeyAndWindow(com.hazelcast.jet.pipeline.StageWithKeyAndWindow) Map(java.util.Map) FunctionEx.identity(com.hazelcast.function.FunctionEx.identity) ItemsByTag(com.hazelcast.jet.datamodel.ItemsByTag) Entry(java.util.Map.Entry) Comparator.comparing(java.util.Comparator.comparing) Nonnull(javax.annotation.Nonnull) Arrays.stream(java.util.Arrays.stream) ItemsByTag(com.hazelcast.jet.datamodel.ItemsByTag) Entry(java.util.Map.Entry) BiConsumerEx(com.hazelcast.function.BiConsumerEx) FunctionEx(com.hazelcast.function.FunctionEx) Objects(java.util.Objects) Tag(com.hazelcast.jet.datamodel.Tag) ItemsByTag(com.hazelcast.jet.datamodel.ItemsByTag) Nonnull(javax.annotation.Nonnull)

Aggregations

BiConsumerEx (com.hazelcast.function.BiConsumerEx)4 FunctionEx (com.hazelcast.function.FunctionEx)4 Functions.wholeItem (com.hazelcast.function.Functions.wholeItem)3 SupplierEx (com.hazelcast.function.SupplierEx)3 LongAccumulator (com.hazelcast.jet.accumulator.LongAccumulator)3 AggregateOperations.summingLong (com.hazelcast.jet.aggregate.AggregateOperations.summingLong)3 Tag.tag0 (com.hazelcast.jet.datamodel.Tag.tag0)3 Tag.tag1 (com.hazelcast.jet.datamodel.Tag.tag1)3 Tag.tag2 (com.hazelcast.jet.datamodel.Tag.tag2)3 HazelcastParallelClassRunner (com.hazelcast.test.HazelcastParallelClassRunner)3 ParallelJVMTest (com.hazelcast.test.annotation.ParallelJVMTest)3 QuickTest (com.hazelcast.test.annotation.QuickTest)3 Assert.assertEquals (org.junit.Assert.assertEquals)3 Assert.assertSame (org.junit.Assert.assertSame)3 Test (org.junit.Test)3 Category (org.junit.experimental.categories.Category)3 RunWith (org.junit.runner.RunWith)3 Tag (com.hazelcast.jet.datamodel.Tag)2 FunctionEx.identity (com.hazelcast.function.FunctionEx.identity)1 SerializationUtil.checkSerializable (com.hazelcast.internal.serialization.impl.SerializationUtil.checkSerializable)1