Search in sources :

Example 1 with AggregateBuilder

use of com.hazelcast.jet.pipeline.AggregateBuilder in project hazelcast-jet by hazelcast.

the class AggregateTransform_IntegrationTest method test_aggregate3_with_aggBuilder.

@Test
public void test_aggregate3_with_aggBuilder() {
    // Given
    JetInstance instance = createJetMember();
    IListJet<Integer> list = instance.getList("list");
    list.add(1);
    list.add(2);
    list.add(3);
    IListJet<String> list1 = instance.getList("list1");
    list1.add("a");
    list1.add("b");
    list1.add("c");
    IListJet<Double> list2 = instance.getList("list2");
    list2.add(6.0d);
    list2.add(7.0d);
    list2.add(8.0d);
    // When
    Pipeline p = Pipeline.create();
    BatchStage<Integer> stage0 = p.drawFrom(Sources.list("list"));
    BatchStage<String> stage1 = p.drawFrom(Sources.list("list1"));
    BatchStage<Double> stage2 = p.drawFrom(Sources.list("list2"));
    AggregateBuilder<Integer> builder = stage0.aggregateBuilder();
    Tag<Integer> tag0 = builder.tag0();
    Tag<String> tag1 = builder.add(stage1);
    Tag<Double> tag2 = builder.add(stage2);
    BatchStage<ThreeBags> resultStage = builder.build(AggregateOperation.withCreate(ThreeBags::threeBags).andAccumulate(tag0, (acc, item0) -> acc.bag0().add(item0)).andAccumulate(tag1, (acc, item1) -> acc.bag1().add(item1)).andAccumulate(tag2, (acc, item2) -> acc.bag2().add(item2)).andCombine(ThreeBags::combineWith).andDeduct(ThreeBags::deduct).andFinish(ThreeBags::finish));
    resultStage.drainTo(Sinks.list("sink"));
    instance.newJob(p).join();
    // Then
    ThreeBags threeBags = (ThreeBags) instance.getHazelcastInstance().getList("sink").iterator().next();
    assertNotNull(threeBags);
    sort(threeBags);
    assertEquals(threeBags(list, list1, list2), threeBags);
}
Also used : JetInstance(com.hazelcast.jet.JetInstance) RunWith(org.junit.runner.RunWith) ThreeBags.threeBags(com.hazelcast.jet.datamodel.ThreeBags.threeBags) TestUtil.set(com.hazelcast.jet.core.TestUtil.set) Collections.singletonList(java.util.Collections.singletonList) BatchStage(com.hazelcast.jet.pipeline.BatchStage) ThreeBags(com.hazelcast.jet.datamodel.ThreeBags) AggregateOperation(com.hazelcast.jet.aggregate.AggregateOperation) Util.entry(com.hazelcast.jet.Util.entry) AggregateOperations.toThreeBags(com.hazelcast.jet.aggregate.AggregateOperations.toThreeBags) Pipeline(com.hazelcast.jet.pipeline.Pipeline) JetTestSupport(com.hazelcast.jet.core.JetTestSupport) AggregateOperations.toTwoBags(com.hazelcast.jet.aggregate.AggregateOperations.toTwoBags) Assert.assertNotNull(org.junit.Assert.assertNotNull) AggregateOperations.toSet(com.hazelcast.jet.aggregate.AggregateOperations.toSet) Tag(com.hazelcast.jet.datamodel.Tag) Sinks(com.hazelcast.jet.pipeline.Sinks) Test(org.junit.Test) IListJet(com.hazelcast.jet.IListJet) AggregateBuilder(com.hazelcast.jet.pipeline.AggregateBuilder) ParallelTest(com.hazelcast.test.annotation.ParallelTest) Category(org.junit.experimental.categories.Category) Sources(com.hazelcast.jet.pipeline.Sources) List(java.util.List) IMap(com.hazelcast.core.IMap) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) TwoBags(com.hazelcast.jet.datamodel.TwoBags) Collections(java.util.Collections) Assert.assertEquals(org.junit.Assert.assertEquals) TestSupport.listToString(com.hazelcast.jet.core.test.TestSupport.listToString) JetInstance(com.hazelcast.jet.JetInstance) TestSupport.listToString(com.hazelcast.jet.core.test.TestSupport.listToString) Pipeline(com.hazelcast.jet.pipeline.Pipeline) ThreeBags(com.hazelcast.jet.datamodel.ThreeBags) AggregateOperations.toThreeBags(com.hazelcast.jet.aggregate.AggregateOperations.toThreeBags) Test(org.junit.Test) ParallelTest(com.hazelcast.test.annotation.ParallelTest)

Aggregations

IMap (com.hazelcast.core.IMap)1 IListJet (com.hazelcast.jet.IListJet)1 JetInstance (com.hazelcast.jet.JetInstance)1 Util.entry (com.hazelcast.jet.Util.entry)1 AggregateOperation (com.hazelcast.jet.aggregate.AggregateOperation)1 AggregateOperations.toSet (com.hazelcast.jet.aggregate.AggregateOperations.toSet)1 AggregateOperations.toThreeBags (com.hazelcast.jet.aggregate.AggregateOperations.toThreeBags)1 AggregateOperations.toTwoBags (com.hazelcast.jet.aggregate.AggregateOperations.toTwoBags)1 JetTestSupport (com.hazelcast.jet.core.JetTestSupport)1 TestUtil.set (com.hazelcast.jet.core.TestUtil.set)1 TestSupport.listToString (com.hazelcast.jet.core.test.TestSupport.listToString)1 Tag (com.hazelcast.jet.datamodel.Tag)1 ThreeBags (com.hazelcast.jet.datamodel.ThreeBags)1 ThreeBags.threeBags (com.hazelcast.jet.datamodel.ThreeBags.threeBags)1 TwoBags (com.hazelcast.jet.datamodel.TwoBags)1 AggregateBuilder (com.hazelcast.jet.pipeline.AggregateBuilder)1 BatchStage (com.hazelcast.jet.pipeline.BatchStage)1 Pipeline (com.hazelcast.jet.pipeline.Pipeline)1 Sinks (com.hazelcast.jet.pipeline.Sinks)1 Sources (com.hazelcast.jet.pipeline.Sources)1