Search in sources :

Example 1 with SerializableConsumer

use of com.twitter.heron.streamlet.SerializableConsumer in project incubator-heron by apache.

the class StreamletImplTest method testCalculatedDefaultStageNames.

@Test
@SuppressWarnings("unchecked")
public void testCalculatedDefaultStageNames() {
    // create SupplierStreamlet
    Streamlet<String> baseStreamlet = StreamletImpl.createSupplierStreamlet(() -> "This is test content");
    SupplierStreamlet<String> supplierStreamlet = (SupplierStreamlet<String>) baseStreamlet;
    assertEquals(supplierStreamlet.getChildren().size(), 0);
    // apply the consumer function
    baseStreamlet.consume((SerializableConsumer<String>) s -> {
    });
    // build SupplierStreamlet
    assertFalse(supplierStreamlet.isBuilt());
    TopologyBuilder builder = new TopologyBuilder();
    Set<String> stageNames = new HashSet<>();
    supplierStreamlet.build(builder, stageNames);
    // verify SupplierStreamlet
    assertTrue(supplierStreamlet.allBuilt());
    assertEquals(1, supplierStreamlet.getChildren().size());
    assertTrue(supplierStreamlet.getChildren().get(0) instanceof ConsumerStreamlet);
    assertEquals("consumer1", supplierStreamlet.getChildren().get(0).getName());
    // verify stageNames
    assertEquals(2, stageNames.size());
    List<String> expectedStageNames = Arrays.asList("consumer1", "supplier1");
    assertTrue(stageNames.containsAll(expectedStageNames));
    // verify ConsumerStreamlet
    ConsumerStreamlet<String> consumerStreamlet = (ConsumerStreamlet<String>) supplierStreamlet.getChildren().get(0);
    assertEquals(0, consumerStreamlet.getChildren().size());
}
Also used : ConsumerStreamlet(com.twitter.heron.streamlet.impl.streamlets.ConsumerStreamlet) Arrays(java.util.Arrays) TopologyBuilder(com.twitter.heron.api.topology.TopologyBuilder) SerializableConsumer(com.twitter.heron.streamlet.SerializableConsumer) Function(java.util.function.Function) JoinStreamlet(com.twitter.heron.streamlet.impl.streamlets.JoinStreamlet) SerializableTransformer(com.twitter.heron.streamlet.SerializableTransformer) HashSet(java.util.HashSet) Config(com.twitter.heron.streamlet.Config) FlatMapStreamlet(com.twitter.heron.streamlet.impl.streamlets.FlatMapStreamlet) ByteAmount(com.twitter.heron.common.basics.ByteAmount) Streamlet(com.twitter.heron.streamlet.Streamlet) WindowConfig(com.twitter.heron.streamlet.WindowConfig) FilterStreamlet(com.twitter.heron.streamlet.impl.streamlets.FilterStreamlet) UnionStreamlet(com.twitter.heron.streamlet.impl.streamlets.UnionStreamlet) Context(com.twitter.heron.streamlet.Context) TransformStreamlet(com.twitter.heron.streamlet.impl.streamlets.TransformStreamlet) ReduceByKeyAndWindowStreamlet(com.twitter.heron.streamlet.impl.streamlets.ReduceByKeyAndWindowStreamlet) Set(java.util.Set) Test(org.junit.Test) Consumer(java.util.function.Consumer) List(java.util.List) MapStreamlet(com.twitter.heron.streamlet.impl.streamlets.MapStreamlet) SupplierStreamlet(com.twitter.heron.streamlet.impl.streamlets.SupplierStreamlet) Assert(org.junit.Assert) ConsumerStreamlet(com.twitter.heron.streamlet.impl.streamlets.ConsumerStreamlet) TopologyBuilder(com.twitter.heron.api.topology.TopologyBuilder) SupplierStreamlet(com.twitter.heron.streamlet.impl.streamlets.SupplierStreamlet) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

TopologyBuilder (com.twitter.heron.api.topology.TopologyBuilder)1 ByteAmount (com.twitter.heron.common.basics.ByteAmount)1 Config (com.twitter.heron.streamlet.Config)1 Context (com.twitter.heron.streamlet.Context)1 SerializableConsumer (com.twitter.heron.streamlet.SerializableConsumer)1 SerializableTransformer (com.twitter.heron.streamlet.SerializableTransformer)1 Streamlet (com.twitter.heron.streamlet.Streamlet)1 WindowConfig (com.twitter.heron.streamlet.WindowConfig)1 ConsumerStreamlet (com.twitter.heron.streamlet.impl.streamlets.ConsumerStreamlet)1 FilterStreamlet (com.twitter.heron.streamlet.impl.streamlets.FilterStreamlet)1 FlatMapStreamlet (com.twitter.heron.streamlet.impl.streamlets.FlatMapStreamlet)1 JoinStreamlet (com.twitter.heron.streamlet.impl.streamlets.JoinStreamlet)1 MapStreamlet (com.twitter.heron.streamlet.impl.streamlets.MapStreamlet)1 ReduceByKeyAndWindowStreamlet (com.twitter.heron.streamlet.impl.streamlets.ReduceByKeyAndWindowStreamlet)1 SupplierStreamlet (com.twitter.heron.streamlet.impl.streamlets.SupplierStreamlet)1 TransformStreamlet (com.twitter.heron.streamlet.impl.streamlets.TransformStreamlet)1 UnionStreamlet (com.twitter.heron.streamlet.impl.streamlets.UnionStreamlet)1 Arrays (java.util.Arrays)1 HashSet (java.util.HashSet)1 List (java.util.List)1