Examples with StatefulProcessorNode org.apache.kafka.streams.kstream.internals.graph.StatefulProcessorNode used on opensource projects

Example 1 with StatefulProcessorNode

use of org.apache.kafka.streams.kstream.internals.graph.StatefulProcessorNode in project kafka by apache.

the class GroupedStreamAggregateBuilder method build.

<KR, VR> KTable<KR, VR> build(final NamedInternal functionName, final StoreBuilder<?> storeBuilder, final KStreamAggProcessorSupplier<K, V, KR, VR> aggregateSupplier, final String queryableStoreName, final Serde<KR> keySerde, final Serde<VR> valueSerde) {
    assert queryableStoreName == null || queryableStoreName.equals(storeBuilder.name());
    final String aggFunctionName = functionName.name();
    String sourceName = this.name;
    GraphNode parentNode = graphNode;
    if (repartitionRequired) {
        final OptimizableRepartitionNodeBuilder<K, V> repartitionNodeBuilder = optimizableRepartitionNodeBuilder();
        final String repartitionTopicPrefix = userProvidedRepartitionTopicName != null ? userProvidedRepartitionTopicName : storeBuilder.name();
        sourceName = createRepartitionSource(repartitionTopicPrefix, repartitionNodeBuilder);
        // the existing repartition node, otherwise we create a new one.
        if (repartitionNode == null || userProvidedRepartitionTopicName == null) {
            repartitionNode = repartitionNodeBuilder.build();
        }
        builder.addGraphNode(parentNode, repartitionNode);
        parentNode = repartitionNode;
    }
    final StatefulProcessorNode<K, V> statefulProcessorNode = new StatefulProcessorNode<>(aggFunctionName, new ProcessorParameters<>(aggregateSupplier, aggFunctionName), storeBuilder);
    builder.addGraphNode(parentNode, statefulProcessorNode);
    return new KTableImpl<>(aggFunctionName, keySerde, valueSerde, sourceName.equals(this.name) ? subTopologySourceNodes : Collections.singleton(sourceName), queryableStoreName, aggregateSupplier, statefulProcessorNode, builder);
}

Example 2 with StatefulProcessorNode

use of org.apache.kafka.streams.kstream.internals.graph.StatefulProcessorNode in project kafka by apache.

the class KGroupedTableImpl method doAggregate.

private <VAgg> KTable<K, VAgg> doAggregate(final ProcessorSupplier<K, Change<V>, K, Change<VAgg>> aggregateSupplier, final NamedInternal named, final String functionName, final MaterializedInternal<K, VAgg, KeyValueStore<Bytes, byte[]>> materialized) {
    final String sinkName = named.suffixWithOrElseGet("-sink", builder, KStreamImpl.SINK_NAME);
    final String sourceName = named.suffixWithOrElseGet("-source", builder, KStreamImpl.SOURCE_NAME);
    final String funcName = named.orElseGenerateWithPrefix(builder, functionName);
    final String repartitionTopic = (userProvidedRepartitionTopicName != null ? userProvidedRepartitionTopicName : materialized.storeName()) + KStreamImpl.REPARTITION_TOPIC_SUFFIX;
    if (repartitionGraphNode == null || userProvidedRepartitionTopicName == null) {
        repartitionGraphNode = createRepartitionNode(sinkName, sourceName, repartitionTopic);
    }
    // the passed in StreamsGraphNode must be the parent of the repartition node
    builder.addGraphNode(this.graphNode, repartitionGraphNode);
    final StatefulProcessorNode statefulProcessorNode = new StatefulProcessorNode<>(funcName, new ProcessorParameters<>(aggregateSupplier, funcName), new TimestampedKeyValueStoreMaterializer<>(materialized).materialize());
    // now the repartition node must be the parent of the StateProcessorNode
    builder.addGraphNode(repartitionGraphNode, statefulProcessorNode);
    // return the KTable representation with the intermediate topic as the sources
    return new KTableImpl<>(funcName, materialized.keySerde(), materialized.valueSerde(), Collections.singleton(sourceName), materialized.queryableStoreName(), aggregateSupplier, statefulProcessorNode, builder);
}