Search in sources :

Example 1 with GlobalInMemoryStateProcessor

use of io.kestra.runner.kafka.streams.GlobalInMemoryStateProcessor in project kestra by kestra-io.

the class ExecutorStore method topology.

public StreamsBuilder topology() {
    StreamsBuilder builder = new KafkaStreamsBuilder();
    builder.addGlobalStore(Stores.keyValueStoreBuilder(Stores.persistentKeyValueStore(FLOW_STATE_STORE_NAME), Serdes.String(), JsonSerde.of(Flow.class)), kafkaAdminService.getTopicName(Flow.class), Consumed.with(Serdes.String(), JsonSerde.of(Flow.class)).withName("GlobalStore.Flow"), () -> new GlobalInMemoryStateProcessor<>(FLOW_STATE_STORE_NAME, flows -> kafkaFlowExecutor.setFlows(flows), store -> kafkaFlowExecutor.setStore(store)));
    builder.addGlobalStore(Stores.keyValueStoreBuilder(Stores.persistentKeyValueStore(TEMPLATE_STATE_STORE_NAME), Serdes.String(), JsonSerde.of(Template.class)), kafkaAdminService.getTopicName(Template.class), Consumed.with(Serdes.String(), JsonSerde.of(Template.class)).withName("GlobalStore.Template"), () -> new GlobalInMemoryStateProcessor<>(TEMPLATE_STATE_STORE_NAME, templates -> kafkaTemplateExecutor.setTemplates(templates)));
    return builder;
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) KafkaStreamsBuilder(io.kestra.runner.kafka.services.KafkaStreamsBuilder) KafkaTemplateExecutor(io.kestra.runner.kafka.KafkaTemplateExecutor) StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) KafkaAdminService(io.kestra.runner.kafka.services.KafkaAdminService) Consumed(org.apache.kafka.streams.kstream.Consumed) Stores(org.apache.kafka.streams.state.Stores) Singleton(jakarta.inject.Singleton) KafkaFlowExecutor(io.kestra.runner.kafka.KafkaFlowExecutor) JsonSerde(io.kestra.runner.kafka.serializers.JsonSerde) Slf4j(lombok.extern.slf4j.Slf4j) Serdes(org.apache.kafka.common.serialization.Serdes) Flow(io.kestra.core.models.flows.Flow) Template(io.kestra.core.models.templates.Template) GlobalInMemoryStateProcessor(io.kestra.runner.kafka.streams.GlobalInMemoryStateProcessor) Inject(jakarta.inject.Inject) KafkaStreamsBuilder(io.kestra.runner.kafka.services.KafkaStreamsBuilder) KafkaQueueEnabled(io.kestra.runner.kafka.KafkaQueueEnabled) KafkaStreamsBuilder(io.kestra.runner.kafka.services.KafkaStreamsBuilder) Flow(io.kestra.core.models.flows.Flow) Template(io.kestra.core.models.templates.Template)

Aggregations

Flow (io.kestra.core.models.flows.Flow)1 Template (io.kestra.core.models.templates.Template)1 KafkaFlowExecutor (io.kestra.runner.kafka.KafkaFlowExecutor)1 KafkaQueueEnabled (io.kestra.runner.kafka.KafkaQueueEnabled)1 KafkaTemplateExecutor (io.kestra.runner.kafka.KafkaTemplateExecutor)1 JsonSerde (io.kestra.runner.kafka.serializers.JsonSerde)1 KafkaAdminService (io.kestra.runner.kafka.services.KafkaAdminService)1 KafkaStreamsBuilder (io.kestra.runner.kafka.services.KafkaStreamsBuilder)1 GlobalInMemoryStateProcessor (io.kestra.runner.kafka.streams.GlobalInMemoryStateProcessor)1 Inject (jakarta.inject.Inject)1 Singleton (jakarta.inject.Singleton)1 Slf4j (lombok.extern.slf4j.Slf4j)1 Serdes (org.apache.kafka.common.serialization.Serdes)1 StreamsBuilder (org.apache.kafka.streams.StreamsBuilder)1 Consumed (org.apache.kafka.streams.kstream.Consumed)1 Stores (org.apache.kafka.streams.state.Stores)1