Search in sources :

Example 1 with KafkaStreamsMetrics

use of io.micrometer.core.instrument.binder.kafka.KafkaStreamsMetrics in project kaldb by slackhq.

the class PreprocessorService method load.

/**
 * Configures and starts a KafkaStream processor, based off of the cached ServiceMetadataStore.
 * This method is reentrant, and will restart any existing KafkaStream processors. Access to this
 * must be synchronized if using this method as part of a listener.
 */
public synchronized void load() {
    try {
        Timer.Sample loadTimer = Timer.start(meterRegistry);
        LOG.info("Loading new Kafka stream processor config");
        if (kafkaStreams != null) {
            LOG.info("Closing existing Kafka stream processor");
            kafkaStreams.close();
            kafkaStreams.cleanUp();
        }
        if (kafkaStreamsMetrics != null) {
            kafkaStreamsMetrics.close();
        }
        // only attempt to register stream processing on valid service configurations
        List<ServiceMetadata> serviceMetadataToProcess = filterValidServiceMetadata(serviceMetadataStore.listSync());
        if (serviceMetadataToProcess.size() > 0) {
            Topology topology = buildTopology(serviceMetadataToProcess, rateLimiter, upstreamTopics, downstreamTopic, dataTransformer);
            kafkaStreams = new KafkaStreams(topology, kafkaProperties);
            kafkaStreamsMetrics = new KafkaStreamsMetrics(kafkaStreams);
            kafkaStreamsMetrics.bindTo(meterRegistry);
            kafkaStreams.start();
            LOG.info("Kafka stream processor config loaded successfully");
        } else {
            LOG.info("No valid service configurations found to process - will retry on next service configuration update");
        }
        loadTimer.stop(configReloadTimer);
    } catch (Exception e) {
        notifyFailed(e);
    }
}
Also used : KafkaStreamsMetrics(io.micrometer.core.instrument.binder.kafka.KafkaStreamsMetrics) KafkaStreams(org.apache.kafka.streams.KafkaStreams) Timer(io.micrometer.core.instrument.Timer) Topology(org.apache.kafka.streams.Topology) ServiceMetadata(com.slack.kaldb.metadata.service.ServiceMetadata)

Aggregations

ServiceMetadata (com.slack.kaldb.metadata.service.ServiceMetadata)1 Timer (io.micrometer.core.instrument.Timer)1 KafkaStreamsMetrics (io.micrometer.core.instrument.binder.kafka.KafkaStreamsMetrics)1 KafkaStreams (org.apache.kafka.streams.KafkaStreams)1 Topology (org.apache.kafka.streams.Topology)1