Search in sources :

Example 51 with KafkaMetric

use of org.apache.kafka.common.metrics.KafkaMetric in project cruise-control by linkedin.

the class CruiseControlMetricsReporter method init.

@Override
public void init(List<KafkaMetric> metrics) {
    for (KafkaMetric kafkaMetric : metrics) {
        addMetricIfInterested(kafkaMetric);
    }
    LOG.info("Added {} Kafka metrics for Cruise Control metrics during initialization.", _interestedMetrics.size());
    _metricsReporterRunner = new KafkaThread("CruiseControlMetricsReporterRunner", this, true);
    _yammerMetricProcessor = new YammerMetricProcessor();
    _metricsReporterRunner.start();
}
Also used : KafkaThread(org.apache.kafka.common.utils.KafkaThread) KafkaMetric(org.apache.kafka.common.metrics.KafkaMetric) YammerMetricProcessor(com.linkedin.kafka.cruisecontrol.metricsreporter.metric.YammerMetricProcessor)

Example 52 with KafkaMetric

use of org.apache.kafka.common.metrics.KafkaMetric in project cruise-control by linkedin.

the class CruiseControlMetricsReporter method reportKafkaMetrics.

private void reportKafkaMetrics(long now) {
    LOG.debug("Reporting KafkaMetrics. {}", _interestedMetrics.values());
    for (KafkaMetric metric : _interestedMetrics.values()) {
        sendCruiseControlMetric(MetricsUtils.toCruiseControlMetric(metric, now, _brokerId));
    }
    LOG.debug("Finished reporting KafkaMetrics.");
}
Also used : KafkaMetric(org.apache.kafka.common.metrics.KafkaMetric)

Example 53 with KafkaMetric

use of org.apache.kafka.common.metrics.KafkaMetric in project ksql by confluentinc.

the class StorageUtilizationMetricsReporterTest method mockMetric.

private KafkaMetric mockMetric(final String group, final String name, Object value, final Map<String, String> tags) {
    final KafkaMetric metric = mock(KafkaMetric.class);
    when(metric.metricName()).thenReturn(new MetricName(name, group, "", tags));
    when(metric.metricValue()).thenReturn(value);
    return metric;
}
Also used : MetricName(org.apache.kafka.common.MetricName) KafkaMetric(org.apache.kafka.common.metrics.KafkaMetric)

Example 54 with KafkaMetric

use of org.apache.kafka.common.metrics.KafkaMetric in project ksql by confluentinc.

the class StorageUtilizationMetricsReporterTest method shouldRemoveObsoleteStateStoreMetrics.

@Test
public void shouldRemoveObsoleteStateStoreMetrics() {
    // Given:
    listener.metricChange(mockMetric(KAFKA_METRIC_GROUP, KAFKA_METRIC_NAME, BigInteger.valueOf(2), ImmutableMap.of("store-id", "s1", "task-id", "t1", "thread-id", THREAD_ID)));
    final KafkaMetric metric = mockMetric(KAFKA_METRIC_GROUP, KAFKA_METRIC_NAME, BigInteger.valueOf(6), ImmutableMap.of("store-id", "s2", "task-id", "t1", "thread-id", THREAD_ID));
    listener.metricChange(metric);
    final Gauge<?> taskGauge = verifyAndGetRegisteredMetric(TASK_STORAGE_METRIC, TASK_ONE_TAGS);
    Object taskValue = taskGauge.value(null, 0);
    assertThat(taskValue, equalTo(BigInteger.valueOf(8)));
    // When:
    listener.metricRemoval(metric);
    // Then:
    taskValue = taskGauge.value(null, 0);
    assertThat(taskValue, equalTo(BigInteger.valueOf(2)));
}
Also used : KafkaMetric(org.apache.kafka.common.metrics.KafkaMetric) Test(org.junit.Test)

Example 55 with KafkaMetric

use of org.apache.kafka.common.metrics.KafkaMetric in project ksql by confluentinc.

the class ProducerCollector method addSensor.

private void addSensor(final String key, final String metricNameString, final MeasurableStat stat, final List<SensorMetric<ProducerRecord<Object, Object>>> results) {
    final String name = "prod-" + key + "-" + metricNameString + "-" + id;
    final MetricName metricName = new MetricName(metricNameString, "producer-metrics", "producer-" + name, ImmutableMap.of("key", key, "id", id));
    final Sensor existingSensor = metrics.getSensor(name);
    final Sensor sensor = metrics.sensor(name);
    // either a new sensor or a new metric with different id
    if (existingSensor == null || metrics.metrics().get(metricName) == null) {
        sensor.add(metricName, stat);
    }
    final KafkaMetric metric = metrics.metrics().get(metricName);
    results.add(new SensorMetric<ProducerRecord<Object, Object>>(sensor, metric, time, false) {

        void record(final ProducerRecord<Object, Object> record) {
            sensor.record(1);
            super.record(record);
        }
    });
}
Also used : MetricName(org.apache.kafka.common.MetricName) ProducerRecord(org.apache.kafka.clients.producer.ProducerRecord) KafkaMetric(org.apache.kafka.common.metrics.KafkaMetric) Sensor(org.apache.kafka.common.metrics.Sensor)

Aggregations

KafkaMetric (org.apache.kafka.common.metrics.KafkaMetric)129 Test (org.junit.Test)84 MetricName (org.apache.kafka.common.MetricName)59 HashMap (java.util.HashMap)30 LinkedHashMap (java.util.LinkedHashMap)24 Test (org.junit.jupiter.api.Test)23 Map (java.util.Map)18 MetricConfig (org.apache.kafka.common.metrics.MetricConfig)16 Metric (org.apache.kafka.common.Metric)15 MemoryRecordsBuilder (org.apache.kafka.common.record.MemoryRecordsBuilder)14 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)13 MeterRegistry (io.micrometer.core.instrument.MeterRegistry)12 SimpleMeterRegistry (io.micrometer.core.instrument.simple.SimpleMeterRegistry)12 Value (org.apache.kafka.common.metrics.stats.Value)12 Windowed (org.apache.kafka.streams.kstream.Windowed)12 Sensor (org.apache.kafka.common.metrics.Sensor)11 TopicPartition (org.apache.kafka.common.TopicPartition)10 Metrics (org.apache.kafka.common.metrics.Metrics)10 List (java.util.List)8 Cluster (org.apache.kafka.common.Cluster)8