Search in sources :

Example 41 with Counter

use of org.apache.flink.metrics.Counter in project flink by apache.

the class PrometheusReporterTaskScopeTest method countersCanBeAddedSeveralTimesIfTheyDifferInLabels.

@Test
void countersCanBeAddedSeveralTimesIfTheyDifferInLabels() throws UnirestException {
    Counter counter1 = new SimpleCounter();
    counter1.inc(1);
    Counter counter2 = new SimpleCounter();
    counter2.inc(2);
    taskMetricGroup1.counter("my_counter", counter1);
    taskMetricGroup2.counter("my_counter", counter2);
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues1)).isEqualTo(1.);
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues2)).isEqualTo(2.);
}
Also used : SimpleCounter(org.apache.flink.metrics.SimpleCounter) Counter(org.apache.flink.metrics.Counter) SimpleCounter(org.apache.flink.metrics.SimpleCounter) Test(org.junit.jupiter.api.Test)

Example 42 with Counter

use of org.apache.flink.metrics.Counter in project flink by apache.

the class PrometheusReporterTaskScopeTest method removingSingleInstanceOfMetricDoesNotBreakOtherInstances.

@Test
void removingSingleInstanceOfMetricDoesNotBreakOtherInstances() throws UnirestException {
    Counter counter1 = new SimpleCounter();
    counter1.inc(1);
    Counter counter2 = new SimpleCounter();
    counter2.inc(2);
    taskMetricGroup1.counter("my_counter", counter1);
    taskMetricGroup2.counter("my_counter", counter2);
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues1)).isEqualTo(1.);
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues2)).isEqualTo(2.);
    taskMetricGroup2.close();
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues1)).isEqualTo(1.);
    taskMetricGroup1.close();
    assertThat(CollectorRegistry.defaultRegistry.getSampleValue("flink_taskmanager_job_task_my_counter", LABEL_NAMES, labelValues1)).isNull();
}
Also used : SimpleCounter(org.apache.flink.metrics.SimpleCounter) Counter(org.apache.flink.metrics.Counter) SimpleCounter(org.apache.flink.metrics.SimpleCounter) Test(org.junit.jupiter.api.Test)

Example 43 with Counter

use of org.apache.flink.metrics.Counter in project flink by apache.

the class StatsDReporterTest method testAddingMetrics.

/**
 * Tests that the registered metrics' names don't contain invalid characters.
 */
@Test
void testAddingMetrics() {
    String counterName = "testCounter";
    final String scope = "scope";
    final char delimiter = '_';
    MetricGroup metricGroup = TestMetricGroup.newBuilder().setMetricIdentifierFunction((metricName, characterFilter) -> scope + delimiter + metricName).build();
    TestingStatsDReporter reporter = new TestingStatsDReporter();
    reporter.open(new MetricConfig());
    SimpleCounter myCounter = new SimpleCounter();
    reporter.notifyOfAddedMetric(myCounter, counterName, metricGroup);
    Map<Counter, String> counters = reporter.getCounters();
    assertThat(counters).containsKey(myCounter);
    String expectedCounterName = reporter.filterCharacters(scope) + delimiter + reporter.filterCharacters(counterName);
    assertThat(counters.get(myCounter)).isEqualTo(expectedCounterName);
}
Also used : MetricConfig(org.apache.flink.metrics.MetricConfig) UnregisteredMetricsGroup(org.apache.flink.metrics.groups.UnregisteredMetricsGroup) TestMetricGroup(org.apache.flink.metrics.util.TestMetricGroup) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) TimeoutException(java.util.concurrent.TimeoutException) IOException(java.io.IOException) TestMeter(org.apache.flink.metrics.util.TestMeter) Metric(org.apache.flink.metrics.Metric) TestCounter(org.apache.flink.metrics.util.TestCounter) Test(org.junit.jupiter.api.Test) DatagramSocket(java.net.DatagramSocket) HashSet(java.util.HashSet) MetricGroup(org.apache.flink.metrics.MetricGroup) SocketException(java.net.SocketException) TestHistogram(org.apache.flink.metrics.util.TestHistogram) Map(java.util.Map) ConfigConstants(org.apache.flink.configuration.ConfigConstants) Gauge(org.apache.flink.metrics.Gauge) SimpleCounter(org.apache.flink.metrics.SimpleCounter) DatagramPacket(java.net.DatagramPacket) Counter(org.apache.flink.metrics.Counter) MetricConfig(org.apache.flink.metrics.MetricConfig) TestCounter(org.apache.flink.metrics.util.TestCounter) SimpleCounter(org.apache.flink.metrics.SimpleCounter) Counter(org.apache.flink.metrics.Counter) SimpleCounter(org.apache.flink.metrics.SimpleCounter) TestMetricGroup(org.apache.flink.metrics.util.TestMetricGroup) MetricGroup(org.apache.flink.metrics.MetricGroup) Test(org.junit.jupiter.api.Test)

Example 44 with Counter

use of org.apache.flink.metrics.Counter in project flink by apache.

the class JMXReporter method notifyOfAddedMetric.

// ------------------------------------------------------------------------
// adding / removing metrics
// ------------------------------------------------------------------------
@Override
public void notifyOfAddedMetric(Metric metric, String metricName, MetricGroup group) {
    final String domain = generateJmxDomain(metricName, group);
    final Hashtable<String, String> table = generateJmxTable(group.getAllVariables());
    AbstractBean jmxMetric;
    ObjectName jmxName;
    try {
        jmxName = new ObjectName(domain, table);
    } catch (MalformedObjectNameException e) {
        /*
             * There is an implementation error on our side if this occurs. Either the domain was
             * modified and no longer conforms to the JMX domain rules or the table wasn't properly
             * generated.
             */
        LOG.debug("Implementation error. The domain or table does not conform to JMX rules.", e);
        return;
    }
    if (metric instanceof Gauge) {
        jmxMetric = new JmxGauge((Gauge<?>) metric);
    } else if (metric instanceof Counter) {
        jmxMetric = new JmxCounter((Counter) metric);
    } else if (metric instanceof Histogram) {
        jmxMetric = new JmxHistogram((Histogram) metric);
    } else if (metric instanceof Meter) {
        jmxMetric = new JmxMeter((Meter) metric);
    } else {
        LOG.error("Cannot add unknown metric type: {}. This indicates that the metric type " + "is not supported by this reporter.", metric.getClass().getName());
        return;
    }
    try {
        synchronized (this) {
            mBeanServer.registerMBean(jmxMetric, jmxName);
            registeredMetrics.put(metric, jmxName);
        }
    } catch (NotCompliantMBeanException e) {
        // implementation error on our side
        LOG.debug("Metric did not comply with JMX MBean rules.", e);
    } catch (InstanceAlreadyExistsException e) {
        LOG.warn("A metric with the name " + jmxName + " was already registered.", e);
    } catch (Throwable t) {
        LOG.warn("Failed to register metric", t);
    }
}
Also used : MalformedObjectNameException(javax.management.MalformedObjectNameException) Histogram(org.apache.flink.metrics.Histogram) Meter(org.apache.flink.metrics.Meter) NotCompliantMBeanException(javax.management.NotCompliantMBeanException) InstanceAlreadyExistsException(javax.management.InstanceAlreadyExistsException) ObjectName(javax.management.ObjectName) Gauge(org.apache.flink.metrics.Gauge) Counter(org.apache.flink.metrics.Counter)

Example 45 with Counter

use of org.apache.flink.metrics.Counter in project flink by apache.

the class InfluxdbReporter method buildReport.

@Nullable
private BatchPoints buildReport() {
    Instant timestamp = Instant.now();
    BatchPoints.Builder report = BatchPoints.database(database);
    report.retentionPolicy(retentionPolicy);
    report.consistency(consistency);
    try {
        for (Map.Entry<Gauge<?>, MeasurementInfo> entry : gauges.entrySet()) {
            report.point(MetricMapper.map(entry.getValue(), timestamp, entry.getKey()));
        }
        for (Map.Entry<Counter, MeasurementInfo> entry : counters.entrySet()) {
            report.point(MetricMapper.map(entry.getValue(), timestamp, entry.getKey()));
        }
        for (Map.Entry<Histogram, MeasurementInfo> entry : histograms.entrySet()) {
            report.point(MetricMapper.map(entry.getValue(), timestamp, entry.getKey()));
        }
        for (Map.Entry<Meter, MeasurementInfo> entry : meters.entrySet()) {
            report.point(MetricMapper.map(entry.getValue(), timestamp, entry.getKey()));
        }
    } catch (ConcurrentModificationException | NoSuchElementException e) {
        // report next time
        return null;
    }
    return report.build();
}
Also used : Histogram(org.apache.flink.metrics.Histogram) ConcurrentModificationException(java.util.ConcurrentModificationException) BatchPoints(org.influxdb.dto.BatchPoints) Meter(org.apache.flink.metrics.Meter) Instant(java.time.Instant) Gauge(org.apache.flink.metrics.Gauge) Counter(org.apache.flink.metrics.Counter) Map(java.util.Map) NoSuchElementException(java.util.NoSuchElementException) Nullable(javax.annotation.Nullable)

Aggregations

Counter (org.apache.flink.metrics.Counter)78 SimpleCounter (org.apache.flink.metrics.SimpleCounter)24 Test (org.junit.Test)22 CountingCollector (org.apache.flink.runtime.operators.util.metrics.CountingCollector)18 Test (org.junit.jupiter.api.Test)16 Histogram (org.apache.flink.metrics.Histogram)15 Meter (org.apache.flink.metrics.Meter)15 Gauge (org.apache.flink.metrics.Gauge)14 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)12 CountingMutableObjectIterator (org.apache.flink.runtime.operators.util.metrics.CountingMutableObjectIterator)10 MetricGroup (org.apache.flink.metrics.MetricGroup)7 TestHistogram (org.apache.flink.metrics.util.TestHistogram)7 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)6 TestMeter (org.apache.flink.metrics.util.TestMeter)5 TaskConfig (org.apache.flink.runtime.operators.util.TaskConfig)5 IOException (java.io.IOException)4 HashMap (java.util.HashMap)4 Map (java.util.Map)4 OperatorIOMetricGroup (org.apache.flink.metrics.groups.OperatorIOMetricGroup)4 MetricListener (org.apache.flink.metrics.testutils.MetricListener)4