Search in sources :

Example 11 with Counter

use of org.apache.beam.sdk.metrics.Counter in project beam by apache.

the class LabeledMetricsTest method testOperationsUpdateCounterFromContainerWhenContainerIsPresent.

@Test
public void testOperationsUpdateCounterFromContainerWhenContainerIsPresent() {
    HashMap<String, String> labels = new HashMap<String, String>();
    String urn = MonitoringInfoConstants.Urns.ELEMENT_COUNT;
    MonitoringInfoMetricName name = MonitoringInfoMetricName.named(urn, labels);
    MetricsContainer mockContainer = Mockito.mock(MetricsContainer.class);
    Counter mockCounter = Mockito.mock(Counter.class);
    when(mockContainer.getCounter(name)).thenReturn(mockCounter);
    Counter counter = LabeledMetrics.counter(name);
    MetricsEnvironment.setCurrentContainer(mockContainer);
    counter.inc();
    verify(mockCounter).inc(1);
    counter.inc(47L);
    verify(mockCounter).inc(47);
    counter.dec(5L);
    verify(mockCounter).inc(-5);
}
Also used : Counter(org.apache.beam.sdk.metrics.Counter) HashMap(java.util.HashMap) MetricsContainer(org.apache.beam.sdk.metrics.MetricsContainer) Test(org.junit.Test)

Example 12 with Counter

use of org.apache.beam.sdk.metrics.Counter in project beam by apache.

the class LabeledMetricsTest method testCounterDoesNotFailOperationsWhenNoMetricsContainerPresent.

@Test
public void testCounterDoesNotFailOperationsWhenNoMetricsContainerPresent() {
    MetricsEnvironment.setCurrentContainer(null);
    assertNull(MetricsEnvironment.getCurrentContainer());
    HashMap<String, String> labels = new HashMap<String, String>();
    String urn = MonitoringInfoConstants.Urns.ELEMENT_COUNT;
    MonitoringInfoMetricName name = MonitoringInfoMetricName.named(urn, labels);
    Counter counter = LabeledMetrics.counter(name);
    counter.inc();
    counter.inc(5L);
    counter.dec();
    counter.dec(5L);
}
Also used : Counter(org.apache.beam.sdk.metrics.Counter) HashMap(java.util.HashMap) Test(org.junit.Test)

Example 13 with Counter

use of org.apache.beam.sdk.metrics.Counter in project beam by apache.

the class FlinkMetricContainerTest method testCounter.

@Test
public void testCounter() {
    SimpleCounter flinkCounter = new SimpleCounter();
    when(metricGroup.counter("namespace.name")).thenReturn(flinkCounter);
    MetricsContainer step = container.getMetricsContainer("step");
    MetricName metricName = MetricName.named("namespace", "name");
    Counter counter = step.getCounter(metricName);
    counter.inc();
    counter.inc();
    assertThat(flinkCounter.getCount(), is(0L));
    container.updateMetrics("step");
    assertThat(flinkCounter.getCount(), is(2L));
}
Also used : MetricName(org.apache.beam.sdk.metrics.MetricName) MonitoringInfoMetricName(org.apache.beam.runners.core.metrics.MonitoringInfoMetricName) SimpleCounter(org.apache.flink.metrics.SimpleCounter) Counter(org.apache.beam.sdk.metrics.Counter) SimpleCounter(org.apache.flink.metrics.SimpleCounter) MetricsContainer(org.apache.beam.sdk.metrics.MetricsContainer) Test(org.junit.Test)

Example 14 with Counter

use of org.apache.beam.sdk.metrics.Counter in project beam by apache.

the class PipelineRunnerTest method testRunPTransform.

@Test
@Category({ NeedsRunner.class, UsesCommittedMetrics.class, UsesCounterMetrics.class })
public void testRunPTransform() {
    final String namespace = PipelineRunnerTest.class.getName();
    final Counter counter = Metrics.counter(namespace, "count");
    final PipelineResult result = PipelineRunner.fromOptions(p.getOptions()).run(new PTransform<PBegin, POutput>() {

        @Override
        public POutput expand(PBegin input) {
            PCollection<Double> output = input.apply(Create.of(1, 2, 3, 4)).apply("ScaleByTwo", MapElements.via(new ScaleFn<>(2.0, counter)));
            PAssert.that(output).containsInAnyOrder(2.0, 4.0, 6.0, 8.0);
            return output;
        }
    });
    // Checking counters to verify the pipeline actually ran.
    assertThat(result.metrics().queryMetrics(MetricsFilter.builder().addNameFilter(MetricNameFilter.inNamespace(namespace)).build()).getCounters(), hasItem(metricsResult(namespace, "count", "ScaleByTwo", 4L, true)));
}
Also used : PCollection(org.apache.beam.sdk.values.PCollection) Counter(org.apache.beam.sdk.metrics.Counter) PipelineResult(org.apache.beam.sdk.PipelineResult) POutput(org.apache.beam.sdk.values.POutput) PBegin(org.apache.beam.sdk.values.PBegin) Category(org.junit.experimental.categories.Category) Test(org.junit.Test)

Aggregations

Counter (org.apache.beam.sdk.metrics.Counter)14 Test (org.junit.Test)7 DoFn (org.apache.beam.sdk.transforms.DoFn)4 HashMap (java.util.HashMap)3 Event (org.apache.beam.sdk.nexmark.model.Event)3 KV (org.apache.beam.sdk.values.KV)3 MonitoringInfoMetricName (org.apache.beam.runners.core.metrics.MonitoringInfoMetricName)2 MetricsContainer (org.apache.beam.sdk.metrics.MetricsContainer)2 TopicPartition (org.apache.kafka.common.TopicPartition)2 Nullable (org.checkerframework.checker.nullness.qual.Nullable)2 CounterMetadata (com.google.api.services.dataflow.model.CounterMetadata)1 CounterStructuredName (com.google.api.services.dataflow.model.CounterStructuredName)1 CounterStructuredNameAndMetadata (com.google.api.services.dataflow.model.CounterStructuredNameAndMetadata)1 CounterUpdate (com.google.api.services.dataflow.model.CounterUpdate)1 ByteArrayInputStream (java.io.ByteArrayInputStream)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 IOException (java.io.IOException)1 OutputStream (java.io.OutputStream)1 HashSet (java.util.HashSet)1 Random (java.util.Random)1