Search in sources :

Example 6 with MetricKey

use of org.apache.beam.sdk.metrics.MetricKey in project beam by apache.

the class JetMetricsContainer method extractUpdates.

private <UpdateT, CellT extends AbstractMetric<UpdateT>> ImmutableList<MetricUpdates.MetricUpdate<UpdateT>> extractUpdates(Map<MetricName, CellT> cells) {
    ImmutableList.Builder<MetricUpdates.MetricUpdate<UpdateT>> updates = ImmutableList.builder();
    for (CellT cell : cells.values()) {
        UpdateT value = cell.getValue();
        if (value != null) {
            MetricKey key = MetricKey.create(stepName, cell.getName());
            MetricUpdates.MetricUpdate<UpdateT> update = MetricUpdates.MetricUpdate.create(key, value);
            updates.add(update);
        }
    }
    return updates.build();
}
Also used : MetricKey(org.apache.beam.sdk.metrics.MetricKey) MetricUpdates(org.apache.beam.runners.core.metrics.MetricUpdates) ImmutableList(org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableList)

Example 7 with MetricKey

use of org.apache.beam.sdk.metrics.MetricKey in project beam by apache.

the class PortableMetrics method convertDistributionMonitoringInfoToDistribution.

private static MetricResult<DistributionResult> convertDistributionMonitoringInfoToDistribution(MetricsApi.MonitoringInfo monitoringInfo) {
    Map<String, String> labelsMap = monitoringInfo.getLabelsMap();
    MetricKey key = MetricKey.create(labelsMap.get(STEP_NAME_LABEL), MetricName.named(labelsMap.get(NAMESPACE_LABEL), labelsMap.get(METRIC_NAME_LABEL)));
    DistributionData data = decodeInt64Distribution(monitoringInfo.getPayload());
    DistributionResult result = DistributionResult.create(data.sum(), data.count(), data.min(), data.max());
    return MetricResult.create(key, false, result);
}
Also used : MetricKey(org.apache.beam.sdk.metrics.MetricKey) DistributionResult(org.apache.beam.sdk.metrics.DistributionResult) DistributionData(org.apache.beam.runners.core.metrics.DistributionData)

Example 8 with MetricKey

use of org.apache.beam.sdk.metrics.MetricKey in project beam by apache.

the class PortableMetrics method convertCounterMonitoringInfoToCounter.

private static MetricResult<Long> convertCounterMonitoringInfoToCounter(MetricsApi.MonitoringInfo counterMonInfo) {
    Map<String, String> labelsMap = counterMonInfo.getLabelsMap();
    MetricKey key = MetricKey.create(labelsMap.get(STEP_NAME_LABEL), MetricName.named(labelsMap.get(NAMESPACE_LABEL), labelsMap.get(METRIC_NAME_LABEL)));
    return MetricResult.create(key, false, decodeInt64Counter(counterMonInfo.getPayload()));
}
Also used : MetricKey(org.apache.beam.sdk.metrics.MetricKey)

Example 9 with MetricKey

use of org.apache.beam.sdk.metrics.MetricKey in project flink by apache.

the class FlinkMetricContainerTest method testGetNameSpaceArray.

@Test
public void testGetNameSpaceArray() {
    String json = "[\"key\", \"value\", \"MetricGroupType.key\", \"MetricGroupType.value\"]";
    MetricKey key = MetricKey.create("step", MetricName.named(json, "name"));
    assertThat(FlinkMetricContainer.getNameSpaceArray(key), is(DEFAULT_SCOPE_COMPONENTS));
}
Also used : MetricKey(org.apache.beam.sdk.metrics.MetricKey) MetricGroupTest(org.apache.flink.runtime.metrics.groups.MetricGroupTest) Test(org.junit.Test)

Example 10 with MetricKey

use of org.apache.beam.sdk.metrics.MetricKey in project flink by apache.

the class FlinkMetricContainerTest method testRegisterMetricGroup.

@Test
public void testRegisterMetricGroup() {
    MetricKey key = MetricKey.create("step", MetricName.named(DEFAULT_NAMESPACE, "name"));
    MetricRegistry registry = NoOpMetricRegistry.INSTANCE;
    GenericMetricGroup root = new GenericMetricGroup(registry, new MetricGroupTest.DummyAbstractMetricGroup(registry), "root");
    MetricGroup metricGroup = FlinkMetricContainer.registerMetricGroup(key, root);
    assertThat(metricGroup.getScopeComponents(), is(Arrays.asList("root", "key", "value").toArray()));
}
Also used : MetricKey(org.apache.beam.sdk.metrics.MetricKey) MetricRegistry(org.apache.flink.runtime.metrics.MetricRegistry) NoOpMetricRegistry(org.apache.flink.runtime.metrics.NoOpMetricRegistry) OperatorMetricGroup(org.apache.flink.metrics.groups.OperatorMetricGroup) MetricGroup(org.apache.flink.metrics.MetricGroup) GenericMetricGroup(org.apache.flink.runtime.metrics.groups.GenericMetricGroup) MetricGroupTest(org.apache.flink.runtime.metrics.groups.MetricGroupTest) GenericMetricGroup(org.apache.flink.runtime.metrics.groups.GenericMetricGroup) MetricGroupTest(org.apache.flink.runtime.metrics.groups.MetricGroupTest) Test(org.junit.Test)

Aggregations

MetricKey (org.apache.beam.sdk.metrics.MetricKey)12 Test (org.junit.Test)4 MetricName (org.apache.beam.sdk.metrics.MetricName)3 MetricGroupTest (org.apache.flink.runtime.metrics.groups.MetricGroupTest)3 ArrayList (java.util.ArrayList)2 VisibleForTesting (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.annotations.VisibleForTesting)2 DistributionData (org.apache.beam.runners.core.metrics.DistributionData)1 GaugeData (org.apache.beam.runners.core.metrics.GaugeData)1 MetricUpdates (org.apache.beam.runners.core.metrics.MetricUpdates)1 FlinkMetricContainer.getFlinkMetricNameString (org.apache.beam.runners.flink.metrics.FlinkMetricContainer.getFlinkMetricNameString)1 DistributionResult (org.apache.beam.sdk.metrics.DistributionResult)1 GaugeResult (org.apache.beam.sdk.metrics.GaugeResult)1 ImmutableList (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableList)1 MetricGroup (org.apache.flink.metrics.MetricGroup)1 OperatorMetricGroup (org.apache.flink.metrics.groups.OperatorMetricGroup)1 MetricRegistry (org.apache.flink.runtime.metrics.MetricRegistry)1 NoOpMetricRegistry (org.apache.flink.runtime.metrics.NoOpMetricRegistry)1 GenericMetricGroup (org.apache.flink.runtime.metrics.groups.GenericMetricGroup)1 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)1