Search in sources :

Example 36 with MonitoringInfo

use of org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo in project beam by apache.

the class MeanByteCountMonitoringInfoToCounterUpdateTransformerTest method testTransformThrowsIfMonitoringInfoWithWrongUrnReceived.

@Test
public void testTransformThrowsIfMonitoringInfoWithWrongUrnReceived() {
    Map<String, NameContext> pcollectionNameMapping = new HashMap<>();
    MonitoringInfo monitoringInfo = MonitoringInfo.newBuilder().setUrn(Urns.ELEMENT_COUNT).setType(TypeUrns.SUM_INT64_TYPE).build();
    MeanByteCountMonitoringInfoToCounterUpdateTransformer testObject = new MeanByteCountMonitoringInfoToCounterUpdateTransformer(mockSpecValidator, pcollectionNameMapping);
    when(mockSpecValidator.validate(any())).thenReturn(Optional.empty());
    exception.expect(RuntimeException.class);
    testObject.transform(monitoringInfo);
}
Also used : MonitoringInfo(org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo) HashMap(java.util.HashMap) NameContext(org.apache.beam.runners.dataflow.worker.counters.NameContext) Test(org.junit.Test)

Example 37 with MonitoringInfo

use of org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo in project beam by apache.

the class ElementCountMonitoringInfoToCounterUpdateTransformerTest method testTransformReturnsNullIfMonitoringInfoWithUnknownPCollectionLabelPresent.

@Test
public void testTransformReturnsNullIfMonitoringInfoWithUnknownPCollectionLabelPresent() {
    Map<String, NameContext> pcollectionNameMapping = new HashMap<>();
    MonitoringInfo monitoringInfo = MonitoringInfo.newBuilder().setUrn(Urns.ELEMENT_COUNT).setType(TypeUrns.SUM_INT64_TYPE).putLabels(MonitoringInfoConstants.Labels.PCOLLECTION, "anyValue").build();
    ElementCountMonitoringInfoToCounterUpdateTransformer testObject = new ElementCountMonitoringInfoToCounterUpdateTransformer(mockSpecValidator, pcollectionNameMapping);
    when(mockSpecValidator.validate(any())).thenReturn(Optional.empty());
    assertNull(testObject.transform(monitoringInfo));
}
Also used : MonitoringInfo(org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo) HashMap(java.util.HashMap) NameContext(org.apache.beam.runners.dataflow.worker.counters.NameContext) Test(org.junit.Test)

Example 38 with MonitoringInfo

use of org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo in project beam by apache.

the class ElementCountMonitoringInfoToCounterUpdateTransformerTest method testTransformReturnsValidCounterUpdateWhenValidMonitoringInfoReceived.

@Test
public void testTransformReturnsValidCounterUpdateWhenValidMonitoringInfoReceived() throws Exception {
    Map<String, NameContext> pcollectionNameMapping = new HashMap<>();
    pcollectionNameMapping.put("anyValue", NameContext.create("anyStageName", "anyOriginName", "anySystemName", "transformedValue"));
    MonitoringInfo monitoringInfo = MonitoringInfo.newBuilder().setUrn(Urns.ELEMENT_COUNT).setType(TypeUrns.SUM_INT64_TYPE).putLabels(MonitoringInfoConstants.Labels.PCOLLECTION, "anyValue").setPayload(encodeInt64Counter(1L)).build();
    ElementCountMonitoringInfoToCounterUpdateTransformer testObject = new ElementCountMonitoringInfoToCounterUpdateTransformer(mockSpecValidator, pcollectionNameMapping);
    when(mockSpecValidator.validate(any())).thenReturn(Optional.empty());
    CounterUpdate result = testObject.transform(monitoringInfo);
    assertNotNull(result);
    assertEqualsAsJson("{cumulative:true, integer:{highBits:0, lowBits:1}, " + "nameAndKind:{kind:'SUM', " + "name:'transformedValue-ElementCount'}}", result);
}
Also used : MonitoringInfo(org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo) HashMap(java.util.HashMap) NameContext(org.apache.beam.runners.dataflow.worker.counters.NameContext) CounterUpdate(com.google.api.services.dataflow.model.CounterUpdate) Test(org.junit.Test)

Example 39 with MonitoringInfo

use of org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo in project beam by apache.

the class ElementCountMonitoringInfoToCounterUpdateTransformerTest method testTransformThrowsIfMonitoringInfoWithWrongUrnPrefixReceived.

@Test
public void testTransformThrowsIfMonitoringInfoWithWrongUrnPrefixReceived() {
    Map<String, NameContext> pcollectionNameMapping = new HashMap<>();
    MonitoringInfo monitoringInfo = MonitoringInfo.newBuilder().setUrn(Urns.USER_SUM_INT64).build();
    ElementCountMonitoringInfoToCounterUpdateTransformer testObject = new ElementCountMonitoringInfoToCounterUpdateTransformer(mockSpecValidator, pcollectionNameMapping);
    when(mockSpecValidator.validate(any())).thenReturn(Optional.empty());
    exception.expect(RuntimeException.class);
    testObject.transform(monitoringInfo);
}
Also used : MonitoringInfo(org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo) HashMap(java.util.HashMap) NameContext(org.apache.beam.runners.dataflow.worker.counters.NameContext) Test(org.junit.Test)

Example 40 with MonitoringInfo

use of org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo in project beam by apache.

the class FnApiMonitoringInfoToCounterUpdateTransformerTest method testTransformReturnsNullOnUnknownUrn.

@Test
public void testTransformReturnsNullOnUnknownUrn() {
    Map<String, MonitoringInfoToCounterUpdateTransformer> genericTransformers = new HashMap<>();
    final String validUrn = "known:urn";
    final String validType = "known:type";
    genericTransformers.put(validUrn, mockTransformer2);
    FnApiMonitoringInfoToCounterUpdateTransformer testObject = new FnApiMonitoringInfoToCounterUpdateTransformer(genericTransformers);
    MonitoringInfo unknownUrn = MonitoringInfo.newBuilder().setUrn("unknown:urn").setType(validType).putLabels(MonitoringInfoConstants.Labels.PTRANSFORM, "anyValue").build();
    assertNull(testObject.transform(unknownUrn));
    verifyZeroInteractions(mockTransformer1, mockTransformer2);
}
Also used : MonitoringInfo(org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo) HashMap(java.util.HashMap) Test(org.junit.Test)

Aggregations

MonitoringInfo (org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo)40 Test (org.junit.Test)39 HashMap (java.util.HashMap)23 DataflowStepContext (org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.DataflowStepContext)11 SimpleMonitoringInfoBuilder (org.apache.beam.runners.core.metrics.SimpleMonitoringInfoBuilder)10 ArrayList (java.util.ArrayList)9 NameContext (org.apache.beam.runners.dataflow.worker.counters.NameContext)9 CounterUpdate (com.google.api.services.dataflow.model.CounterUpdate)7 WindowedValue (org.apache.beam.sdk.util.WindowedValue)4 SimpleCounter (org.apache.flink.metrics.SimpleCounter)4 MetricGroupTest (org.apache.flink.runtime.metrics.groups.MetricGroupTest)4 ExecutionStateTracker (org.apache.beam.runners.core.metrics.ExecutionStateTracker)3 MetricsContainerStepMap (org.apache.beam.runners.core.metrics.MetricsContainerStepMap)3 FnDataReceiver (org.apache.beam.sdk.fn.data.FnDataReceiver)3 ByteString (org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString)3 DistributionData (org.apache.beam.runners.core.metrics.DistributionData)2 StringUtf8Coder (org.apache.beam.sdk.coders.StringUtf8Coder)2 DistributionResult (org.apache.beam.sdk.metrics.DistributionResult)2 ArgumentMatcher (org.mockito.ArgumentMatcher)2 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)2