Search in sources :

Example 11 with FlinkMetricContainer

use of org.apache.beam.runners.flink.metrics.FlinkMetricContainer in project beam by apache.

the class FlinkStatefulDoFnFunctionTest method testAccumulatorRegistrationOnOperatorClose.

@Test
public void testAccumulatorRegistrationOnOperatorClose() throws Exception {
    FlinkStatefulDoFnFunction doFnFunction = new TestDoFnFunction("step", WindowingStrategy.globalDefault(), Collections.emptyMap(), PipelineOptionsFactory.create(), Collections.emptyMap(), new TupleTag<>(), null, Collections.emptyMap(), DoFnSchemaInformation.create(), Collections.emptyMap());
    doFnFunction.open(new Configuration());
    String metricContainerFieldName = "metricContainer";
    FlinkMetricContainer monitoredContainer = Mockito.spy((FlinkMetricContainer) Whitebox.getInternalState(doFnFunction, metricContainerFieldName));
    Whitebox.setInternalState(doFnFunction, metricContainerFieldName, monitoredContainer);
    doFnFunction.close();
    Mockito.verify(monitoredContainer).registerMetricsForPipelineResult();
}
Also used : Configuration(org.apache.flink.configuration.Configuration) FlinkMetricContainer(org.apache.beam.runners.flink.metrics.FlinkMetricContainer) Test(org.junit.Test)

Example 12 with FlinkMetricContainer

use of org.apache.beam.runners.flink.metrics.FlinkMetricContainer in project beam by apache.

the class FlinkDoFnFunctionTest method testAccumulatorRegistrationOnOperatorClose.

@Test
public void testAccumulatorRegistrationOnOperatorClose() throws Exception {
    FlinkDoFnFunction doFnFunction = new TestDoFnFunction("step", WindowingStrategy.globalDefault(), Collections.emptyMap(), PipelineOptionsFactory.create(), Collections.emptyMap(), new TupleTag<>(), null, Collections.emptyMap(), DoFnSchemaInformation.create(), Collections.emptyMap());
    doFnFunction.open(new Configuration());
    String metricContainerFieldName = "metricContainer";
    FlinkMetricContainer monitoredContainer = Mockito.spy((FlinkMetricContainer) Whitebox.getInternalState(doFnFunction, metricContainerFieldName));
    Whitebox.setInternalState(doFnFunction, metricContainerFieldName, monitoredContainer);
    doFnFunction.close();
    Mockito.verify(monitoredContainer).registerMetricsForPipelineResult();
}
Also used : Configuration(org.apache.flink.configuration.Configuration) FlinkMetricContainer(org.apache.beam.runners.flink.metrics.FlinkMetricContainer) Test(org.junit.Test)

Aggregations

FlinkMetricContainer (org.apache.beam.runners.flink.metrics.FlinkMetricContainer)12 Test (org.junit.Test)5 FlinkPipelineOptions (org.apache.beam.runners.flink.FlinkPipelineOptions)4 SerializablePipelineOptions (org.apache.beam.runners.core.construction.SerializablePipelineOptions)3 PipelineOptions (org.apache.beam.sdk.options.PipelineOptions)3 Configuration (org.apache.flink.configuration.Configuration)3 ArrayList (java.util.ArrayList)2 DoFnRunners (org.apache.beam.runners.core.DoFnRunners)2 WindowedValue (org.apache.beam.sdk.util.WindowedValue)2 TupleTag (org.apache.beam.sdk.values.TupleTag)2 Watermark (org.apache.flink.streaming.api.watermark.Watermark)2 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 OutputStream (java.io.OutputStream)1 Serializable (java.io.Serializable)1 ByteBuffer (java.nio.ByteBuffer)1 Collection (java.util.Collection)1 HashMap (java.util.HashMap)1 Iterator (java.util.Iterator)1