Search in sources :

Example 6 with CounterDistribution

use of org.apache.beam.runners.dataflow.worker.counters.CounterFactory.CounterDistribution in project beam by apache.

the class DataflowExecutionStateTrackerTest method assertElementProcessingTimeCounter.

private void assertElementProcessingTimeCounter(NameContext step, int millis, int bucketOffset) {
    CounterName counterName = ElementExecutionTracker.COUNTER_NAME.withOriginalName(step);
    Counter<?, CounterDistribution> counter = (Counter<?, CounterFactory.CounterDistribution>) counterSet.getExistingCounter(counterName);
    assertNotNull(counter);
    CounterFactory.CounterDistribution distribution = counter.getAggregate();
    assertThat(distribution, equalTo(CounterFactory.CounterDistribution.builder().minMax(millis, millis).count(1).sum(millis).sumOfSquares(millis * millis).buckets(bucketOffset, Lists.newArrayList(1L)).build()));
}
Also used : CounterDistribution(org.apache.beam.runners.dataflow.worker.counters.CounterFactory.CounterDistribution) Counter(org.apache.beam.runners.dataflow.worker.counters.Counter) CounterName(org.apache.beam.runners.dataflow.worker.counters.CounterName) CounterDistribution(org.apache.beam.runners.dataflow.worker.counters.CounterFactory.CounterDistribution) CounterFactory(org.apache.beam.runners.dataflow.worker.counters.CounterFactory)

Aggregations

CounterDistribution (org.apache.beam.runners.dataflow.worker.counters.CounterFactory.CounterDistribution)6 Test (org.junit.Test)5 CounterName (org.apache.beam.runners.dataflow.worker.counters.CounterName)4 Counter (org.apache.beam.runners.dataflow.worker.counters.Counter)3 DataflowPipelineDebugOptions (org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions)2 DataflowExecutionStateTracker (org.apache.beam.runners.dataflow.worker.DataflowExecutionContext.DataflowExecutionStateTracker)2 TestDataflowExecutionState (org.apache.beam.runners.dataflow.worker.TestOperationContext.TestDataflowExecutionState)2 NameContext (org.apache.beam.runners.dataflow.worker.counters.NameContext)2 TestReader (org.apache.beam.runners.dataflow.worker.util.common.worker.ExecutorTestUtils.TestReader)2 PipelineOptions (org.apache.beam.sdk.options.PipelineOptions)2 CounterUpdate (com.google.api.services.dataflow.model.CounterUpdate)1 ArrayList (java.util.ArrayList)1 ExecutionStateSampler (org.apache.beam.runners.core.metrics.ExecutionStateSampler)1 CounterFactory (org.apache.beam.runners.dataflow.worker.counters.CounterFactory)1 Operation (org.apache.beam.runners.dataflow.worker.util.common.worker.Operation)1 OutputReceiver (org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver)1 ParDoOperation (org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation)1 ReadOperation (org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation)1 TestOutputReceiver (org.apache.beam.runners.dataflow.worker.util.common.worker.TestOutputReceiver)1