Search in sources :

Example 21 with Accumulator

use of org.apache.flink.api.common.accumulators.Accumulator in project flink by apache.

the class ExecutionGraphDeploymentTest method testAccumulatorsAndMetricsStorage.

/**
	 * Verifies that {@link Execution#cancelingComplete(Map, IOMetrics)} and {@link Execution#markFailed(Throwable, Map, IOMetrics)}
	 * store the given accumulators and metrics correctly.
	 */
@Test
public void testAccumulatorsAndMetricsStorage() throws Exception {
    final JobVertexID jid1 = new JobVertexID();
    final JobVertexID jid2 = new JobVertexID();
    JobVertex v1 = new JobVertex("v1", jid1);
    JobVertex v2 = new JobVertex("v2", jid2);
    Map<ExecutionAttemptID, Execution> executions = setupExecution(v1, 1, v2, 1).f1;
    IOMetrics ioMetrics = new IOMetrics(0, 0, 0, 0, 0, 0.0, 0.0, 0.0, 0.0, 0.0);
    Map<String, Accumulator<?, ?>> accumulators = Collections.emptyMap();
    Execution execution1 = executions.values().iterator().next();
    execution1.cancel();
    execution1.cancelingComplete(accumulators, ioMetrics);
    assertEquals(ioMetrics, execution1.getIOMetrics());
    assertEquals(accumulators, execution1.getUserAccumulators());
    Execution execution2 = executions.values().iterator().next();
    execution2.markFailed(new Throwable(), accumulators, ioMetrics);
    assertEquals(ioMetrics, execution2.getIOMetrics());
    assertEquals(accumulators, execution2.getUserAccumulators());
}
Also used : Accumulator(org.apache.flink.api.common.accumulators.Accumulator) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) Test(org.junit.Test)

Aggregations

Accumulator (org.apache.flink.api.common.accumulators.Accumulator)21 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)15 Test (org.junit.Test)14 UnregisteredMetricsGroup (org.apache.flink.metrics.groups.UnregisteredMetricsGroup)12 Future (java.util.concurrent.Future)11 HashMap (java.util.HashMap)9 TaskInfo (org.apache.flink.api.common.TaskInfo)7 RuntimeUDFContext (org.apache.flink.api.common.functions.util.RuntimeUDFContext)6 Configuration (org.apache.flink.configuration.Configuration)5 ArrayList (java.util.ArrayList)4 RuntimeContext (org.apache.flink.api.common.functions.RuntimeContext)4 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)3 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)3 IOException (java.io.IOException)2 NoSuchElementException (java.util.NoSuchElementException)2 ExecutionException (java.util.concurrent.ExecutionException)2 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 IntCounter (org.apache.flink.api.common.accumulators.IntCounter)2 JobException (org.apache.flink.runtime.JobException)2 StoppingException (org.apache.flink.runtime.StoppingException)2