Search in sources :

Example 11 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class DefaultExecutionGraphCacheTest method testCacheEntryCleanup.

/**
 * Tests that cache entries are cleaned up when their TTL has expired upon calling {@link
 * DefaultExecutionGraphCache#cleanup()}.
 */
@Test
public void testCacheEntryCleanup() throws Exception {
    final Time timeout = Time.milliseconds(100L);
    final Time timeToLive = Time.milliseconds(1L);
    final JobID expectedJobId2 = new JobID();
    final ExecutionGraphInfo expectedExecutionGraphInfo2 = new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().build());
    final AtomicInteger requestJobCalls = new AtomicInteger(0);
    final TestingRestfulGateway restfulGateway = new TestingRestfulGateway.Builder().setRequestExecutionGraphInfoFunction(jobId -> {
        requestJobCalls.incrementAndGet();
        if (jobId.equals(expectedJobId)) {
            return CompletableFuture.completedFuture(expectedExecutionGraphInfo);
        } else if (jobId.equals(expectedJobId2)) {
            return CompletableFuture.completedFuture(expectedExecutionGraphInfo2);
        } else {
            throw new AssertionError("Invalid job id received.");
        }
    }).build();
    try (ExecutionGraphCache executionGraphCache = new DefaultExecutionGraphCache(timeout, timeToLive)) {
        CompletableFuture<ExecutionGraphInfo> executionGraph1Future = executionGraphCache.getExecutionGraphInfo(expectedJobId, restfulGateway);
        CompletableFuture<ExecutionGraphInfo> executionGraph2Future = executionGraphCache.getExecutionGraphInfo(expectedJobId2, restfulGateway);
        assertEquals(expectedExecutionGraphInfo, executionGraph1Future.get());
        assertEquals(expectedExecutionGraphInfo2, executionGraph2Future.get());
        assertThat(requestJobCalls.get(), Matchers.equalTo(2));
        Thread.sleep(timeToLive.toMilliseconds());
        executionGraphCache.cleanup();
        assertTrue(executionGraphCache.size() == 0);
    }
}
Also used : FlinkException(org.apache.flink.util.FlinkException) Arrays(java.util.Arrays) BeforeClass(org.junit.BeforeClass) CompletableFuture(java.util.concurrent.CompletableFuture) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Assert.assertThat(org.junit.Assert.assertThat) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) TestLogger(org.apache.flink.util.TestLogger) Assert.fail(org.junit.Assert.fail) ExecutorService(java.util.concurrent.ExecutorService) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Collection(java.util.Collection) Matchers(org.hamcrest.Matchers) Assert.assertTrue(org.junit.Assert.assertTrue) RestfulGateway(org.apache.flink.runtime.webmonitor.RestfulGateway) Test(org.junit.Test) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) Preconditions(org.apache.flink.util.Preconditions) ExecutorUtils(org.apache.flink.util.ExecutorUtils) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) JobID(org.apache.flink.api.common.JobID) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Time(org.apache.flink.api.common.time.Time) Assert.assertEquals(org.junit.Assert.assertEquals) ConcurrentLinkedQueue(java.util.concurrent.ConcurrentLinkedQueue) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Time(org.apache.flink.api.common.time.Time) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) JobID(org.apache.flink.api.common.JobID) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) Test(org.junit.Test)

Example 12 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class JobResultTest method testCancelledJobThrowsJobCancellationException.

@Test
public void testCancelledJobThrowsJobCancellationException() throws Exception {
    final FlinkException cause = new FlinkException("Test exception");
    final JobResult jobResult = JobResult.createFrom(new ArchivedExecutionGraphBuilder().setJobID(new JobID()).setState(JobStatus.CANCELED).setFailureCause(new ErrorInfo(cause, 42L)).build());
    try {
        jobResult.toJobExecutionResult(getClass().getClassLoader());
        fail("Job should fail with an JobCancellationException.");
    } catch (JobCancellationException expected) {
        // the failure cause in the execution graph should not be the cause of the canceled job
        // result
        assertThat(expected.getCause(), is(nullValue()));
    }
}
Also used : JobCancellationException(org.apache.flink.runtime.client.JobCancellationException) ErrorInfo(org.apache.flink.runtime.executiongraph.ErrorInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) FlinkException(org.apache.flink.util.FlinkException) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 13 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class JobResultTest method testFailedJobThrowsJobExecutionException.

@Test
public void testFailedJobThrowsJobExecutionException() throws Exception {
    final FlinkException cause = new FlinkException("Test exception");
    final JobResult jobResult = JobResult.createFrom(new ArchivedExecutionGraphBuilder().setJobID(new JobID()).setState(JobStatus.FAILED).setFailureCause(new ErrorInfo(cause, 42L)).build());
    try {
        jobResult.toJobExecutionResult(getClass().getClassLoader());
        fail("Job should fail with JobExecutionException.");
    } catch (JobExecutionException expected) {
        assertThat(expected.getCause(), is(equalTo(cause)));
    }
}
Also used : JobExecutionException(org.apache.flink.runtime.client.JobExecutionException) ErrorInfo(org.apache.flink.runtime.executiongraph.ErrorInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) FlinkException(org.apache.flink.util.FlinkException) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 14 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class JobResultTest method testFailedJobIsFailureResult.

@Test
public void testFailedJobIsFailureResult() {
    final JobResult jobResult = JobResult.createFrom(new ArchivedExecutionGraphBuilder().setJobID(new JobID()).setState(JobStatus.FAILED).setFailureCause(new ErrorInfo(new FlinkException("Test exception"), 42L)).build());
    assertThat(jobResult.isSuccess(), is(false));
}
Also used : ErrorInfo(org.apache.flink.runtime.executiongraph.ErrorInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) JobID(org.apache.flink.api.common.JobID) FlinkException(org.apache.flink.util.FlinkException) Test(org.junit.Test)

Example 15 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class DefaultJobMasterServiceProcessTest method testSuccessOnTerminalState.

@Test
public void testSuccessOnTerminalState() throws Exception {
    final CompletableFuture<JobMasterService> jobMasterServiceFuture = new CompletableFuture<>();
    DefaultJobMasterServiceProcess serviceProcess = createTestInstance(jobMasterServiceFuture);
    jobMasterServiceFuture.complete(new TestingJobMasterService());
    ArchivedExecutionGraph archivedExecutionGraph = new ArchivedExecutionGraphBuilder().setState(JobStatus.FINISHED).build();
    serviceProcess.jobReachedGloballyTerminalState(new ExecutionGraphInfo(archivedExecutionGraph));
    assertThat(serviceProcess.getResultFuture()).isCompletedWithValueMatching(JobManagerRunnerResult::isSuccess).isCompletedWithValueMatching(r -> r.getExecutionGraphInfo().getArchivedExecutionGraph().getState() == JobStatus.FINISHED);
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ArchivedExecutionGraph(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Test(org.junit.jupiter.api.Test)

Aggregations

ArchivedExecutionGraphBuilder (org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder)29 ExecutionGraphInfo (org.apache.flink.runtime.scheduler.ExecutionGraphInfo)23 Test (org.junit.Test)22 JobID (org.apache.flink.api.common.JobID)12 CompletableFuture (java.util.concurrent.CompletableFuture)8 Time (org.apache.flink.api.common.time.Time)6 ErrorInfo (org.apache.flink.runtime.executiongraph.ErrorInfo)6 FlinkException (org.apache.flink.util.FlinkException)6 JobStatus (org.apache.flink.api.common.JobStatus)5 TestingJobManagerRunner (org.apache.flink.runtime.jobmaster.TestingJobManagerRunner)5 File (java.io.File)4 ArrayList (java.util.ArrayList)4 ExecutionException (java.util.concurrent.ExecutionException)4 ArchivedExecutionGraph (org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph)4 ArrayDeque (java.util.ArrayDeque)3 Arrays (java.util.Arrays)3 TimeUnit (java.util.concurrent.TimeUnit)3 Configuration (org.apache.flink.configuration.Configuration)3 BlobServer (org.apache.flink.runtime.blob.BlobServer)3 IOException (java.io.IOException)2