Search in sources :

Example 6 with FlinkJobNotFoundException

use of org.apache.flink.runtime.messages.FlinkJobNotFoundException in project flink by apache.

the class JobMasterQueryableStateTest method testRequestKvStateWithIrrelevantRegistration.

@Test
public void testRequestKvStateWithIrrelevantRegistration() throws Exception {
    final JobMaster jobMaster = new JobMasterBuilder(JOB_GRAPH, rpcService).createJobMaster();
    jobMaster.start();
    final JobMasterGateway jobMasterGateway = jobMaster.getSelfGateway(JobMasterGateway.class);
    registerSlotsRequiredForJobExecution(jobMasterGateway, JOB_GRAPH.getJobID());
    try {
        // register an irrelevant KvState
        try {
            registerKvState(jobMasterGateway, new JobID(), new JobVertexID(), "any-name");
            fail("Expected to fail with FlinkJobNotFoundException.");
        } catch (Exception e) {
            assertThat(e, containsCause(FlinkJobNotFoundException.class));
        }
    } finally {
        RpcUtils.terminateRpcEndpoint(jobMaster, testingTimeout);
    }
}
Also used : JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) JobMasterBuilder(org.apache.flink.runtime.jobmaster.utils.JobMasterBuilder) JobID(org.apache.flink.api.common.JobID) UnknownHostException(java.net.UnknownHostException) ExecutionException(java.util.concurrent.ExecutionException) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Test(org.junit.Test)

Example 7 with FlinkJobNotFoundException

use of org.apache.flink.runtime.messages.FlinkJobNotFoundException in project flink by apache.

the class ApplicationDispatcherBootstrapTest method testDuplicateJobSubmissionWithTerminatedJobIdWithUnknownResultAttached.

/**
 * In this scenario, job result is no longer present in the {@link
 * org.apache.flink.runtime.dispatcher.Dispatcher dispatcher} (job has terminated and job
 * manager failed over), but we know that job has already terminated from {@link
 * org.apache.flink.runtime.highavailability.JobResultStore}.
 */
@Test
public void testDuplicateJobSubmissionWithTerminatedJobIdWithUnknownResultAttached() throws Throwable {
    final JobID testJobID = new JobID(0, 2);
    final Configuration configurationUnderTest = getConfiguration();
    configurationUnderTest.set(PipelineOptionsInternal.PIPELINE_FIXED_JOB_ID, testJobID.toHexString());
    configurationUnderTest.set(HighAvailabilityOptions.HA_MODE, HighAvailabilityMode.ZOOKEEPER.name());
    final TestingDispatcherGateway.Builder dispatcherBuilder = TestingDispatcherGateway.newBuilder().setSubmitFunction(jobGraph -> FutureUtils.completedExceptionally(DuplicateJobSubmissionException.ofGloballyTerminated(testJobID))).setRequestJobStatusFunction(jobId -> FutureUtils.completedExceptionally(new FlinkJobNotFoundException(jobId))).setRequestJobResultFunction(jobId -> FutureUtils.completedExceptionally(new FlinkJobNotFoundException(jobId)));
    final CompletableFuture<Void> applicationFuture = runApplication(dispatcherBuilder, configurationUnderTest, 1);
    applicationFuture.get(TIMEOUT_SECONDS, TimeUnit.SECONDS);
}
Also used : CoreMatchers.is(org.hamcrest.CoreMatchers.is) ProgramInvocationException(org.apache.flink.client.program.ProgramInvocationException) ScheduledFuture(java.util.concurrent.ScheduledFuture) ExceptionUtils(org.apache.flink.util.ExceptionUtils) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) Duration(java.time.Duration) Assertions(org.assertj.core.api.Assertions) FailingJob(org.apache.flink.client.testjar.FailingJob) ScheduledExecutor(org.apache.flink.util.concurrent.ScheduledExecutor) Acknowledge(org.apache.flink.runtime.messages.Acknowledge) Executors(java.util.concurrent.Executors) ExecutorUtils(org.apache.flink.util.ExecutorUtils) Test(org.junit.jupiter.api.Test) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) SerializedThrowable(org.apache.flink.util.SerializedThrowable) Optional(java.util.Optional) PackagedProgram(org.apache.flink.client.program.PackagedProgram) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) Assertions.fail(org.junit.jupiter.api.Assertions.fail) FlinkException(org.apache.flink.util.FlinkException) ScheduledExecutorServiceAdapter(org.apache.flink.util.concurrent.ScheduledExecutorServiceAdapter) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) EnumSource(org.junit.jupiter.params.provider.EnumSource) CompletableFuture(java.util.concurrent.CompletableFuture) JobStatus(org.apache.flink.api.common.JobStatus) DispatcherGateway(org.apache.flink.runtime.dispatcher.DispatcherGateway) Supplier(java.util.function.Supplier) EmbeddedExecutor(org.apache.flink.client.deployment.application.executors.EmbeddedExecutor) PipelineOptionsInternal(org.apache.flink.configuration.PipelineOptionsInternal) MultiExecuteJob(org.apache.flink.client.testjar.MultiExecuteJob) JobResult(org.apache.flink.runtime.jobmaster.JobResult) TestLoggerExtension(org.apache.flink.util.TestLoggerExtension) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) FatalErrorHandler(org.apache.flink.runtime.rpc.FatalErrorHandler) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) BiConsumer(java.util.function.BiConsumer) DeploymentOptions(org.apache.flink.configuration.DeploymentOptions) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) JobExecutionException(org.apache.flink.runtime.client.JobExecutionException) HighAvailabilityMode(org.apache.flink.runtime.jobmanager.HighAvailabilityMode) FlinkRuntimeException(org.apache.flink.util.FlinkRuntimeException) ApplicationStatus(org.apache.flink.runtime.clusterframework.ApplicationStatus) Configuration(org.apache.flink.configuration.Configuration) JobCancellationException(org.apache.flink.runtime.client.JobCancellationException) ConcurrentLinkedDeque(java.util.concurrent.ConcurrentLinkedDeque) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) AfterEach(org.junit.jupiter.api.AfterEach) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) JobID(org.apache.flink.api.common.JobID) TestingDispatcherGateway(org.apache.flink.runtime.webmonitor.TestingDispatcherGateway) Collections(java.util.Collections) HighAvailabilityOptions(org.apache.flink.configuration.HighAvailabilityOptions) DuplicateJobSubmissionException(org.apache.flink.runtime.client.DuplicateJobSubmissionException) TestingDispatcherGateway(org.apache.flink.runtime.webmonitor.TestingDispatcherGateway) Configuration(org.apache.flink.configuration.Configuration) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) JobID(org.apache.flink.api.common.JobID) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 8 with FlinkJobNotFoundException

use of org.apache.flink.runtime.messages.FlinkJobNotFoundException in project flink by apache.

the class JobMasterQueryableStateTest method testRequestKvStateOfWrongJob.

@Test
public void testRequestKvStateOfWrongJob() throws Exception {
    final JobMaster jobMaster = new JobMasterBuilder(JOB_GRAPH, rpcService).createJobMaster();
    jobMaster.start();
    final JobMasterGateway jobMasterGateway = jobMaster.getSelfGateway(JobMasterGateway.class);
    registerSlotsRequiredForJobExecution(jobMasterGateway, JOB_GRAPH.getJobID());
    try {
        // lookup location
        try {
            jobMasterGateway.requestKvStateLocation(new JobID(), "unknown").get();
            fail("Expected to fail with FlinkJobNotFoundException");
        } catch (Exception e) {
            assertThat(e, containsCause(FlinkJobNotFoundException.class));
        }
    } finally {
        RpcUtils.terminateRpcEndpoint(jobMaster, testingTimeout);
    }
}
Also used : JobMasterBuilder(org.apache.flink.runtime.jobmaster.utils.JobMasterBuilder) JobID(org.apache.flink.api.common.JobID) UnknownHostException(java.net.UnknownHostException) ExecutionException(java.util.concurrent.ExecutionException) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Test(org.junit.Test)

Example 9 with FlinkJobNotFoundException

use of org.apache.flink.runtime.messages.FlinkJobNotFoundException in project flink by apache.

the class Dispatcher method cancelJob.

@Override
public CompletableFuture<Acknowledge> cancelJob(JobID jobId, Time timeout) {
    Optional<JobManagerRunner> maybeJob = getJobManagerRunner(jobId);
    if (maybeJob.isPresent()) {
        return maybeJob.get().cancel(timeout);
    }
    final ExecutionGraphInfo executionGraphInfo = executionGraphInfoStore.get(jobId);
    if (executionGraphInfo != null) {
        final JobStatus jobStatus = executionGraphInfo.getArchivedExecutionGraph().getState();
        if (jobStatus == JobStatus.CANCELED) {
            return CompletableFuture.completedFuture(Acknowledge.get());
        } else {
            return FutureUtils.completedExceptionally(new FlinkJobTerminatedWithoutCancellationException(jobId, jobStatus));
        }
    }
    log.debug("Dispatcher is unable to cancel job {}: not found", jobId);
    return FutureUtils.completedExceptionally(new FlinkJobNotFoundException(jobId));
}
Also used : JobStatus(org.apache.flink.api.common.JobStatus) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) FlinkJobTerminatedWithoutCancellationException(org.apache.flink.runtime.messages.FlinkJobTerminatedWithoutCancellationException) JobManagerRunner(org.apache.flink.runtime.jobmaster.JobManagerRunner)

Example 10 with FlinkJobNotFoundException

use of org.apache.flink.runtime.messages.FlinkJobNotFoundException in project flink by apache.

the class AbstractExecutionGraphHandler method handleRequest.

@Override
protected CompletableFuture<R> handleRequest(@Nonnull HandlerRequest<EmptyRequestBody> request, @Nonnull RestfulGateway gateway) throws RestHandlerException {
    JobID jobId = request.getPathParameter(JobIDPathParameter.class);
    CompletableFuture<ExecutionGraphInfo> executionGraphFuture = executionGraphCache.getExecutionGraphInfo(jobId, gateway);
    return executionGraphFuture.thenApplyAsync(executionGraph -> {
        try {
            return handleRequest(request, executionGraph);
        } catch (RestHandlerException rhe) {
            throw new CompletionException(rhe);
        }
    }, executor).exceptionally(throwable -> {
        throwable = ExceptionUtils.stripCompletionException(throwable);
        if (throwable instanceof FlinkJobNotFoundException) {
            throw new CompletionException(new NotFoundException(String.format("Job %s not found", jobId), throwable));
        } else {
            throw new CompletionException(throwable);
        }
    });
}
Also used : ExecutionGraphCache(org.apache.flink.runtime.rest.handler.legacy.ExecutionGraphCache) Executor(java.util.concurrent.Executor) GatewayRetriever(org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever) RestfulGateway(org.apache.flink.runtime.webmonitor.RestfulGateway) ExceptionUtils(org.apache.flink.util.ExceptionUtils) CompletableFuture(java.util.concurrent.CompletableFuture) CompletionException(java.util.concurrent.CompletionException) MessageHeaders(org.apache.flink.runtime.rest.messages.MessageHeaders) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) Preconditions(org.apache.flink.util.Preconditions) NotFoundException(org.apache.flink.runtime.rest.NotFoundException) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) JobID(org.apache.flink.api.common.JobID) JobMessageParameters(org.apache.flink.runtime.rest.messages.JobMessageParameters) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Map(java.util.Map) HandlerRequest(org.apache.flink.runtime.rest.handler.HandlerRequest) Nonnull(javax.annotation.Nonnull) Time(org.apache.flink.api.common.time.Time) AbstractRestHandler(org.apache.flink.runtime.rest.handler.AbstractRestHandler) JobIDPathParameter(org.apache.flink.runtime.rest.messages.JobIDPathParameter) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) CompletionException(java.util.concurrent.CompletionException) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) NotFoundException(org.apache.flink.runtime.rest.NotFoundException) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) JobID(org.apache.flink.api.common.JobID) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException)

Aggregations

FlinkJobNotFoundException (org.apache.flink.runtime.messages.FlinkJobNotFoundException)12 JobID (org.apache.flink.api.common.JobID)9 CompletableFuture (java.util.concurrent.CompletableFuture)6 ExecutionException (java.util.concurrent.ExecutionException)6 JobStatus (org.apache.flink.api.common.JobStatus)5 ExceptionUtils (org.apache.flink.util.ExceptionUtils)5 Collections (java.util.Collections)4 Time (org.apache.flink.api.common.time.Time)4 JobResult (org.apache.flink.runtime.jobmaster.JobResult)4 FlinkException (org.apache.flink.util.FlinkException)4 FutureUtils (org.apache.flink.util.concurrent.FutureUtils)4 Optional (java.util.Optional)3 CompletionException (java.util.concurrent.CompletionException)3 Configuration (org.apache.flink.configuration.Configuration)3 DuplicateJobSubmissionException (org.apache.flink.runtime.client.DuplicateJobSubmissionException)3 ApplicationStatus (org.apache.flink.runtime.clusterframework.ApplicationStatus)3 Acknowledge (org.apache.flink.runtime.messages.Acknowledge)3 FlinkJobTerminatedWithoutCancellationException (org.apache.flink.runtime.messages.FlinkJobTerminatedWithoutCancellationException)3 ExecutionGraphInfo (org.apache.flink.runtime.scheduler.ExecutionGraphInfo)3 Test (org.junit.Test)3