Search in sources :

Example 6 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class MiniDispatcherTest method testShutdownIfJobCancelledInNormalMode.

@Test
public void testShutdownIfJobCancelledInNormalMode() throws Exception {
    final MiniDispatcher miniDispatcher = createMiniDispatcher(ClusterEntrypoint.ExecutionMode.NORMAL);
    miniDispatcher.start();
    try {
        // wait until we have submitted the job
        final TestingJobManagerRunner testingJobManagerRunner = testingJobManagerRunnerFactory.takeCreatedJobManagerRunner();
        assertFalse(miniDispatcher.getTerminationFuture().isDone());
        final DispatcherGateway dispatcherGateway = miniDispatcher.getSelfGateway(DispatcherGateway.class);
        dispatcherGateway.cancelJob(jobGraph.getJobID(), Time.seconds(10L));
        testingJobManagerRunner.completeResultFuture(new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setJobID(jobGraph.getJobID()).setState(JobStatus.CANCELED).build()));
        ApplicationStatus applicationStatus = miniDispatcher.getShutDownFuture().get();
        assertThat(applicationStatus, is(ApplicationStatus.CANCELED));
    } finally {
        RpcUtils.terminateRpcEndpoint(miniDispatcher, timeout);
    }
}
Also used : ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ApplicationStatus(org.apache.flink.runtime.clusterframework.ApplicationStatus) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) TestingJobManagerRunner(org.apache.flink.runtime.jobmaster.TestingJobManagerRunner) Test(org.junit.Test)

Example 7 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class MiniDispatcherTest method setupClass.

@BeforeClass
public static void setupClass() throws IOException {
    jobGraph = JobGraphTestUtils.singleNoOpJobGraph();
    executionGraphInfo = new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setJobID(jobGraph.getJobID()).setState(JobStatus.FINISHED).build());
    rpcService = new TestingRpcService();
    configuration = new Configuration();
    blobServer = new BlobServer(configuration, temporaryFolder.newFolder(), new VoidBlobStore());
}
Also used : VoidBlobStore(org.apache.flink.runtime.blob.VoidBlobStore) Configuration(org.apache.flink.configuration.Configuration) ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) TestingRpcService(org.apache.flink.runtime.rpc.TestingRpcService) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) BlobServer(org.apache.flink.runtime.blob.BlobServer) BeforeClass(org.junit.BeforeClass)

Example 8 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class MemoryExecutionGraphInfoStoreTest method testCloseCleansUp.

/**
 * Tests that all job graphs are cleaned up after closing the store.
 */
@Test
public void testCloseCleansUp() throws IOException {
    try (final MemoryExecutionGraphInfoStore executionGraphInfoStore = createMemoryExecutionGraphInfoStore()) {
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(0));
        executionGraphInfoStore.put(new ExecutionGraphInfo(new ArchivedExecutionGraphBuilder().setState(JobStatus.FINISHED).build()));
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(1));
        executionGraphInfoStore.close();
        assertThat(executionGraphInfoStore.size(), Matchers.equalTo(0));
    }
}
Also used : ExecutionGraphInfo(org.apache.flink.runtime.scheduler.ExecutionGraphInfo) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Test(org.junit.Test)

Example 9 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class JobConfigHandlerTest method handleRequest_executionConfigWithSecretValues_excludesSecretValuesFromResponse.

@Test
public void handleRequest_executionConfigWithSecretValues_excludesSecretValuesFromResponse() throws HandlerRequestException {
    final JobConfigHandler jobConfigHandler = new JobConfigHandler(() -> null, TestingUtils.TIMEOUT, Collections.emptyMap(), JobConfigHeaders.getInstance(), new DefaultExecutionGraphCache(TestingUtils.TIMEOUT, TestingUtils.TIMEOUT), TestingUtils.defaultExecutor());
    final Map<String, String> globalJobParameters = new HashMap<>();
    globalJobParameters.put("foobar", "barfoo");
    globalJobParameters.put("bar.secret.foo", "my secret");
    globalJobParameters.put("password.to.my.safe", "12345");
    final ArchivedExecutionConfig archivedExecutionConfig = new ArchivedExecutionConfigBuilder().setGlobalJobParameters(globalJobParameters).build();
    final AccessExecutionGraph archivedExecutionGraph = new ArchivedExecutionGraphBuilder().setArchivedExecutionConfig(archivedExecutionConfig).build();
    final HandlerRequest<EmptyRequestBody> handlerRequest = createRequest(archivedExecutionGraph.getJobID());
    final JobConfigInfo jobConfigInfoResponse = jobConfigHandler.handleRequest(handlerRequest, archivedExecutionGraph);
    final Map<String, String> filteredGlobalJobParameters = filterSecretValues(globalJobParameters);
    assertThat(jobConfigInfoResponse.getExecutionConfigInfo().getGlobalJobParameters(), is(equalTo(filteredGlobalJobParameters)));
}
Also used : JobConfigInfo(org.apache.flink.runtime.rest.messages.JobConfigInfo) ArchivedExecutionConfig(org.apache.flink.api.common.ArchivedExecutionConfig) HashMap(java.util.HashMap) DefaultExecutionGraphCache(org.apache.flink.runtime.rest.handler.legacy.DefaultExecutionGraphCache) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) ArchivedExecutionConfigBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionConfigBuilder) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) Test(org.junit.Test)

Example 10 with ArchivedExecutionGraphBuilder

use of org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder in project flink by apache.

the class JobExecutionResultHandlerTest method testCompletedResult.

@Test
public void testCompletedResult() throws Exception {
    final JobStatus jobStatus = JobStatus.FINISHED;
    final ArchivedExecutionGraph executionGraph = new ArchivedExecutionGraphBuilder().setJobID(TEST_JOB_ID).setState(jobStatus).build();
    final TestingRestfulGateway testingRestfulGateway = new TestingRestfulGateway.Builder().setRequestJobStatusFunction(jobId -> {
        assertThat(jobId, equalTo(TEST_JOB_ID));
        return CompletableFuture.completedFuture(jobStatus);
    }).setRequestJobResultFunction(jobId -> {
        assertThat(jobId, equalTo(TEST_JOB_ID));
        return CompletableFuture.completedFuture(JobResult.createFrom(executionGraph));
    }).build();
    final JobExecutionResultResponseBody responseBody = jobExecutionResultHandler.handleRequest(testRequest, testingRestfulGateway).get();
    assertThat(responseBody.getStatus().getId(), equalTo(QueueStatus.Id.COMPLETED));
    assertThat(responseBody.getJobExecutionResult(), not(nullValue()));
}
Also used : JobStatus(org.apache.flink.api.common.JobStatus) QueueStatus(org.apache.flink.runtime.rest.messages.queue.QueueStatus) Matchers.not(org.hamcrest.Matchers.not) ExceptionUtils(org.apache.flink.util.ExceptionUtils) CompletableFuture(java.util.concurrent.CompletableFuture) JobStatus(org.apache.flink.api.common.JobStatus) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) HttpResponseStatus(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) Assert.assertThat(org.junit.Assert.assertThat) JobResult(org.apache.flink.runtime.jobmaster.JobResult) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) Matchers.nullValue(org.hamcrest.Matchers.nullValue) TestLogger(org.apache.flink.util.TestLogger) HandlerRequest(org.apache.flink.runtime.rest.handler.HandlerRequest) Assert.fail(org.junit.Assert.fail) Before(org.junit.Before) ArchivedExecutionGraph(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) Test(org.junit.Test) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) ExecutionException(java.util.concurrent.ExecutionException) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) JobID(org.apache.flink.api.common.JobID) JobMessageParameters(org.apache.flink.runtime.rest.messages.JobMessageParameters) FlinkJobNotFoundException(org.apache.flink.runtime.messages.FlinkJobNotFoundException) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Collections(java.util.Collections) Time(org.apache.flink.api.common.time.Time) JobExecutionResultResponseBody(org.apache.flink.runtime.rest.messages.job.JobExecutionResultResponseBody) JobExecutionResultResponseBody(org.apache.flink.runtime.rest.messages.job.JobExecutionResultResponseBody) ArchivedExecutionGraph(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph) ArchivedExecutionGraphBuilder(org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) Test(org.junit.Test)

Aggregations

ArchivedExecutionGraphBuilder (org.apache.flink.runtime.rest.handler.legacy.utils.ArchivedExecutionGraphBuilder)29 ExecutionGraphInfo (org.apache.flink.runtime.scheduler.ExecutionGraphInfo)23 Test (org.junit.Test)22 JobID (org.apache.flink.api.common.JobID)12 CompletableFuture (java.util.concurrent.CompletableFuture)8 Time (org.apache.flink.api.common.time.Time)6 ErrorInfo (org.apache.flink.runtime.executiongraph.ErrorInfo)6 FlinkException (org.apache.flink.util.FlinkException)6 JobStatus (org.apache.flink.api.common.JobStatus)5 TestingJobManagerRunner (org.apache.flink.runtime.jobmaster.TestingJobManagerRunner)5 File (java.io.File)4 ArrayList (java.util.ArrayList)4 ExecutionException (java.util.concurrent.ExecutionException)4 ArchivedExecutionGraph (org.apache.flink.runtime.executiongraph.ArchivedExecutionGraph)4 ArrayDeque (java.util.ArrayDeque)3 Arrays (java.util.Arrays)3 TimeUnit (java.util.concurrent.TimeUnit)3 Configuration (org.apache.flink.configuration.Configuration)3 BlobServer (org.apache.flink.runtime.blob.BlobServer)3 IOException (java.io.IOException)2