Search in sources :

Example 6 with RestHandlerException

use of org.apache.flink.runtime.rest.handler.RestHandlerException in project flink by apache.

the class JobSubmitHandler method handleRequest.

@Override
protected CompletableFuture<JobSubmitResponseBody> handleRequest(@Nonnull HandlerRequest<JobSubmitRequestBody> request, @Nonnull DispatcherGateway gateway) throws RestHandlerException {
    final Collection<File> uploadedFiles = request.getUploadedFiles();
    final Map<String, Path> nameToFile = uploadedFiles.stream().collect(Collectors.toMap(File::getName, Path::fromLocalFile));
    if (uploadedFiles.size() != nameToFile.size()) {
        throw new RestHandlerException(String.format("The number of uploaded files was %s than the expected count. Expected: %s Actual %s", uploadedFiles.size() < nameToFile.size() ? "lower" : "higher", nameToFile.size(), uploadedFiles.size()), HttpResponseStatus.BAD_REQUEST);
    }
    final JobSubmitRequestBody requestBody = request.getRequestBody();
    if (requestBody.jobGraphFileName == null) {
        throw new RestHandlerException(String.format("The %s field must not be omitted or be null.", JobSubmitRequestBody.FIELD_NAME_JOB_GRAPH), HttpResponseStatus.BAD_REQUEST);
    }
    CompletableFuture<JobGraph> jobGraphFuture = loadJobGraph(requestBody, nameToFile);
    Collection<Path> jarFiles = getJarFilesToUpload(requestBody.jarFileNames, nameToFile);
    Collection<Tuple2<String, Path>> artifacts = getArtifactFilesToUpload(requestBody.artifactFileNames, nameToFile);
    CompletableFuture<JobGraph> finalizedJobGraphFuture = uploadJobGraphFiles(gateway, jobGraphFuture, jarFiles, artifacts, configuration);
    CompletableFuture<Acknowledge> jobSubmissionFuture = finalizedJobGraphFuture.thenCompose(jobGraph -> gateway.submitJob(jobGraph, timeout));
    return jobSubmissionFuture.thenCombine(jobGraphFuture, (ack, jobGraph) -> new JobSubmitResponseBody("/jobs/" + jobGraph.getJobID()));
}
Also used : Path(org.apache.flink.core.fs.Path) Acknowledge(org.apache.flink.runtime.messages.Acknowledge) JobSubmitResponseBody(org.apache.flink.runtime.rest.messages.job.JobSubmitResponseBody) JobSubmitRequestBody(org.apache.flink.runtime.rest.messages.job.JobSubmitRequestBody) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) Tuple2(org.apache.flink.api.java.tuple.Tuple2) File(java.io.File)

Example 7 with RestHandlerException

use of org.apache.flink.runtime.rest.handler.RestHandlerException in project flink by apache.

the class AbstractCheckpointHandler method handleRequest.

@Override
protected R handleRequest(HandlerRequest<EmptyRequestBody> request, AccessExecutionGraph executionGraph) throws RestHandlerException {
    final long checkpointId = request.getPathParameter(CheckpointIdPathParameter.class);
    final CheckpointStatsSnapshot checkpointStatsSnapshot = executionGraph.getCheckpointStatsSnapshot();
    if (checkpointStatsSnapshot != null) {
        AbstractCheckpointStats checkpointStats = checkpointStatsSnapshot.getHistory().getCheckpointById(checkpointId);
        if (checkpointStats != null) {
            checkpointStatsCache.tryAdd(checkpointStats);
        } else {
            checkpointStats = checkpointStatsCache.tryGet(checkpointId);
        }
        if (checkpointStats != null) {
            return handleCheckpointRequest(request, checkpointStats);
        } else {
            throw new RestHandlerException("Could not find checkpointing statistics for checkpoint " + checkpointId + '.', HttpResponseStatus.NOT_FOUND);
        }
    } else {
        throw new RestHandlerException("Checkpointing was not enabled for job " + executionGraph.getJobID() + '.', HttpResponseStatus.NOT_FOUND);
    }
}
Also used : AbstractCheckpointStats(org.apache.flink.runtime.checkpoint.AbstractCheckpointStats) CheckpointStatsSnapshot(org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException)

Example 8 with RestHandlerException

use of org.apache.flink.runtime.rest.handler.RestHandlerException in project flink by apache.

the class CheckpointingStatisticsHandler method archiveJsonWithPath.

@Override
public Collection<ArchivedJson> archiveJsonWithPath(AccessExecutionGraph graph) throws IOException {
    ResponseBody json;
    try {
        json = createCheckpointingStatistics(graph);
    } catch (RestHandlerException rhe) {
        json = new ErrorResponseBody(rhe.getMessage());
    }
    String path = getMessageHeaders().getTargetRestEndpointURL().replace(':' + JobIDPathParameter.KEY, graph.getJobID().toString());
    return Collections.singletonList(new ArchivedJson(path, json));
}
Also used : ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody)

Example 9 with RestHandlerException

use of org.apache.flink.runtime.rest.handler.RestHandlerException in project flink by apache.

the class JobManagerLogListHandler method handleRequest.

@Override
protected CompletableFuture<LogListInfo> handleRequest(@Nonnull HandlerRequest<EmptyRequestBody> request, @Nonnull RestfulGateway gateway) throws RestHandlerException {
    if (logDir == null) {
        return CompletableFuture.completedFuture(new LogListInfo(Collections.emptyList()));
    }
    final File[] logFiles = logDir.listFiles();
    if (logFiles == null) {
        return FutureUtils.completedExceptionally(new IOException("Could not list files in " + logDir));
    }
    final List<LogInfo> logs = Arrays.stream(logFiles).filter(File::isFile).map(logFile -> new LogInfo(logFile.getName(), logFile.length(), logFile.lastModified())).collect(Collectors.toList());
    return CompletableFuture.completedFuture(new LogListInfo(logs));
}
Also used : LogListInfo(org.apache.flink.runtime.rest.messages.LogListInfo) Arrays(java.util.Arrays) GatewayRetriever(org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever) RestfulGateway(org.apache.flink.runtime.webmonitor.RestfulGateway) IOException(java.io.IOException) CompletableFuture(java.util.concurrent.CompletableFuture) LogInfo(org.apache.flink.runtime.rest.messages.LogInfo) MessageHeaders(org.apache.flink.runtime.rest.messages.MessageHeaders) Collectors(java.util.stream.Collectors) File(java.io.File) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) EmptyMessageParameters(org.apache.flink.runtime.rest.messages.EmptyMessageParameters) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) List(java.util.List) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) Map(java.util.Map) LogListInfo(org.apache.flink.runtime.rest.messages.LogListInfo) HandlerRequest(org.apache.flink.runtime.rest.handler.HandlerRequest) Nonnull(javax.annotation.Nonnull) Collections(java.util.Collections) Time(org.apache.flink.api.common.time.Time) AbstractRestHandler(org.apache.flink.runtime.rest.handler.AbstractRestHandler) Nullable(javax.annotation.Nullable) LogInfo(org.apache.flink.runtime.rest.messages.LogInfo) IOException(java.io.IOException) File(java.io.File)

Example 10 with RestHandlerException

use of org.apache.flink.runtime.rest.handler.RestHandlerException in project flink by apache.

the class RestClusterClientSavepointTriggerTest method testTriggerSavepointRetry.

@Test
public void testTriggerSavepointRetry() throws Exception {
    final TriggerId triggerId = new TriggerId();
    final String expectedSavepointDir = "hello";
    final AtomicBoolean failRequest = new AtomicBoolean(true);
    try (final RestServerEndpoint restServerEndpoint = createRestServerEndpoint(request -> triggerId, trigger -> {
        if (failRequest.compareAndSet(true, false)) {
            throw new RestHandlerException("expected", HttpResponseStatus.SERVICE_UNAVAILABLE);
        } else {
            return new SavepointInfo(expectedSavepointDir, null);
        }
    })) {
        final RestClusterClient<?> restClusterClient = createRestClusterClient(restServerEndpoint.getServerAddress().getPort());
        final String savepointPath = restClusterClient.triggerSavepoint(new JobID(), null, SavepointFormatType.CANONICAL).get();
        assertEquals(expectedSavepointDir, savepointPath);
    }
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) TriggerId(org.apache.flink.runtime.rest.messages.TriggerId) TestRestServerEndpoint(org.apache.flink.runtime.rest.util.TestRestServerEndpoint) RestServerEndpoint(org.apache.flink.runtime.rest.RestServerEndpoint) SavepointInfo(org.apache.flink.runtime.rest.messages.job.savepoints.SavepointInfo) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Aggregations

RestHandlerException (org.apache.flink.runtime.rest.handler.RestHandlerException)39 Test (org.junit.Test)13 IOException (java.io.IOException)11 CompletionException (java.util.concurrent.CompletionException)11 EmptyRequestBody (org.apache.flink.runtime.rest.messages.EmptyRequestBody)9 CompletableFuture (java.util.concurrent.CompletableFuture)8 HandlerRequest (org.apache.flink.runtime.rest.handler.HandlerRequest)8 File (java.io.File)7 ExecutionException (java.util.concurrent.ExecutionException)7 JobID (org.apache.flink.api.common.JobID)7 Time (org.apache.flink.api.common.time.Time)6 RestfulGateway (org.apache.flink.runtime.webmonitor.RestfulGateway)6 Path (java.nio.file.Path)5 ArrayList (java.util.ArrayList)5 Collections (java.util.Collections)5 Map (java.util.Map)5 TestingRestfulGateway (org.apache.flink.runtime.webmonitor.TestingRestfulGateway)5 HttpResponseStatus (org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus)5 ExceptionUtils (org.apache.flink.util.ExceptionUtils)5 FutureUtils (org.apache.flink.util.concurrent.FutureUtils)5