Search in sources :

Example 6 with ResponseBody

use of org.apache.flink.runtime.rest.messages.ResponseBody in project flink by apache.

the class RestClient method submitRequest.

private <P extends ResponseBody> CompletableFuture<P> submitRequest(String targetAddress, int targetPort, Request httpRequest, JavaType responseType) {
    final ChannelFuture connectFuture = bootstrap.connect(targetAddress, targetPort);
    final CompletableFuture<Channel> channelFuture = new CompletableFuture<>();
    connectFuture.addListener((ChannelFuture future) -> {
        if (future.isSuccess()) {
            channelFuture.complete(future.channel());
        } else {
            channelFuture.completeExceptionally(future.cause());
        }
    });
    return channelFuture.thenComposeAsync(channel -> {
        ClientHandler handler = channel.pipeline().get(ClientHandler.class);
        CompletableFuture<JsonResponse> future;
        boolean success = false;
        try {
            if (handler == null) {
                throw new IOException("Netty pipeline was not properly initialized.");
            } else {
                httpRequest.writeTo(channel);
                future = handler.getJsonFuture();
                success = true;
            }
        } catch (IOException e) {
            future = FutureUtils.completedExceptionally(new ConnectionException("Could not write request.", e));
        } finally {
            if (!success) {
                channel.close();
            }
        }
        return future;
    }, executor).thenComposeAsync((JsonResponse rawResponse) -> parseResponse(rawResponse, responseType), executor);
}
Also used : ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) SimpleChannelInboundHandler(org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler) MessageParameters(org.apache.flink.runtime.rest.messages.MessageParameters) FullHttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse) ChunkedWriteHandler(org.apache.flink.shaded.netty4.io.netty.handler.stream.ChunkedWriteHandler) OutboundChannelHandlerFactory(org.apache.flink.runtime.io.network.netty.OutboundChannelHandlerFactory) MemoryAttribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.MemoryAttribute) LoggerFactory(org.slf4j.LoggerFactory) ExceptionUtils(org.apache.flink.util.ExceptionUtils) JsonParser(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser) JsonNode(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonNode) NetUtils(org.apache.flink.util.NetUtils) EmptyMessageParameters(org.apache.flink.runtime.rest.messages.EmptyMessageParameters) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) HttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse) RequestBody(org.apache.flink.runtime.rest.messages.RequestBody) HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) RestClientException(org.apache.flink.runtime.rest.util.RestClientException) Path(java.nio.file.Path) ChannelHandlerContext(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) Attribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.Attribute) ExecutorThreadFactory(org.apache.flink.util.concurrent.ExecutorThreadFactory) RestConstants(org.apache.flink.runtime.rest.util.RestConstants) Collection(java.util.Collection) NioEventLoopGroup(org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoopGroup) HttpVersion(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion) HttpObjectAggregator(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpObjectAggregator) IdleStateHandler(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateHandler) ServiceLoader(java.util.ServiceLoader) JavaType(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JavaType) Preconditions(org.apache.flink.util.Preconditions) ByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf) Collectors(java.util.stream.Collectors) List(java.util.List) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) HttpClientCodec(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpClientCodec) DefaultHttpDataFactory(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.DefaultHttpDataFactory) Optional(java.util.Optional) IdleStateEvent(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateEvent) Time(org.apache.flink.api.common.time.Time) RestMapperUtils(org.apache.flink.runtime.rest.util.RestMapperUtils) Bootstrap(org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap) ChannelInitializer(org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer) ConfigurationException(org.apache.flink.util.ConfigurationException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SSLHandlerFactory(org.apache.flink.runtime.io.network.netty.SSLHandlerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) ChannelHandler(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler) MessageHeaders(org.apache.flink.runtime.rest.messages.MessageHeaders) HttpResponseStatus(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus) ArrayList(java.util.ArrayList) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) ByteBufInputStream(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBufInputStream) HttpHeaders(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaders) REQUEST_ENTITY_TOO_LARGE(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus.REQUEST_ENTITY_TOO_LARGE) ObjectMapper(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper) ConfigConstants(org.apache.flink.configuration.ConfigConstants) RestOptions(org.apache.flink.configuration.RestOptions) RestAPIVersion(org.apache.flink.runtime.rest.versioning.RestAPIVersion) ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) ChannelOption(org.apache.flink.shaded.netty4.io.netty.channel.ChannelOption) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) AutoCloseableAsync(org.apache.flink.util.AutoCloseableAsync) Files(java.nio.file.Files) Executor(java.util.concurrent.Executor) StringWriter(java.io.StringWriter) Configuration(org.apache.flink.configuration.Configuration) Unpooled(org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled) IOException(java.io.IOException) JsonProcessingException(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException) File(java.io.File) VisibleForTesting(org.apache.flink.annotation.VisibleForTesting) TimeUnit(java.util.concurrent.TimeUnit) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody) TooLongFrameException(org.apache.flink.shaded.netty4.io.netty.handler.codec.TooLongFrameException) HttpPostRequestEncoder(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.HttpPostRequestEncoder) HttpMethod(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod) Comparator(java.util.Comparator) Collections(java.util.Collections) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) InputStream(java.io.InputStream) CompletableFuture(java.util.concurrent.CompletableFuture) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) IOException(java.io.IOException)

Example 7 with ResponseBody

use of org.apache.flink.runtime.rest.messages.ResponseBody in project flink by apache.

the class CheckpointConfigHandler method archiveJsonWithPath.

@Override
public Collection<ArchivedJson> archiveJsonWithPath(AccessExecutionGraph graph) throws IOException {
    ResponseBody response;
    try {
        response = createCheckpointConfigInfo(graph);
    } catch (RestHandlerException rhe) {
        response = new ErrorResponseBody(rhe.getMessage());
    }
    String path = CheckpointConfigHeaders.getInstance().getTargetRestEndpointURL().replace(':' + JobIDPathParameter.KEY, graph.getJobID().toString());
    return Collections.singletonList(new ArchivedJson(path, response));
}
Also used : ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) RestHandlerException(org.apache.flink.runtime.rest.handler.RestHandlerException) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody)

Example 8 with ResponseBody

use of org.apache.flink.runtime.rest.messages.ResponseBody in project flink by apache.

the class JobAccumulatorsHandler method archiveJsonWithPath.

@Override
public Collection<ArchivedJson> archiveJsonWithPath(AccessExecutionGraph graph) throws IOException {
    ResponseBody json = createJobAccumulatorsInfo(graph, true);
    String path = getMessageHeaders().getTargetRestEndpointURL().replace(':' + JobIDPathParameter.KEY, graph.getJobID().toString());
    return Collections.singleton(new ArchivedJson(path, json));
}
Also used : ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody)

Example 9 with ResponseBody

use of org.apache.flink.runtime.rest.messages.ResponseBody in project flink by apache.

the class JobConfigHandler method archiveJsonWithPath.

@Override
public Collection<ArchivedJson> archiveJsonWithPath(AccessExecutionGraph graph) throws IOException {
    ResponseBody json = createJobConfigInfo(graph);
    String path = getMessageHeaders().getTargetRestEndpointURL().replace(':' + JobIDPathParameter.KEY, graph.getJobID().toString());
    return Collections.singleton(new ArchivedJson(path, json));
}
Also used : ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody)

Example 10 with ResponseBody

use of org.apache.flink.runtime.rest.messages.ResponseBody in project flink by apache.

the class JobExceptionsHandler method archiveJsonWithPath.

@Override
public Collection<ArchivedJson> archiveJsonWithPath(ExecutionGraphInfo executionGraphInfo) throws IOException {
    ResponseBody json = createJobExceptionsInfo(executionGraphInfo, MAX_NUMBER_EXCEPTION_TO_REPORT);
    String path = getMessageHeaders().getTargetRestEndpointURL().replace(':' + JobIDPathParameter.KEY, executionGraphInfo.getJobId().toString());
    return Collections.singletonList(new ArchivedJson(path, json));
}
Also used : ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody)

Aggregations

ResponseBody (org.apache.flink.runtime.rest.messages.ResponseBody)16 ArchivedJson (org.apache.flink.runtime.webmonitor.history.ArchivedJson)15 ArrayList (java.util.ArrayList)8 AccessExecutionJobVertex (org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex)5 ErrorResponseBody (org.apache.flink.runtime.rest.messages.ErrorResponseBody)3 AbstractCheckpointStats (org.apache.flink.runtime.checkpoint.AbstractCheckpointStats)2 CheckpointStatsHistory (org.apache.flink.runtime.checkpoint.CheckpointStatsHistory)2 CheckpointStatsSnapshot (org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot)2 AccessExecution (org.apache.flink.runtime.executiongraph.AccessExecution)2 AccessExecutionVertex (org.apache.flink.runtime.executiongraph.AccessExecutionVertex)2 RestHandlerException (org.apache.flink.runtime.rest.handler.RestHandlerException)2 File (java.io.File)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 StringWriter (java.io.StringWriter)1 Files (java.nio.file.Files)1 Path (java.nio.file.Path)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1 Comparator (java.util.Comparator)1