Search in sources :

Example 11 with VertxCompletableFuture

use of io.confluent.ksql.util.VertxCompletableFuture in project ksql by confluentinc.

the class PullQueryRunner method run.

@Override
protected void run(long runMs) throws Exception {
    Semaphore sem = new Semaphore(MAX_CONCURRENT_REQUESTS);
    long start = System.currentTimeMillis();
    do {
        sem.acquire();
        VertxCompletableFuture<HttpResponse<Buffer>> vcf = new VertxCompletableFuture<>();
        client.post(8089, "localhost", "/query-stream").sendJsonObject(DEFAULT_PULL_QUERY_REQUEST_BODY, vcf);
        vcf.thenAccept(resp -> {
            count();
            sem.release();
        });
    } while (System.currentTimeMillis() - start < runMs);
}
Also used : HttpResponse(io.vertx.ext.web.client.HttpResponse) Semaphore(java.util.concurrent.Semaphore) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture)

Example 12 with VertxCompletableFuture

use of io.confluent.ksql.util.VertxCompletableFuture in project ksql by confluentinc.

the class TlsTest method shouldFailToUseDisabledTlsVersion.

@Test
public void shouldFailToUseDisabledTlsVersion() {
    // Given
    WebClientOptions clientOptions = createClientOptions().setEnabledSecureTransportProtocols(Collections.singleton("TLSv1.1"));
    WebClient client = WebClient.create(vertx, clientOptions);
    // When
    JsonObject requestBody = new JsonObject().put("ksql", "show streams;");
    VertxCompletableFuture<HttpResponse<Buffer>> requestFuture = new VertxCompletableFuture<>();
    client.post("/ksql").sendBuffer(requestBody.toBuffer(), requestFuture);
    // Then
    try {
        requestFuture.get();
    } catch (Exception e) {
        assertThat(e, // thrown from CompletableFuture.get()
        instanceOf(ExecutionException.class));
        assertThat(e.getMessage(), containsString("javax.net.ssl.SSLHandshakeException: Failed to create SSL connection"));
    }
}
Also used : WebClientOptions(io.vertx.ext.web.client.WebClientOptions) JsonObject(io.vertx.core.json.JsonObject) HttpResponse(io.vertx.ext.web.client.HttpResponse) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) WebClient(io.vertx.ext.web.client.WebClient) ExecutionException(java.util.concurrent.ExecutionException) Test(org.junit.Test)

Example 13 with VertxCompletableFuture

use of io.confluent.ksql.util.VertxCompletableFuture in project ksql by confluentinc.

the class KsqlTarget method execute.

private CompletableFuture<ResponseWithBody> execute(final HttpMethod httpMethod, final String path, final Optional<String> mediaType, final Object requestBody, final BiConsumer<HttpClientResponse, CompletableFuture<ResponseWithBody>> responseHandler) {
    final VertxCompletableFuture<ResponseWithBody> vcf = new VertxCompletableFuture<>();
    final HttpClientRequest httpClientRequest = httpClient.request(httpMethod, socketAddress, socketAddress.port(), host, path, resp -> responseHandler.accept(resp, vcf)).exceptionHandler(vcf::completeExceptionally);
    if (mediaType.isPresent()) {
        httpClientRequest.putHeader("Accept", mediaType.get());
    } else {
        httpClientRequest.putHeader("Accept", "application/json");
    }
    authHeader.ifPresent(v -> httpClientRequest.putHeader("Authorization", v));
    additionalHeaders.forEach(httpClientRequest::putHeader);
    if (requestBody != null) {
        httpClientRequest.end(serialize(requestBody));
    } else {
        httpClientRequest.end();
    }
    return vcf;
}
Also used : QueryStreamArgs(io.confluent.ksql.rest.entity.QueryStreamArgs) KsqlClientUtil.deserialize(io.confluent.ksql.rest.client.KsqlClientUtil.deserialize) LagReportingResponse(io.confluent.ksql.rest.entity.LagReportingResponse) StreamedRow(io.confluent.ksql.rest.entity.StreamedRow) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) LoggerFactory(org.slf4j.LoggerFactory) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) HeartbeatMessage(io.confluent.ksql.rest.entity.HeartbeatMessage) KsqlEntityList(io.confluent.ksql.rest.entity.KsqlEntityList) CompletableFuture(java.util.concurrent.CompletableFuture) ClusterStatusResponse(io.confluent.ksql.rest.entity.ClusterStatusResponse) CommandStatuses(io.confluent.ksql.rest.entity.CommandStatuses) HeartbeatResponse(io.confluent.ksql.rest.entity.HeartbeatResponse) Context(io.vertx.core.Context) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ServerClusterId(io.confluent.ksql.rest.entity.ServerClusterId) HttpClientRequest(io.vertx.core.http.HttpClientRequest) HttpClientResponse(io.vertx.core.http.HttpClientResponse) KsqlHostInfoEntity(io.confluent.ksql.rest.entity.KsqlHostInfoEntity) ServerInfo(io.confluent.ksql.rest.entity.ServerInfo) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Objects.requireNonNull(java.util.Objects.requireNonNull) Map(java.util.Map) BiConsumer(java.util.function.BiConsumer) RecordParser(io.vertx.core.parsetools.RecordParser) LagReportingMessage(io.confluent.ksql.rest.entity.LagReportingMessage) SocketAddress(io.vertx.core.net.SocketAddress) LocalProperties(io.confluent.ksql.properties.LocalProperties) CommandStatus(io.confluent.ksql.rest.entity.CommandStatus) Logger(org.slf4j.Logger) Vertx(io.vertx.core.Vertx) Collectors(java.util.stream.Collectors) ServerMetadata(io.confluent.ksql.rest.entity.ServerMetadata) Consumer(java.util.function.Consumer) List(java.util.List) KsqlRestClientException(io.confluent.ksql.rest.client.exception.KsqlRestClientException) Buffer(io.vertx.core.buffer.Buffer) HttpMethod(io.vertx.core.http.HttpMethod) HealthCheckResponse(io.confluent.ksql.rest.entity.HealthCheckResponse) Optional(java.util.Optional) KsqlClientUtil.serialize(io.confluent.ksql.rest.client.KsqlClientUtil.serialize) KsqlRequest(io.confluent.ksql.rest.entity.KsqlRequest) Collections(java.util.Collections) HttpClient(io.vertx.core.http.HttpClient) HttpClientRequest(io.vertx.core.http.HttpClientRequest) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture)

Example 14 with VertxCompletableFuture

use of io.confluent.ksql.util.VertxCompletableFuture in project ksql by confluentinc.

the class Server method start.

public synchronized void start() {
    if (!deploymentIds.isEmpty()) {
        throw new IllegalStateException("Already started");
    }
    final int idleConnectionTimeoutSeconds = config.getInt(KsqlRestConfig.IDLE_CONNECTION_TIMEOUT_SECONDS);
    this.workerExecutor = vertx.createSharedWorkerExecutor("ksql-workers", config.getInt(KsqlRestConfig.WORKER_POOL_SIZE));
    final LoggingRateLimiter loggingRateLimiter = new LoggingRateLimiter(config);
    configureTlsCertReload(config);
    final List<URI> listenUris = parseListeners(config);
    final Optional<URI> internalListenUri = parseInternalListener(config, listenUris);
    final List<URI> allListenUris = new ArrayList<>(listenUris);
    internalListenUri.ifPresent(allListenUris::add);
    final int instances = config.getInt(KsqlRestConfig.VERTICLE_INSTANCES);
    log.debug("Deploying " + instances + " instances of server verticle");
    final List<CompletableFuture<String>> deployFutures = new ArrayList<>();
    final Map<URI, URI> uris = new ConcurrentHashMap<>();
    for (URI listener : allListenUris) {
        final Optional<Boolean> isInternalListener = internalListenUri.map(uri -> uri.equals(listener));
        for (int i = 0; i < instances; i++) {
            final VertxCompletableFuture<String> vcf = new VertxCompletableFuture<>();
            final ServerVerticle serverVerticle = new ServerVerticle(endpoints, createHttpServerOptions(config, listener.getHost(), listener.getPort(), listener.getScheme().equalsIgnoreCase("https"), isInternalListener.orElse(false), idleConnectionTimeoutSeconds), this, isInternalListener, loggingRateLimiter);
            vertx.deployVerticle(serverVerticle, vcf);
            final int index = i;
            final CompletableFuture<String> deployFuture = vcf.thenApply(s -> {
                if (index == 0) {
                    try {
                        final URI uriWithPort = new URI(listener.getScheme(), null, listener.getHost(), serverVerticle.actualPort(), null, null, null);
                        uris.put(listener, uriWithPort);
                    } catch (URISyntaxException e) {
                        throw new KsqlException(e);
                    }
                }
                return s;
            });
            deployFutures.add(deployFuture);
        }
    }
    final CompletableFuture<Void> allDeployFuture = CompletableFuture.allOf(deployFutures.toArray(new CompletableFuture<?>[0]));
    try {
        allDeployFuture.get();
        for (CompletableFuture<String> deployFuture : deployFutures) {
            deploymentIds.add(deployFuture.get());
        }
    } catch (Exception e) {
        throw new KsqlException("Failed to start API server", e);
    }
    for (URI uri : listenUris) {
        listeners.add(uris.get(uri));
    }
    if (internalListenUri.isPresent()) {
        internalListener = uris.get(internalListenUri.get());
    }
    log.info("API server started");
}
Also used : ArrayList(java.util.ArrayList) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) URISyntaxException(java.net.URISyntaxException) KsqlException(io.confluent.ksql.util.KsqlException) URI(java.net.URI) URISyntaxException(java.net.URISyntaxException) ConfigException(org.apache.kafka.common.config.ConfigException) KsqlException(io.confluent.ksql.util.KsqlException) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) CompletableFuture(java.util.concurrent.CompletableFuture) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap)

Example 15 with VertxCompletableFuture

use of io.confluent.ksql.util.VertxCompletableFuture in project ksql by confluentinc.

the class Server method stop.

public synchronized void stop() {
    if (deploymentIds.isEmpty()) {
        throw new IllegalStateException("Not started");
    }
    if (workerExecutor != null) {
        workerExecutor.close();
    }
    if (fileWatcher != null) {
        fileWatcher.shutdown();
    }
    final List<CompletableFuture<Void>> undeployFutures = new ArrayList<>();
    for (String deploymentID : deploymentIds) {
        final VertxCompletableFuture<Void> future = new VertxCompletableFuture<>();
        vertx.undeploy(deploymentID, future);
        undeployFutures.add(future);
    }
    try {
        CompletableFuture.allOf(undeployFutures.toArray(new CompletableFuture<?>[0])).get();
    } catch (Exception e) {
        throw new KsqlException("Failure in stopping API server", e);
    }
    deploymentIds.clear();
    listeners.clear();
    log.info("API server stopped");
}
Also used : VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) CompletableFuture(java.util.concurrent.CompletableFuture) ArrayList(java.util.ArrayList) VertxCompletableFuture(io.confluent.ksql.util.VertxCompletableFuture) KsqlException(io.confluent.ksql.util.KsqlException) URISyntaxException(java.net.URISyntaxException) ConfigException(org.apache.kafka.common.config.ConfigException) KsqlException(io.confluent.ksql.util.KsqlException)

Aggregations

VertxCompletableFuture (io.confluent.ksql.util.VertxCompletableFuture)20 HttpResponse (io.vertx.ext.web.client.HttpResponse)16 Buffer (io.vertx.core.buffer.Buffer)14 JsonObject (io.vertx.core.json.JsonObject)12 Test (org.junit.Test)12 QueryResponse (io.confluent.ksql.api.utils.QueryResponse)6 ReceiveStream (io.confluent.ksql.api.utils.ReceiveStream)4 CompletableFuture (java.util.concurrent.CompletableFuture)4 InsertsResponse (io.confluent.ksql.api.utils.InsertsResponse)3 ArrayList (java.util.ArrayList)3 IntegrationTest (io.confluent.common.utils.IntegrationTest)2 KsqlEngine (io.confluent.ksql.engine.KsqlEngine)2 CommandStatus (io.confluent.ksql.rest.entity.CommandStatus)2 CommandStatuses (io.confluent.ksql.rest.entity.CommandStatuses)2 HealthCheckResponse (io.confluent.ksql.rest.entity.HealthCheckResponse)2 KsqlEntityList (io.confluent.ksql.rest.entity.KsqlEntityList)2 KsqlRequest (io.confluent.ksql.rest.entity.KsqlRequest)2 ServerClusterId (io.confluent.ksql.rest.entity.ServerClusterId)2 ServerInfo (io.confluent.ksql.rest.entity.ServerInfo)2 ServerMetadata (io.confluent.ksql.rest.entity.ServerMetadata)2