Search in sources :

Example 6 with RoutingOptions

use of io.confluent.ksql.execution.streams.RoutingOptions in project ksql by confluentinc.

the class QueryExecutor method handleTablePullQuery.

private QueryMetadataHolder handleTablePullQuery(final ImmutableAnalysis analysis, final ServiceContext serviceContext, final ConfiguredStatement<Query> configured, final Map<String, Object> requestProperties, final Optional<Boolean> isInternalRequest, final SlidingWindowRateLimiter pullBandRateLimiter, final AtomicReference<PullQueryResult> resultForMetrics, final Optional<ConsistencyOffsetVector> consistencyOffsetVector) {
    final RoutingOptions routingOptions = new PullQueryConfigRoutingOptions(configured.getSessionConfig().getConfig(false), configured.getSessionConfig().getOverrides(), requestProperties);
    final PullQueryConfigPlannerOptions plannerOptions = new PullQueryConfigPlannerOptions(configured.getSessionConfig().getConfig(false), configured.getSessionConfig().getOverrides());
    // A request is considered forwarded if the request has the forwarded flag or if the request
    // is from an internal listener.
    final boolean isAlreadyForwarded = routingOptions.getIsSkipForwardRequest() && // Trust the forward request option if isInternalRequest isn't available.
    isInternalRequest.orElse(true);
    // Only check the rate limit at the forwarding host
    Decrementer decrementer = null;
    try {
        if (!isAlreadyForwarded) {
            rateLimiter.checkLimit();
            decrementer = concurrencyLimiter.increment();
        }
        pullBandRateLimiter.allow(KsqlQueryType.PULL);
        final Optional<Decrementer> optionalDecrementer = Optional.ofNullable(decrementer);
        final PullQueryResult result = ksqlEngine.executeTablePullQuery(analysis, serviceContext, configured, routing, routingOptions, plannerOptions, pullQueryMetrics, false, consistencyOffsetVector);
        resultForMetrics.set(result);
        result.onCompletionOrException((v, t) -> optionalDecrementer.ifPresent(Decrementer::decrementAtMostOnce));
        return QueryMetadataHolder.of(result);
    } catch (final Throwable t) {
        if (decrementer != null) {
            decrementer.decrementAtMostOnce();
        }
        throw t;
    }
}
Also used : RoutingOptions(io.confluent.ksql.execution.streams.RoutingOptions) PullQueryConfigRoutingOptions(io.confluent.ksql.rest.server.resources.streaming.PullQueryConfigRoutingOptions) PushQueryConfigRoutingOptions(io.confluent.ksql.rest.server.resources.streaming.PushQueryConfigRoutingOptions) PullQueryConfigRoutingOptions(io.confluent.ksql.rest.server.resources.streaming.PullQueryConfigRoutingOptions) PullQueryConfigPlannerOptions(io.confluent.ksql.rest.server.resources.streaming.PullQueryConfigPlannerOptions) Decrementer(io.confluent.ksql.rest.util.ConcurrencyLimiter.Decrementer) PullQueryResult(io.confluent.ksql.physical.pull.PullQueryResult)

Aggregations

RoutingOptions (io.confluent.ksql.execution.streams.RoutingOptions)6 ImmutableMap (com.google.common.collect.ImmutableMap)5 Preconditions (com.google.common.base.Preconditions)4 PullQueryExecutorMetrics (io.confluent.ksql.internal.PullQueryExecutorMetrics)4 Query (io.confluent.ksql.parser.tree.Query)4 List (java.util.List)4 Optional (java.util.Optional)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 ImmutableList (com.google.common.collect.ImmutableList)3 ThreadFactoryBuilder (com.google.common.util.concurrent.ThreadFactoryBuilder)3 Host (io.confluent.ksql.execution.streams.RoutingFilter.Host)3 RoutingFilterFactory (io.confluent.ksql.execution.streams.RoutingFilter.RoutingFilterFactory)3 KsqlNode (io.confluent.ksql.execution.streams.materialization.Locator.KsqlNode)3 KsqlPartitionLocation (io.confluent.ksql.execution.streams.materialization.Locator.KsqlPartitionLocation)3 MaterializationException (io.confluent.ksql.execution.streams.materialization.MaterializationException)3 NotUpToBoundException (io.confluent.ksql.execution.streams.materialization.ks.NotUpToBoundException)3 PullPhysicalPlanType (io.confluent.ksql.physical.pull.PullPhysicalPlan.PullPhysicalPlanType)3 PullQueryQueue (io.confluent.ksql.query.PullQueryQueue)3 QueryId (io.confluent.ksql.query.QueryId)3 RestResponse (io.confluent.ksql.rest.client.RestResponse)3