Search in sources :

Example 1 with SandboxedSharedKafkaStreamsRuntimeImpl

use of io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl in project ksql by confluentinc.

the class QueryRegistryImpl method createOrReplacePersistentQuery.

// CHECKSTYLE_RULES.OFF: ParameterNumberCheck
@Override
public PersistentQueryMetadata createOrReplacePersistentQuery(final SessionConfig config, final ServiceContext serviceContext, final ProcessingLogContext processingLogContext, final MetaStore metaStore, final String statementText, final QueryId queryId, final Optional<DataSource> sinkDataSource, final Set<DataSource> sources, final ExecutionStep<?> physicalPlan, final String planSummary, final KsqlConstants.PersistentQueryType persistentQueryType, final Optional<String> sharedRuntimeId) {
    // CHECKSTYLE_RULES.ON: ParameterNumberCheck
    final QueryBuilder queryBuilder = queryBuilderFactory.create(config, processingLogContext, serviceContext, metaStore, streams, !sandbox);
    final KsqlConfig ksqlConfig = config.getConfig(true);
    final PersistentQueryMetadata query;
    final PersistentQueryMetadata oldQuery = persistentQueries.get(queryId);
    if (sharedRuntimeId.isPresent() && ksqlConfig.getBoolean(KsqlConfig.KSQL_SHARED_RUNTIME_ENABLED) && (oldQuery == null || oldQuery instanceof BinPackedPersistentQueryMetadataImpl)) {
        if (sandbox) {
            throwOnNonQueryLevelConfigs(config.getOverrides());
            streams.addAll(sourceStreams.stream().map(SandboxedSharedKafkaStreamsRuntimeImpl::new).collect(Collectors.toList()));
        }
        query = queryBuilder.buildPersistentQueryInSharedRuntime(ksqlConfig, persistentQueryType, statementText, queryId, sinkDataSource, sources, physicalPlan, planSummary, new ListenerImpl(), () -> ImmutableList.copyOf(getPersistentQueries().values()), sharedRuntimeId.get(), metricCollectors);
        query.register();
    } else {
        query = queryBuilder.buildPersistentQueryInDedicatedRuntime(ksqlConfig, persistentQueryType, statementText, queryId, sinkDataSource, sources, physicalPlan, planSummary, new ListenerImpl(), () -> ImmutableList.copyOf(getPersistentQueries().values()), new StreamsBuilder(), metricCollectors);
    }
    registerPersistentQuery(serviceContext, metaStore, query);
    return query;
}
Also used : StreamsBuilder(org.apache.kafka.streams.StreamsBuilder) SandboxedSharedKafkaStreamsRuntimeImpl(io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl) KsqlConfig(io.confluent.ksql.util.KsqlConfig) PersistentQueryMetadata(io.confluent.ksql.util.PersistentQueryMetadata) BinPackedPersistentQueryMetadataImpl(io.confluent.ksql.util.BinPackedPersistentQueryMetadataImpl) SandboxedBinPackedPersistentQueryMetadataImpl(io.confluent.ksql.util.SandboxedBinPackedPersistentQueryMetadataImpl)

Example 2 with SandboxedSharedKafkaStreamsRuntimeImpl

use of io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl in project ksql by confluentinc.

the class QueryBuilder method getKafkaStreamsInstance.

private SharedKafkaStreamsRuntime getKafkaStreamsInstance(final String applicationId, final Set<SourceName> sources, final QueryId queryId, final MetricCollectors metricCollectors) {
    for (final SharedKafkaStreamsRuntime sharedKafkaStreamsRuntime : streams) {
        if (sharedKafkaStreamsRuntime.getApplicationId().equals(applicationId) || (sharedKafkaStreamsRuntime.getApplicationId().equals(applicationId + "-validation") && !real)) {
            return sharedKafkaStreamsRuntime;
        }
    }
    final SharedKafkaStreamsRuntime stream;
    final KsqlConfig ksqlConfig = config.getConfig(true);
    if (real) {
        stream = new SharedKafkaStreamsRuntimeImpl(kafkaStreamsBuilder, getConfiguredQueryErrorClassifier(ksqlConfig, applicationId), ksqlConfig.getInt(KsqlConfig.KSQL_QUERY_ERROR_MAX_QUEUE_SIZE), ksqlConfig.getLong(KsqlConfig.KSQL_SHUTDOWN_TIMEOUT_MS_CONFIG), buildStreamsProperties(applicationId, Optional.empty(), metricCollectors, config.getConfig(true), processingLogContext));
    } else {
        stream = new SandboxedSharedKafkaStreamsRuntimeImpl(kafkaStreamsBuilder, buildStreamsProperties(applicationId + "-validation", Optional.empty(), metricCollectors, config.getConfig(true), processingLogContext));
    }
    streams.add(stream);
    return stream;
}
Also used : SharedKafkaStreamsRuntime(io.confluent.ksql.util.SharedKafkaStreamsRuntime) SandboxedSharedKafkaStreamsRuntimeImpl(io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl) SandboxedSharedKafkaStreamsRuntimeImpl(io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl) SharedKafkaStreamsRuntimeImpl(io.confluent.ksql.util.SharedKafkaStreamsRuntimeImpl) KsqlConfig(io.confluent.ksql.util.KsqlConfig)

Aggregations

KsqlConfig (io.confluent.ksql.util.KsqlConfig)2 SandboxedSharedKafkaStreamsRuntimeImpl (io.confluent.ksql.util.SandboxedSharedKafkaStreamsRuntimeImpl)2 BinPackedPersistentQueryMetadataImpl (io.confluent.ksql.util.BinPackedPersistentQueryMetadataImpl)1 PersistentQueryMetadata (io.confluent.ksql.util.PersistentQueryMetadata)1 SandboxedBinPackedPersistentQueryMetadataImpl (io.confluent.ksql.util.SandboxedBinPackedPersistentQueryMetadataImpl)1 SharedKafkaStreamsRuntime (io.confluent.ksql.util.SharedKafkaStreamsRuntime)1 SharedKafkaStreamsRuntimeImpl (io.confluent.ksql.util.SharedKafkaStreamsRuntimeImpl)1 StreamsBuilder (org.apache.kafka.streams.StreamsBuilder)1