Search in sources :

Example 11 with InternalFunctionRegistry

use of io.confluent.ksql.function.InternalFunctionRegistry in project ksql by confluentinc.

the class TopicCreateInjectorTest method setUp.

@Before
public void setUp() {
    parser = new DefaultKsqlParser();
    metaStore = new MetaStoreImpl(new InternalFunctionRegistry());
    overrides = new HashMap<>();
    config = new KsqlConfig(new HashMap<>());
    injector = new TopicCreateInjector(topicClient, metaStore);
    final KsqlTopic sourceTopic = new KsqlTopic("source", KeyFormat.nonWindowed(FormatInfo.of(FormatFactory.KAFKA.name()), SerdeFeatures.of()), ValueFormat.of(FormatInfo.of(FormatFactory.JSON.name()), SerdeFeatures.of()));
    final KsqlStream<?> source = new KsqlStream<>("", SourceName.of("SOURCE"), SCHEMA, Optional.empty(), false, sourceTopic, false);
    metaStore.putSource(source, false);
    final KsqlTopic joinTopic = new KsqlTopic("jSource", KeyFormat.nonWindowed(FormatInfo.of(FormatFactory.KAFKA.name()), SerdeFeatures.of()), ValueFormat.of(FormatInfo.of(FormatFactory.JSON.name()), SerdeFeatures.of()));
    final KsqlStream<?> joinSource = new KsqlStream<>("", SourceName.of("J_SOURCE"), SCHEMA, Optional.empty(), false, joinTopic, false);
    metaStore.putSource(joinSource, false);
    when(topicClient.describeTopic("source")).thenReturn(sourceDescription);
    when(topicClient.isTopicExists("source")).thenReturn(true);
    when(builder.withName(any())).thenReturn(builder);
    when(builder.withWithClause(any(), any(), any())).thenReturn(builder);
    when(builder.withSource(any())).thenReturn(builder);
    when(builder.build()).thenReturn(new TopicProperties("name", 1, (short) 1));
}
Also used : KsqlStream(io.confluent.ksql.metastore.model.KsqlStream) HashMap(java.util.HashMap) MetaStoreImpl(io.confluent.ksql.metastore.MetaStoreImpl) KsqlConfig(io.confluent.ksql.util.KsqlConfig) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry) DefaultKsqlParser(io.confluent.ksql.parser.DefaultKsqlParser) KsqlTopic(io.confluent.ksql.execution.ddl.commands.KsqlTopic) Before(org.junit.Before)

Example 12 with InternalFunctionRegistry

use of io.confluent.ksql.function.InternalFunctionRegistry in project ksql by confluentinc.

the class TemporaryEngine method before.

@Override
protected void before() {
    final InternalFunctionRegistry functionRegistry = new InternalFunctionRegistry();
    metaStore = new MetaStoreImpl(functionRegistry);
    serviceContext = TestServiceContext.create();
    engine = (KsqlEngineTestUtil.createKsqlEngine(getServiceContext(), metaStore, new MetricCollectors()));
    ksqlConfig = KsqlConfigTestUtil.create("localhost:9092", ImmutableMap.<String, Object>builder().putAll(configs).put("ksql.command.topic.suffix", "commands").put(KsqlRestConfig.LISTENERS_CONFIG, "http://localhost:8088").build());
    final SqlTypeParser typeParser = SqlTypeParser.create(TypeRegistry.EMPTY);
    final Optional<Metrics> noMetrics = Optional.empty();
    final UdfLoader udfLoader = new UdfLoader(functionRegistry, noMetrics, typeParser, true);
    udfLoader.loadUdfFromClass(TestUdf1.class, "test");
    final UdtfLoader udtfLoader = new UdtfLoader(functionRegistry, noMetrics, typeParser, true);
    udtfLoader.loadUdtfFromClass(TestUdtf1.class, "whatever");
    udtfLoader.loadUdtfFromClass(TestUdtf2.class, "whatever");
}
Also used : Metrics(org.apache.kafka.common.metrics.Metrics) MetaStoreImpl(io.confluent.ksql.metastore.MetaStoreImpl) MetricCollectors(io.confluent.ksql.metrics.MetricCollectors) SqlTypeParser(io.confluent.ksql.schema.ksql.SqlTypeParser) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry) UdfLoader(io.confluent.ksql.function.UdfLoader) UdtfLoader(io.confluent.ksql.function.UdtfLoader)

Example 13 with InternalFunctionRegistry

use of io.confluent.ksql.function.InternalFunctionRegistry in project ksql by confluentinc.

the class RecoveryTest method createKsqlEngine.

private KsqlEngine createKsqlEngine(final QueryIdGenerator queryIdGenerator) {
    final KsqlEngineMetrics engineMetrics = mock(KsqlEngineMetrics.class);
    when(engineMetrics.getQueryEventListener()).thenReturn(mock(QueryEventListener.class));
    return KsqlEngineTestUtil.createKsqlEngine(serviceContext, new MetaStoreImpl(new InternalFunctionRegistry()), ignored -> engineMetrics, queryIdGenerator, ksqlConfig, new MetricCollectors());
}
Also used : KsqlEngineMetrics(io.confluent.ksql.internal.KsqlEngineMetrics) QueryEventListener(io.confluent.ksql.engine.QueryEventListener) MetaStoreImpl(io.confluent.ksql.metastore.MetaStoreImpl) MetricCollectors(io.confluent.ksql.metrics.MetricCollectors) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry)

Example 14 with InternalFunctionRegistry

use of io.confluent.ksql.function.InternalFunctionRegistry in project ksql by confluentinc.

the class InteractiveStatementExecutorTest method setUp.

@Before
public void setUp() {
    ksqlConfig = KsqlConfigTestUtil.create(CLUSTER, ImmutableMap.of(StreamsConfig.APPLICATION_SERVER_CONFIG, "http://host:1234"));
    final FakeKafkaTopicClient fakeKafkaTopicClient = new FakeKafkaTopicClient();
    fakeKafkaTopicClient.createTopic("pageview_topic", 1, (short) 1, emptyMap());
    fakeKafkaTopicClient.createTopic("foo", 1, (short) 1, emptyMap());
    fakeKafkaTopicClient.createTopic("pageview_topic_json", 1, (short) 1, emptyMap());
    serviceContext = TestServiceContext.create(fakeKafkaTopicClient);
    final SpecificQueryIdGenerator hybridQueryIdGenerator = new SpecificQueryIdGenerator();
    final MetricCollectors metricCollectors = new MetricCollectors();
    ksqlEngine = KsqlEngineTestUtil.createKsqlEngine(serviceContext, new MetaStoreImpl(new InternalFunctionRegistry()), (engine) -> new KsqlEngineMetrics("", engine, Collections.emptyMap(), Optional.empty(), metricCollectors), hybridQueryIdGenerator, ksqlConfig, metricCollectors);
    statementParser = new StatementParser(ksqlEngine);
    statementExecutor = new InteractiveStatementExecutor(serviceContext, ksqlEngine, statementParser, hybridQueryIdGenerator, InternalTopicSerdes.deserializer(Command.class));
    statementExecutorWithMocks = new InteractiveStatementExecutor(serviceContext, mockEngine, mockParser, mockQueryIdGenerator, commandDeserializer);
    statementExecutor.configure(ksqlConfig);
    statementExecutorWithMocks.configure(ksqlConfig);
    plannedCommand = new Command(CREATE_STREAM_FOO_STATEMENT, emptyMap(), ksqlConfig.getAllConfigPropsWithSecretsObfuscated(), Optional.of(plan));
}
Also used : CoreMatchers(org.hamcrest.CoreMatchers) KsqlPlan(io.confluent.ksql.engine.KsqlPlan) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry) ArgumentMatchers.argThat(org.mockito.ArgumentMatchers.argThat) Matchers.not(org.hamcrest.Matchers.not) KsqlEngineTestUtil(io.confluent.ksql.engine.KsqlEngineTestUtil) ServiceContext(io.confluent.ksql.services.ServiceContext) CommandId(io.confluent.ksql.rest.entity.CommandId) ArgumentMatcher(org.mockito.ArgumentMatcher) Pair(io.confluent.ksql.util.Pair) ExecuteResult(io.confluent.ksql.KsqlExecutionContext.ExecuteResult) Map(java.util.Map) After(org.junit.After) QueryId(io.confluent.ksql.query.QueryId) FakeKafkaTopicClient(io.confluent.ksql.services.FakeKafkaTopicClient) ClassRule(org.junit.ClassRule) PersistentQueryMetadata(io.confluent.ksql.util.PersistentQueryMetadata) ZooKeeperClientException(kafka.zookeeper.ZooKeeperClientException) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) CommandStatus(io.confluent.ksql.rest.entity.CommandStatus) ImmutableMap(com.google.common.collect.ImmutableMap) DropStream(io.confluent.ksql.parser.tree.DropStream) StatementParser(io.confluent.ksql.rest.server.StatementParser) ConfiguredStatement(io.confluent.ksql.statement.ConfiguredStatement) KsqlConfig(io.confluent.ksql.util.KsqlConfig) TestServiceContext(io.confluent.ksql.services.TestServiceContext) MetaStoreImpl(io.confluent.ksql.metastore.MetaStoreImpl) List(java.util.List) TerminateQuery(io.confluent.ksql.parser.tree.TerminateQuery) Status(io.confluent.ksql.rest.entity.CommandStatus.Status) KsqlEngineMetrics(io.confluent.ksql.internal.KsqlEngineMetrics) KsqlException(io.confluent.ksql.util.KsqlException) Optional(java.util.Optional) Matchers.is(org.hamcrest.Matchers.is) Statement(io.confluent.ksql.parser.tree.Statement) Retry(io.confluent.ksql.integration.Retry) MockitoJUnitRunner(org.mockito.junit.MockitoJUnitRunner) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) Mockito.eq(org.mockito.Mockito.eq) Mockito.mock(org.mockito.Mockito.mock) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) StreamsConfig(org.apache.kafka.streams.StreamsConfig) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) Action(io.confluent.ksql.rest.entity.CommandId.Action) EmbeddedSingleNodeKafkaCluster(io.confluent.ksql.test.util.EmbeddedSingleNodeKafkaCluster) CoreMatchers.equalTo(org.hamcrest.CoreMatchers.equalTo) Mock(org.mockito.Mock) Assert.assertThrows(org.junit.Assert.assertThrows) RunWith(org.junit.runner.RunWith) SpecificQueryIdGenerator(io.confluent.ksql.query.id.SpecificQueryIdGenerator) MetricCollectors(io.confluent.ksql.metrics.MetricCollectors) CreateStreamAsSelect(io.confluent.ksql.parser.tree.CreateStreamAsSelect) ArrayList(java.util.ArrayList) SessionConfig(io.confluent.ksql.config.SessionConfig) CreateStream(io.confluent.ksql.parser.tree.CreateStream) ArgumentCaptor(org.mockito.ArgumentCaptor) ImmutableList(com.google.common.collect.ImmutableList) ConfiguredKsqlPlan(io.confluent.ksql.planner.plan.ConfiguredKsqlPlan) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Deserializer(org.apache.kafka.common.serialization.Deserializer) Before(org.junit.Before) Collections.emptyMap(java.util.Collections.emptyMap) InOrder(org.mockito.InOrder) KsqlEngine(io.confluent.ksql.engine.KsqlEngine) Mockito.times(org.mockito.Mockito.times) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) Mockito.verify(org.mockito.Mockito.verify) Type(io.confluent.ksql.rest.entity.CommandId.Type) KsqlStatementException(io.confluent.ksql.util.KsqlStatementException) TimeUnit(java.util.concurrent.TimeUnit) RuleChain(org.junit.rules.RuleChain) KsqlConfigTestUtil(io.confluent.ksql.KsqlConfigTestUtil) Mockito(org.mockito.Mockito) Assert(org.junit.Assert) Collections(java.util.Collections) Assert.assertEquals(org.junit.Assert.assertEquals) PreparedStatement(io.confluent.ksql.parser.KsqlParser.PreparedStatement) KsqlEngineMetrics(io.confluent.ksql.internal.KsqlEngineMetrics) StatementParser(io.confluent.ksql.rest.server.StatementParser) MetaStoreImpl(io.confluent.ksql.metastore.MetaStoreImpl) FakeKafkaTopicClient(io.confluent.ksql.services.FakeKafkaTopicClient) MetricCollectors(io.confluent.ksql.metrics.MetricCollectors) SpecificQueryIdGenerator(io.confluent.ksql.query.id.SpecificQueryIdGenerator) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry) Before(org.junit.Before)

Example 15 with InternalFunctionRegistry

use of io.confluent.ksql.function.InternalFunctionRegistry in project ksql by confluentinc.

the class AggregateNodeTest method buildAggregateNode.

private static AggregateNode buildAggregateNode(final String queryString) {
    final MetaStore newMetaStore = MetaStoreFixture.getNewMetaStore(new InternalFunctionRegistry());
    final KsqlBareOutputNode planNode = (KsqlBareOutputNode) AnalysisTestUtil.buildLogicalPlan(KSQL_CONFIG, queryString, newMetaStore);
    return (AggregateNode) planNode.getSource();
}
Also used : MetaStore(io.confluent.ksql.metastore.MetaStore) InternalFunctionRegistry(io.confluent.ksql.function.InternalFunctionRegistry)

Aggregations

InternalFunctionRegistry (io.confluent.ksql.function.InternalFunctionRegistry)23 MetaStoreImpl (io.confluent.ksql.metastore.MetaStoreImpl)13 Before (org.junit.Before)12 MetricCollectors (io.confluent.ksql.metrics.MetricCollectors)10 KsqlConfig (io.confluent.ksql.util.KsqlConfig)9 KsqlEngine (io.confluent.ksql.engine.KsqlEngine)8 MutableFunctionRegistry (io.confluent.ksql.function.MutableFunctionRegistry)6 SequentialQueryIdGenerator (io.confluent.ksql.query.id.SequentialQueryIdGenerator)6 ServiceContext (io.confluent.ksql.services.ServiceContext)5 KsqlTopic (io.confluent.ksql.execution.ddl.commands.KsqlTopic)4 KsqlEngineMetrics (io.confluent.ksql.internal.KsqlEngineMetrics)4 ImmutableList (com.google.common.collect.ImmutableList)3 ImmutableMap (com.google.common.collect.ImmutableMap)3 SessionConfig (io.confluent.ksql.config.SessionConfig)3 List (java.util.List)3 Map (java.util.Map)3 Optional (java.util.Optional)3 KsqlConfigTestUtil (io.confluent.ksql.KsqlConfigTestUtil)2 KsqlEngineTestUtil (io.confluent.ksql.engine.KsqlEngineTestUtil)2 KsqlPlan (io.confluent.ksql.engine.KsqlPlan)2