Search in sources :

Example 16 with TopologyConfig

use of org.apache.kafka.streams.processor.internals.namedtopology.TopologyConfig in project kafka by apache.

the class StreamTaskTest method createStatelessTaskWithForwardingTopology.

private StreamTask createStatelessTaskWithForwardingTopology(final SourceNode<Integer, Integer> sourceNode) {
    final ProcessorTopology topology = withSources(asList(sourceNode, processorStreamTime), singletonMap(topic1, sourceNode));
    sourceNode.addChild(processorStreamTime);
    EasyMock.expect(stateManager.changelogPartitions()).andReturn(Collections.emptySet());
    EasyMock.expect(recordCollector.offsets()).andReturn(Collections.emptyMap()).anyTimes();
    EasyMock.replay(stateManager, recordCollector);
    final StreamsConfig config = createConfig();
    final InternalProcessorContext context = new ProcessorContextImpl(taskId, config, stateManager, streamsMetrics, null);
    return new StreamTask(taskId, singleton(partition1), topology, consumer, new TopologyConfig(null, config, new Properties()).getTaskConfig(), new StreamsMetricsImpl(metrics, "test", StreamsConfig.METRICS_LATEST, time), stateDirectory, cache, time, stateManager, recordCollector, context, logContext);
}
Also used : Utils.mkProperties(org.apache.kafka.common.utils.Utils.mkProperties) Properties(java.util.Properties) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) TopologyConfig(org.apache.kafka.streams.processor.internals.namedtopology.TopologyConfig) StreamsConfig(org.apache.kafka.streams.StreamsConfig)

Example 17 with TopologyConfig

use of org.apache.kafka.streams.processor.internals.namedtopology.TopologyConfig in project kafka by apache.

the class StreamThreadStateStoreProviderTest method createStreamsTask.

private StreamTask createStreamsTask(final StreamsConfig streamsConfig, final MockClientSupplier clientSupplier, final ProcessorTopology topology, final TaskId taskId) {
    final Metrics metrics = new Metrics();
    final LogContext logContext = new LogContext("test-stream-task ");
    final Set<TopicPartition> partitions = Collections.singleton(new TopicPartition(topicName, taskId.partition()));
    final ProcessorStateManager stateManager = new ProcessorStateManager(taskId, Task.TaskType.ACTIVE, StreamsConfigUtils.eosEnabled(streamsConfig), logContext, stateDirectory, new StoreChangelogReader(new MockTime(), streamsConfig, logContext, clientSupplier.adminClient, clientSupplier.restoreConsumer, new MockStateRestoreListener()), topology.storeToChangelogTopic(), partitions);
    final RecordCollector recordCollector = new RecordCollectorImpl(logContext, taskId, new StreamsProducer(streamsConfig, "threadId", clientSupplier, new TaskId(0, 0), UUID.randomUUID(), logContext, Time.SYSTEM), streamsConfig.defaultProductionExceptionHandler(), new MockStreamsMetrics(metrics));
    final StreamsMetricsImpl streamsMetrics = new MockStreamsMetrics(metrics);
    final InternalProcessorContext context = new ProcessorContextImpl(taskId, streamsConfig, stateManager, streamsMetrics, null);
    return new StreamTask(taskId, partitions, topology, clientSupplier.consumer, new TopologyConfig(null, streamsConfig, new Properties()).getTaskConfig(), streamsMetrics, stateDirectory, EasyMock.createNiceMock(ThreadCache.class), new MockTime(), stateManager, recordCollector, context, logContext);
}
Also used : TaskId(org.apache.kafka.streams.processor.TaskId) RecordCollector(org.apache.kafka.streams.processor.internals.RecordCollector) StreamsProducer(org.apache.kafka.streams.processor.internals.StreamsProducer) InternalProcessorContext(org.apache.kafka.streams.processor.internals.InternalProcessorContext) LogContext(org.apache.kafka.common.utils.LogContext) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) MockStateRestoreListener(org.apache.kafka.test.MockStateRestoreListener) ProcessorContextImpl(org.apache.kafka.streams.processor.internals.ProcessorContextImpl) Properties(java.util.Properties) RecordCollectorImpl(org.apache.kafka.streams.processor.internals.RecordCollectorImpl) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) TopicPartition(org.apache.kafka.common.TopicPartition) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) StoreChangelogReader(org.apache.kafka.streams.processor.internals.StoreChangelogReader) ProcessorStateManager(org.apache.kafka.streams.processor.internals.ProcessorStateManager) MockTime(org.apache.kafka.common.utils.MockTime) StreamTask(org.apache.kafka.streams.processor.internals.StreamTask) TopologyConfig(org.apache.kafka.streams.processor.internals.namedtopology.TopologyConfig)

Aggregations

TopologyConfig (org.apache.kafka.streams.processor.internals.namedtopology.TopologyConfig)17 Properties (java.util.Properties)14 Utils.mkProperties (org.apache.kafka.common.utils.Utils.mkProperties)13 StreamsConfig (org.apache.kafka.streams.StreamsConfig)7 StreamsMetricsImpl (org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)5 Test (org.junit.Test)5 TopicPartition (org.apache.kafka.common.TopicPartition)4 LogContext (org.apache.kafka.common.utils.LogContext)3 MockKeyValueStore (org.apache.kafka.test.MockKeyValueStore)3 MockTime (org.apache.kafka.common.utils.MockTime)2 LogAndContinueExceptionHandler (org.apache.kafka.streams.errors.LogAndContinueExceptionHandler)2 TaskId (org.apache.kafka.streams.processor.TaskId)2 ThreadCache (org.apache.kafka.streams.state.internals.ThreadCache)2 MockTimestampExtractor (org.apache.kafka.test.MockTimestampExtractor)2 File (java.io.File)1 HashSet (java.util.HashSet)1 Set (java.util.Set)1 MockConsumer (org.apache.kafka.clients.consumer.MockConsumer)1 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)1 TimeoutException (org.apache.kafka.common.errors.TimeoutException)1