Search in sources :

Example 26 with StreamsMetricsImpl

use of org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl in project kafka by apache.

the class MeteredKeyValueStoreTest method before.

@Before
public void before() {
    final Time mockTime = new MockTime();
    metered = new MeteredKeyValueStore<>(inner, STORE_TYPE, mockTime, Serdes.String(), Serdes.String());
    metrics.config().recordLevel(Sensor.RecordingLevel.DEBUG);
    expect(context.applicationId()).andStubReturn(APPLICATION_ID);
    expect(context.metrics()).andStubReturn(new StreamsMetricsImpl(metrics, "test", StreamsConfig.METRICS_LATEST, mockTime));
    expect(context.taskId()).andStubReturn(taskId);
    expect(context.changelogFor(STORE_NAME)).andStubReturn(CHANGELOG_TOPIC);
    expect(inner.name()).andStubReturn(STORE_NAME);
    tags = mkMap(mkEntry(THREAD_ID_TAG_KEY, threadId), mkEntry("task-id", taskId.toString()), mkEntry(STORE_TYPE + "-state-id", STORE_NAME));
}
Also used : MockTime(org.apache.kafka.common.utils.MockTime) Time(org.apache.kafka.common.utils.Time) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) MockTime(org.apache.kafka.common.utils.MockTime) Before(org.junit.Before)

Example 27 with StreamsMetricsImpl

use of org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl in project kafka by apache.

the class TopologyTestDriver method setupMetrics.

private StreamsMetricsImpl setupMetrics(final StreamsConfig streamsConfig) {
    final String threadId = Thread.currentThread().getName();
    final MetricConfig metricConfig = new MetricConfig().samples(streamsConfig.getInt(StreamsConfig.METRICS_NUM_SAMPLES_CONFIG)).recordLevel(Sensor.RecordingLevel.forName(streamsConfig.getString(StreamsConfig.METRICS_RECORDING_LEVEL_CONFIG))).timeWindow(streamsConfig.getLong(StreamsConfig.METRICS_SAMPLE_WINDOW_MS_CONFIG), TimeUnit.MILLISECONDS);
    metrics = new Metrics(metricConfig, mockWallClockTime);
    final StreamsMetricsImpl streamsMetrics = new StreamsMetricsImpl(metrics, "test-client", streamsConfig.getString(StreamsConfig.BUILT_IN_METRICS_VERSION_CONFIG), mockWallClockTime);
    TaskMetrics.droppedRecordsSensor(threadId, TASK_ID.toString(), streamsMetrics);
    return streamsMetrics;
}
Also used : MetricConfig(org.apache.kafka.common.metrics.MetricConfig) Metrics(org.apache.kafka.common.metrics.Metrics) TaskMetrics(org.apache.kafka.streams.processor.internals.metrics.TaskMetrics) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)

Example 28 with StreamsMetricsImpl

use of org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl in project kafka by apache.

the class KStreamKTableJoinProcessor method init.

@Override
public void init(final ProcessorContext<K1, VOut> context) {
    super.init(context);
    final StreamsMetricsImpl metrics = (StreamsMetricsImpl) context.metrics();
    droppedRecordsSensor = droppedRecordsSensor(Thread.currentThread().getName(), context.taskId().toString(), metrics);
    valueGetter.init(context);
}
Also used : StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)

Example 29 with StreamsMetricsImpl

use of org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl in project kafka by apache.

the class KStreamSessionWindowAggregateProcessorTest method setup.

private void setup(final boolean enableCache) {
    final StreamsMetricsImpl streamsMetrics = new StreamsMetricsImpl(metrics, "test", StreamsConfig.METRICS_LATEST, new MockTime());
    context = new InternalMockProcessorContext<Windowed<String>, Change<Long>>(TestUtils.tempDirectory(), Serdes.String(), Serdes.String(), streamsMetrics, new StreamsConfig(StreamsTestUtils.getStreamsConfig()), MockRecordCollector::new, new ThreadCache(new LogContext("testCache "), 100000, streamsMetrics), Time.SYSTEM) {

        @Override
        public <K extends Windowed<String>, V extends Change<Long>> void forward(final Record<K, V> record) {
            results.add(new KeyValueTimestamp<>(record.key(), record.value(), record.timestamp()));
        }
    };
    // Set initial timestamp for CachingSessionStore to prepare entry from as default
    // InternalMockProcessorContext#timestamp returns -1.
    context.setTime(0L);
    TaskMetrics.droppedRecordsSensor(threadId, context.taskId().toString(), streamsMetrics);
    initStore(enableCache);
    processor.init(context);
}
Also used : LogContext(org.apache.kafka.common.utils.LogContext) Windowed(org.apache.kafka.streams.kstream.Windowed) ThreadCache(org.apache.kafka.streams.state.internals.ThreadCache) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) KeyValueTimestamp(org.apache.kafka.streams.KeyValueTimestamp) MockTime(org.apache.kafka.common.utils.MockTime) StreamsConfig(org.apache.kafka.streams.StreamsConfig)

Example 30 with StreamsMetricsImpl

use of org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl in project kafka by apache.

the class ProcessorNodeTest method testTopologyLevelClassCastExceptionDirect.

@Test
public void testTopologyLevelClassCastExceptionDirect() {
    final Metrics metrics = new Metrics();
    final StreamsMetricsImpl streamsMetrics = new StreamsMetricsImpl(metrics, "test-client", StreamsConfig.METRICS_LATEST, new MockTime());
    final InternalMockProcessorContext<Object, Object> context = new InternalMockProcessorContext<>(streamsMetrics);
    final ProcessorNode<Object, Object, Object, Object> node = new ProcessorNode<>("pname", new ClassCastProcessor(), Collections.emptySet());
    node.init(context);
    final StreamsException se = assertThrows(StreamsException.class, () -> node.process(new Record<>("aKey", "aValue", 0)));
    assertThat(se.getCause(), instanceOf(ClassCastException.class));
    assertThat(se.getMessage(), containsString("default Serdes"));
    assertThat(se.getMessage(), containsString("input types"));
    assertThat(se.getMessage(), containsString("pname"));
}
Also used : Metrics(org.apache.kafka.common.metrics.Metrics) StreamsException(org.apache.kafka.streams.errors.StreamsException) Record(org.apache.kafka.streams.processor.api.Record) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) InternalMockProcessorContext(org.apache.kafka.test.InternalMockProcessorContext) MockTime(org.apache.kafka.common.utils.MockTime) Test(org.junit.Test)

Aggregations

StreamsMetricsImpl (org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)54 Test (org.junit.Test)31 Metrics (org.apache.kafka.common.metrics.Metrics)22 MockTime (org.apache.kafka.common.utils.MockTime)22 LogContext (org.apache.kafka.common.utils.LogContext)18 StreamsConfig (org.apache.kafka.streams.StreamsConfig)18 TaskId (org.apache.kafka.streams.processor.TaskId)16 ConsumerGroupMetadata (org.apache.kafka.clients.consumer.ConsumerGroupMetadata)14 Properties (java.util.Properties)12 Before (org.junit.Before)11 File (java.io.File)9 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)9 AtomicLong (java.util.concurrent.atomic.AtomicLong)9 LinkedList (java.util.LinkedList)8 Set (java.util.Set)8 MockConsumer (org.apache.kafka.clients.consumer.MockConsumer)8 TopicPartition (org.apache.kafka.common.TopicPartition)8 MockStateRestoreListener (org.apache.kafka.test.MockStateRestoreListener)8 HashMap (java.util.HashMap)7 KeyValue (org.apache.kafka.streams.KeyValue)7