Search in sources :

Example 56 with MockStreamsMetrics

use of org.apache.kafka.streams.processor.internals.MockStreamsMetrics in project kafka by apache.

the class ThreadCacheTest method shouldThrowIfNoPeekNextKey.

@Test(expected = NoSuchElementException.class)
public void shouldThrowIfNoPeekNextKey() throws Exception {
    final ThreadCache cache = new ThreadCache("testCache", 10000L, new MockStreamsMetrics(new Metrics()));
    final ThreadCache.MemoryLRUCacheBytesIterator iterator = cache.range("", Bytes.wrap(new byte[] { 0 }), Bytes.wrap(new byte[] { 1 }));
    iterator.peekNextKey();
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Example 57 with MockStreamsMetrics

use of org.apache.kafka.streams.processor.internals.MockStreamsMetrics in project kafka by apache.

the class CachingWindowStoreTest method setUp.

@Before
public void setUp() throws Exception {
    keySchema = new WindowKeySchema();
    underlying = new RocksDBSegmentedBytesStore("test", 30000, 3, keySchema);
    final RocksDBWindowStore<Bytes, byte[]> windowStore = new RocksDBWindowStore<>(underlying, Serdes.Bytes(), Serdes.ByteArray(), false);
    cacheListener = new CachingKeyValueStoreTest.CacheFlushListenerStub<>();
    cachingStore = new CachingWindowStore<>(windowStore, Serdes.String(), Serdes.String(), WINDOW_SIZE);
    cachingStore.setFlushListener(cacheListener);
    cache = new ThreadCache("testCache", MAX_CACHE_SIZE_BYTES, new MockStreamsMetrics(new Metrics()));
    topic = "topic";
    final MockProcessorContext context = new MockProcessorContext(TestUtils.tempDirectory(), null, null, (RecordCollector) null, cache);
    context.setRecordContext(new ProcessorRecordContext(DEFAULT_TIMESTAMP, 0, 0, topic));
    cachingStore.init(context, cachingStore);
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Bytes(org.apache.kafka.common.utils.Bytes) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) Before(org.junit.Before)

Example 58 with MockStreamsMetrics

use of org.apache.kafka.streams.processor.internals.MockStreamsMetrics in project kafka by apache.

the class ChangeLoggingSegmentedBytesStoreTest method setUp.

@SuppressWarnings("unchecked")
@Before
public void setUp() throws Exception {
    final NoOpRecordCollector collector = new NoOpRecordCollector() {

        @Override
        public <K, V> void send(final String topic, K key, V value, Integer partition, Long timestamp, Serializer<K> keySerializer, Serializer<V> valueSerializer) {
            sent.put(key, value);
        }
    };
    final MockProcessorContext context = new MockProcessorContext(TestUtils.tempDirectory(), Serdes.String(), Serdes.Long(), collector, new ThreadCache("testCache", 0, new MockStreamsMetrics(new Metrics())));
    context.setTime(0);
    store.init(context, store);
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) NoOpRecordCollector(org.apache.kafka.test.NoOpRecordCollector) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Serializer(org.apache.kafka.common.serialization.Serializer) Before(org.junit.Before)

Example 59 with MockStreamsMetrics

use of org.apache.kafka.streams.processor.internals.MockStreamsMetrics in project kafka by apache.

the class CachingSessionStoreTest method setUp.

@Before
public void setUp() throws Exception {
    underlying = new RocksDBSegmentedBytesStore("test", 60000, 3, new SessionKeySchema());
    final RocksDBSessionStore<Bytes, byte[]> sessionStore = new RocksDBSessionStore<>(underlying, Serdes.Bytes(), Serdes.ByteArray());
    cachingStore = new CachingSessionStore<>(sessionStore, Serdes.String(), Serdes.Long());
    cache = new ThreadCache("testCache", MAX_CACHE_SIZE_BYTES, new MockStreamsMetrics(new Metrics()));
    final MockProcessorContext context = new MockProcessorContext(TestUtils.tempDirectory(), null, null, (RecordCollector) null, cache);
    context.setRecordContext(new ProcessorRecordContext(DEFAULT_TIMESTAMP, 0, 0, "topic"));
    cachingStore.init(context, cachingStore);
}
Also used : Bytes(org.apache.kafka.common.utils.Bytes) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) ProcessorRecordContext(org.apache.kafka.streams.processor.internals.ProcessorRecordContext) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Before(org.junit.Before)

Example 60 with MockStreamsMetrics

use of org.apache.kafka.streams.processor.internals.MockStreamsMetrics in project apache-kafka-on-k8s by banzaicloud.

the class ThreadCacheTest method shouldNotLoopForEverWhenEvictingAndCurrentCacheIsEmpty.

@Test
public void shouldNotLoopForEverWhenEvictingAndCurrentCacheIsEmpty() {
    final int maxCacheSizeInBytes = 100;
    final ThreadCache threadCache = new ThreadCache(logContext, maxCacheSizeInBytes, new MockStreamsMetrics(new Metrics()));
    // trigger a put into another cache on eviction from "name"
    threadCache.addDirtyEntryFlushListener(namespace, new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
            // put an item into an empty cache when the total cache size
            // is already > than maxCacheSizeBytes
            threadCache.put(namespace1, Bytes.wrap(new byte[] { 0 }), dirtyEntry(new byte[2]));
        }
    });
    threadCache.addDirtyEntryFlushListener(namespace1, new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
        // 
        }
    });
    threadCache.addDirtyEntryFlushListener(namespace2, new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
        }
    });
    threadCache.put(namespace2, Bytes.wrap(new byte[] { 1 }), dirtyEntry(new byte[1]));
    threadCache.put(namespace, Bytes.wrap(new byte[] { 1 }), dirtyEntry(new byte[1]));
    // Put a large item such that when the eldest item is removed
    // cache sizeInBytes() > maxCacheSizeBytes
    int remaining = (int) (maxCacheSizeInBytes - threadCache.sizeBytes());
    threadCache.put(namespace, Bytes.wrap(new byte[] { 2 }), dirtyEntry(new byte[remaining + 100]));
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Aggregations

MockStreamsMetrics (org.apache.kafka.streams.processor.internals.MockStreamsMetrics)103 Metrics (org.apache.kafka.common.metrics.Metrics)101 Test (org.junit.Test)59 Before (org.junit.Before)38 LogContext (org.apache.kafka.common.utils.LogContext)32 Bytes (org.apache.kafka.common.utils.Bytes)27 ArrayList (java.util.ArrayList)17 NoOpRecordCollector (org.apache.kafka.test.NoOpRecordCollector)15 InternalMockProcessorContext (org.apache.kafka.test.InternalMockProcessorContext)14 ProcessorRecordContext (org.apache.kafka.streams.processor.internals.ProcessorRecordContext)12 MockProcessorContext (org.apache.kafka.test.MockProcessorContext)10 File (java.io.File)7 RecordHeaders (org.apache.kafka.common.header.internals.RecordHeaders)7 MockTime (org.apache.kafka.common.utils.MockTime)7 KeyValue (org.apache.kafka.streams.KeyValue)7 MockRecordCollector (org.apache.kafka.test.MockRecordCollector)7 StreamsMetricsImpl (org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)6 Properties (java.util.Properties)5 Serializer (org.apache.kafka.common.serialization.Serializer)4 StreamsConfig (org.apache.kafka.streams.StreamsConfig)4