Search in sources :

Example 31 with Metrics

use of org.apache.kafka.common.metrics.Metrics in project kafka by apache.

the class ThreadCacheTest method shouldEvictImmediatelyIfCacheSizeIsVerySmall.

@Test
public void shouldEvictImmediatelyIfCacheSizeIsVerySmall() throws Exception {
    final ThreadCache cache = new ThreadCache("testCache", 1, new MockStreamsMetrics(new Metrics()));
    shouldEvictImmediatelyIfCacheSizeIsZeroOrVerySmall(cache);
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Example 32 with Metrics

use of org.apache.kafka.common.metrics.Metrics in project kafka by apache.

the class RocksDBSegmentedBytesStoreTest method before.

@Before
public void before() {
    bytesStore = new RocksDBSegmentedBytesStore(storeName, retention, numSegments, new SessionKeySchema());
    stateDir = TestUtils.tempDirectory();
    final MockProcessorContext context = new MockProcessorContext(stateDir, Serdes.String(), Serdes.Long(), new NoOpRecordCollector(), new ThreadCache("testCache", 0, new MockStreamsMetrics(new Metrics())));
    bytesStore.init(context, bytesStore);
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) NoOpRecordCollector(org.apache.kafka.test.NoOpRecordCollector) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) MockProcessorContext(org.apache.kafka.test.MockProcessorContext) Before(org.junit.Before)

Example 33 with Metrics

use of org.apache.kafka.common.metrics.Metrics in project kafka by apache.

the class ThreadCacheTest method shouldNotLoopForEverWhenEvictingAndCurrentCacheIsEmpty.

@Test
public void shouldNotLoopForEverWhenEvictingAndCurrentCacheIsEmpty() throws Exception {
    final int maxCacheSizeInBytes = 100;
    final ThreadCache threadCache = new ThreadCache("testCache", maxCacheSizeInBytes, new MockStreamsMetrics(new Metrics()));
    // trigger a put into another cache on eviction from "name"
    threadCache.addDirtyEntryFlushListener("name", new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
            // put an item into an empty cache when the total cache size
            // is already > than maxCacheSizeBytes
            threadCache.put("other", Bytes.wrap(new byte[] { 0 }), dirtyEntry(new byte[2]));
        }
    });
    threadCache.addDirtyEntryFlushListener("other", new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
        //
        }
    });
    threadCache.addDirtyEntryFlushListener("another", new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
        }
    });
    threadCache.put("another", Bytes.wrap(new byte[] { 1 }), dirtyEntry(new byte[1]));
    threadCache.put("name", Bytes.wrap(new byte[] { 1 }), dirtyEntry(new byte[1]));
    // Put a large item such that when the eldest item is removed
    // cache sizeInBytes() > maxCacheSizeBytes
    int remaining = (int) (maxCacheSizeInBytes - threadCache.sizeBytes());
    threadCache.put("name", Bytes.wrap(new byte[] { 2 }), dirtyEntry(new byte[remaining + 100]));
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Example 34 with Metrics

use of org.apache.kafka.common.metrics.Metrics in project kafka by apache.

the class ThreadCacheTest method shouldEvictAfterPutAll.

@Test
public void shouldEvictAfterPutAll() throws Exception {
    final List<ThreadCache.DirtyEntry> received = new ArrayList<>();
    final String namespace = "namespace";
    final ThreadCache cache = new ThreadCache("testCache", 1, new MockStreamsMetrics(new Metrics()));
    cache.addDirtyEntryFlushListener(namespace, new ThreadCache.DirtyEntryFlushListener() {

        @Override
        public void apply(final List<ThreadCache.DirtyEntry> dirty) {
            received.addAll(dirty);
        }
    });
    cache.putAll(namespace, Arrays.asList(KeyValue.pair(Bytes.wrap(new byte[] { 0 }), dirtyEntry(new byte[] { 5 })), KeyValue.pair(Bytes.wrap(new byte[] { 1 }), dirtyEntry(new byte[] { 6 }))));
    assertEquals(cache.evicts(), 2);
    assertEquals(received.size(), 2);
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) ArrayList(java.util.ArrayList) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Example 35 with Metrics

use of org.apache.kafka.common.metrics.Metrics in project kafka by apache.

the class ThreadCacheTest method shouldReturnFalseIfNoNextKey.

@Test
public void shouldReturnFalseIfNoNextKey() throws Exception {
    final ThreadCache cache = new ThreadCache("testCache", 10000L, new MockStreamsMetrics(new Metrics()));
    final ThreadCache.MemoryLRUCacheBytesIterator iterator = cache.range("", Bytes.wrap(new byte[] { 0 }), Bytes.wrap(new byte[] { 1 }));
    assertFalse(iterator.hasNext());
}
Also used : MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Metrics(org.apache.kafka.common.metrics.Metrics) MockStreamsMetrics(org.apache.kafka.streams.processor.internals.MockStreamsMetrics) Test(org.junit.Test)

Aggregations

Metrics (org.apache.kafka.common.metrics.Metrics)103 Test (org.junit.Test)76 MockStreamsMetrics (org.apache.kafka.streams.processor.internals.MockStreamsMetrics)41 HashMap (java.util.HashMap)31 StreamsConfig (org.apache.kafka.streams.StreamsConfig)28 TaskId (org.apache.kafka.streams.processor.TaskId)27 Before (org.junit.Before)22 MockTime (org.apache.kafka.common.utils.MockTime)21 TopicPartition (org.apache.kafka.common.TopicPartition)20 HashSet (java.util.HashSet)19 StreamsMetrics (org.apache.kafka.streams.StreamsMetrics)17 MockClientSupplier (org.apache.kafka.test.MockClientSupplier)17 UUID (java.util.UUID)16 PartitionAssignor (org.apache.kafka.clients.consumer.internals.PartitionAssignor)15 Bytes (org.apache.kafka.common.utils.Bytes)14 MockProcessorSupplier (org.apache.kafka.test.MockProcessorSupplier)14 KStreamBuilder (org.apache.kafka.streams.kstream.KStreamBuilder)13 SubscriptionInfo (org.apache.kafka.streams.processor.internals.assignment.SubscriptionInfo)13 MockProcessorContext (org.apache.kafka.test.MockProcessorContext)13 MockInternalTopicManager (org.apache.kafka.test.MockInternalTopicManager)11