Search in sources :

Example 16 with MetricRegistry

use of com.codahale.metrics.MetricRegistry in project graylog2-server by Graylog2.

the class KafkaJournalTest method segmentRotation.

@Test
public void segmentRotation() throws Exception {
    final Size segmentSize = Size.kilobytes(1L);
    final KafkaJournal journal = new KafkaJournal(journalDirectory, scheduler, segmentSize, Duration.standardHours(1), Size.kilobytes(10L), Duration.standardDays(1), 1_000_000, Duration.standardMinutes(1), 100, new MetricRegistry(), serverStatus);
    createBulkChunks(journal, segmentSize, 3);
    final File[] files = journalDirectory.listFiles();
    assertNotNull(files);
    assertTrue("there should be files in the journal directory", files.length > 0);
    final File[] messageJournalDir = journalDirectory.listFiles((FileFilter) and(directoryFileFilter(), nameFileFilter("messagejournal-0")));
    assertTrue(messageJournalDir.length == 1);
    final File[] logFiles = messageJournalDir[0].listFiles((FileFilter) and(fileFileFilter(), suffixFileFilter(".log")));
    assertEquals("should have two journal segments", 3, logFiles.length);
}
Also used : Size(com.github.joschi.jadconfig.util.Size) MetricRegistry(com.codahale.metrics.MetricRegistry) File(java.io.File) Test(org.junit.Test)

Example 17 with MetricRegistry

use of com.codahale.metrics.MetricRegistry in project graylog2-server by Graylog2.

the class KafkaJournalTest method readAtLeastOne.

@Test
public void readAtLeastOne() throws Exception {
    final Journal journal = new KafkaJournal(journalDirectory, scheduler, Size.megabytes(100L), Duration.standardHours(1), Size.megabytes(5L), Duration.standardHours(1), 1_000_000, Duration.standardMinutes(1), 100, new MetricRegistry(), serverStatus);
    final byte[] idBytes = "id".getBytes(UTF_8);
    final byte[] messageBytes = "message1".getBytes(UTF_8);
    final long position = journal.write(idBytes, messageBytes);
    // Trying to read 0 should always read at least 1 entry.
    final List<Journal.JournalReadEntry> messages = journal.read(0);
    final Journal.JournalReadEntry firstMessage = Iterators.getOnlyElement(messages.iterator());
    assertEquals("message1", new String(firstMessage.getPayload(), UTF_8));
}
Also used : MetricRegistry(com.codahale.metrics.MetricRegistry) Test(org.junit.Test)

Example 18 with MetricRegistry

use of com.codahale.metrics.MetricRegistry in project graylog2-server by Graylog2.

the class KafkaJournalTest method serverStatusThrottledIfJournalUtilizationIsHigherThanThreshold.

@Test
public void serverStatusThrottledIfJournalUtilizationIsHigherThanThreshold() throws Exception {
    serverStatus.running();
    final Size segmentSize = Size.kilobytes(1L);
    final KafkaJournal journal = new KafkaJournal(journalDirectory, scheduler, segmentSize, Duration.standardSeconds(1L), Size.kilobytes(4L), Duration.standardHours(1L), 1_000_000, Duration.standardSeconds(1L), 90, new MetricRegistry(), serverStatus);
    createBulkChunks(journal, segmentSize, 4);
    journal.flushDirtyLogs();
    journal.cleanupLogs();
    assertThat(serverStatus.getLifecycle()).isEqualTo(Lifecycle.THROTTLED);
}
Also used : Size(com.github.joschi.jadconfig.util.Size) MetricRegistry(com.codahale.metrics.MetricRegistry) Test(org.junit.Test)

Example 19 with MetricRegistry

use of com.codahale.metrics.MetricRegistry in project graylog2-server by Graylog2.

the class KafkaJournalTest method writeAndRead.

@Test
public void writeAndRead() throws IOException {
    final Journal journal = new KafkaJournal(journalDirectory, scheduler, Size.megabytes(100L), Duration.standardHours(1), Size.megabytes(5L), Duration.standardHours(1), 1_000_000, Duration.standardMinutes(1), 100, new MetricRegistry(), serverStatus);
    final byte[] idBytes = "id".getBytes(UTF_8);
    final byte[] messageBytes = "message".getBytes(UTF_8);
    final long position = journal.write(idBytes, messageBytes);
    final List<Journal.JournalReadEntry> messages = journal.read(1);
    final Journal.JournalReadEntry firstMessage = Iterators.getOnlyElement(messages.iterator());
    assertEquals("message", new String(firstMessage.getPayload(), UTF_8));
}
Also used : MetricRegistry(com.codahale.metrics.MetricRegistry) Test(org.junit.Test)

Example 20 with MetricRegistry

use of com.codahale.metrics.MetricRegistry in project graylog2-server by Graylog2.

the class KafkaJournalTest method maxSegmentSize.

@Test
public void maxSegmentSize() throws Exception {
    final Size segmentSize = Size.kilobytes(1L);
    final KafkaJournal journal = new KafkaJournal(journalDirectory, scheduler, segmentSize, Duration.standardHours(1), Size.kilobytes(10L), Duration.standardDays(1), 1_000_000, Duration.standardMinutes(1), 100, new MetricRegistry(), serverStatus);
    long size = 0L;
    long maxSize = segmentSize.toBytes();
    final List<Journal.Entry> list = Lists.newArrayList();
    while (size <= maxSize) {
        final byte[] idBytes = ("the1-id").getBytes(UTF_8);
        final byte[] messageBytes = ("the1-message").getBytes(UTF_8);
        size += idBytes.length + messageBytes.length;
        list.add(journal.createEntry(idBytes, messageBytes));
    }
    // Make sure all messages have been written
    assertThat(journal.write(list)).isEqualTo(list.size() - 1);
}
Also used : Size(com.github.joschi.jadconfig.util.Size) MetricRegistry(com.codahale.metrics.MetricRegistry) Test(org.junit.Test)

Aggregations

MetricRegistry (com.codahale.metrics.MetricRegistry)505 Test (org.junit.Test)177 Before (org.junit.Before)61 Test (org.junit.jupiter.api.Test)45 VerifiableProperties (com.github.ambry.config.VerifiableProperties)42 ArrayList (java.util.ArrayList)33 Counter (com.codahale.metrics.Counter)30 File (java.io.File)29 Properties (java.util.Properties)28 List (java.util.List)23 Metric (com.codahale.metrics.Metric)22 Map (java.util.Map)22 IOException (java.io.IOException)21 HashMap (java.util.HashMap)20 Size (com.github.joschi.jadconfig.util.Size)17 CountDownLatch (java.util.concurrent.CountDownLatch)17 TimeUnit (java.util.concurrent.TimeUnit)17 Timer (com.codahale.metrics.Timer)15 DefaultTaggedMetricRegistry (com.palantir.tritium.metrics.registry.DefaultTaggedMetricRegistry)15 ResourceConfig (org.glassfish.jersey.server.ResourceConfig)15