Search in sources :

Example 6 with ServerStatus

use of org.graylog2.plugin.ServerStatus in project graylog2-server by Graylog2.

the class MessageFilterChainProcessorTest method testFiltersAreOrdered.

@Test
public void testFiltersAreOrdered() {
    final DummyFilter third = new DummyFilter(30);
    final DummyFilter first = new DummyFilter(10);
    final DummyFilter second = new DummyFilter(20);
    final Set<MessageFilter> filters = ImmutableSet.of(third, first, second);
    final MessageFilterChainProcessor processor = new MessageFilterChainProcessor(new MetricRegistry(), filters, acknowledger, serverStatus);
    final List<MessageFilter> filterRegistry = processor.getFilterRegistry();
    Assert.assertEquals(filterRegistry.get(0), first);
    Assert.assertEquals(filterRegistry.get(1), second);
    Assert.assertEquals(filterRegistry.get(2), third);
}
Also used : MetricRegistry(com.codahale.metrics.MetricRegistry) MessageFilter(org.graylog2.plugin.filters.MessageFilter) Test(org.junit.Test)

Example 7 with ServerStatus

use of org.graylog2.plugin.ServerStatus in project graylog2-server by Graylog2.

the class KafkaJournalTest method segmentAgeCleanup.

@Test
public void segmentAgeCleanup() throws Exception {
    final InstantMillisProvider clock = new InstantMillisProvider(DateTime.now(DateTimeZone.UTC));
    DateTimeUtils.setCurrentMillisProvider(clock);
    try {
        final Size segmentSize = Size.kilobytes(1L);
        final KafkaJournal journal = new KafkaJournal(journalDirectory, scheduler, segmentSize, Duration.standardHours(1), Size.kilobytes(10L), Duration.standardMinutes(1), 1_000_000, Duration.standardMinutes(1), 100, new MetricRegistry(), serverStatus);
        final File messageJournalDir = new File(journalDirectory, "messagejournal-0");
        assertTrue(messageJournalDir.exists());
        // we need to fix up the last modified times of the actual files.
        long[] lastModifiedTs = new long[2];
        // create two chunks, 30 seconds apart
        createBulkChunks(journal, segmentSize, 1);
        journal.flushDirtyLogs();
        lastModifiedTs[0] = clock.getMillis();
        clock.tick(Period.seconds(30));
        createBulkChunks(journal, segmentSize, 1);
        journal.flushDirtyLogs();
        lastModifiedTs[1] = clock.getMillis();
        int i = 0;
        for (final LogSegment segment : journal.getSegments()) {
            assertTrue(i < 2);
            segment.lastModified_$eq(lastModifiedTs[i]);
            i++;
        }
        int cleanedLogs = journal.cleanupLogs();
        assertEquals("no segments should've been cleaned", cleanedLogs, 0);
        assertEquals("two segments segment should remain", countSegmentsInDir(messageJournalDir), 2);
        // move clock beyond the retention period and clean again
        clock.tick(Period.seconds(120));
        cleanedLogs = journal.cleanupLogs();
        assertEquals("two segments should've been cleaned (only one will actually be removed...)", cleanedLogs, 2);
        assertEquals("one segment should remain", countSegmentsInDir(messageJournalDir), 1);
    } finally {
        DateTimeUtils.setCurrentMillisSystem();
    }
}
Also used : LogSegment(kafka.log.LogSegment) Size(com.github.joschi.jadconfig.util.Size) InstantMillisProvider(org.graylog2.plugin.InstantMillisProvider) MetricRegistry(com.codahale.metrics.MetricRegistry) File(java.io.File) Test(org.junit.Test)

Example 8 with ServerStatus

use of org.graylog2.plugin.ServerStatus in project graylog2-server by Graylog2.

the class KafkaJournalTest method setUp.

@Before
public void setUp() throws IOException {
    scheduler = new ScheduledThreadPoolExecutor(1);
    scheduler.prestartCoreThread();
    journalDirectory = temporaryFolder.newFolder();
    final File nodeId = temporaryFolder.newFile("node-id");
    Files.write(UUID.randomUUID().toString(), nodeId, StandardCharsets.UTF_8);
    final Configuration configuration = new Configuration() {

        @Override
        public String getNodeIdFile() {
            return nodeId.getAbsolutePath();
        }
    };
    serverStatus = new ServerStatus(configuration, EnumSet.of(ServerStatus.Capability.MASTER), new EventBus("KafkaJournalTest"), NullAuditEventSender::new);
}
Also used : Configuration(org.graylog2.Configuration) ScheduledThreadPoolExecutor(java.util.concurrent.ScheduledThreadPoolExecutor) ServerStatus(org.graylog2.plugin.ServerStatus) EventBus(com.google.common.eventbus.EventBus) File(java.io.File) Before(org.junit.Before)

Example 9 with ServerStatus

use of org.graylog2.plugin.ServerStatus in project graylog2-server by Graylog2.

the class MessageFilterChainProcessorTest method testMessagesCanBeDropped.

@Test
public void testMessagesCanBeDropped() {
    final MessageFilter first = new DummyFilter(10);
    final MessageFilter second = new RemovingMessageFilter();
    final Set<MessageFilter> filters = ImmutableSet.of(first, second);
    final MessageFilterChainProcessor processor = new MessageFilterChainProcessor(new MetricRegistry(), filters, acknowledger, serverStatus);
    final Message message = new Message("message", "source", new DateTime(2016, 1, 1, 0, 0, DateTimeZone.UTC));
    final Messages result = processor.process(message);
    assertThat(result).isEmpty();
}
Also used : Messages(org.graylog2.plugin.Messages) Message(org.graylog2.plugin.Message) MetricRegistry(com.codahale.metrics.MetricRegistry) MessageFilter(org.graylog2.plugin.filters.MessageFilter) DateTime(org.joda.time.DateTime) Test(org.junit.Test)

Example 10 with ServerStatus

use of org.graylog2.plugin.ServerStatus in project graylog2-server by Graylog2.

the class MessageFilterChainProcessorTest method testMessagesRecordProcessingFailures.

@Test
public void testMessagesRecordProcessingFailures() {
    final MessageFilter first = new ExceptingMessageFilter();
    final Set<MessageFilter> filters = ImmutableSet.of(first);
    final MessageFilterChainProcessor processor = new MessageFilterChainProcessor(new MetricRegistry(), filters, acknowledger, serverStatus);
    final Message message = new Message("message", "source", new DateTime(2016, 1, 1, 0, 0, DateTimeZone.UTC));
    final Messages result = processor.process(message);
    assertThat(result).hasSize(1);
    // passed message is mutated, so we can assert on that
    assertThat(message.processingErrors()).hasSize(1);
    assertThat(message.processingErrors().get(0)).satisfies(pe -> {
        assertThat(pe.getCause()).isEqualTo(ProcessingFailureCause.MessageFilterException);
        assertThat(pe.getMessage()).startsWith("Could not apply filter [Excepting filter] on message <");
        assertThat(pe.getDetails()).isEqualTo("BOOM!");
    });
}
Also used : Messages(org.graylog2.plugin.Messages) Message(org.graylog2.plugin.Message) MetricRegistry(com.codahale.metrics.MetricRegistry) MessageFilter(org.graylog2.plugin.filters.MessageFilter) DateTime(org.joda.time.DateTime) Test(org.junit.Test)

Aggregations

MetricRegistry (com.codahale.metrics.MetricRegistry)8 Message (org.graylog2.plugin.Message)6 MessageFilter (org.graylog2.plugin.filters.MessageFilter)6 Test (org.junit.Test)6 File (java.io.File)4 ServerStatus (org.graylog2.plugin.ServerStatus)4 EventBus (com.google.common.eventbus.EventBus)3 Messages (org.graylog2.plugin.Messages)3 DateTime (org.joda.time.DateTime)3 Before (org.junit.Before)3 Size (com.github.joschi.jadconfig.util.Size)2 ScheduledThreadPoolExecutor (java.util.concurrent.ScheduledThreadPoolExecutor)2 Configuration (org.graylog2.Configuration)2 InstantMillisProvider (org.graylog2.plugin.InstantMillisProvider)2 Activity (org.graylog2.shared.system.activities.Activity)2 ActivityWriter (org.graylog2.shared.system.activities.ActivityWriter)2 Timer (com.codahale.metrics.Timer)1 Service (com.google.common.util.concurrent.Service)1 ServiceManager (com.google.common.util.concurrent.ServiceManager)1 ProvisionException (com.google.inject.ProvisionException)1