Search in sources :

Example 1 with MessageQueueException

use of org.graylog2.shared.messageq.MessageQueueException in project graylog2-server by Graylog2.

the class LocalKafkaMessageQueueWriter method write.

@Override
public void write(List<RawMessageEvent> entries) throws MessageQueueException {
    final AtomicLong msgBytes = new AtomicLong(0);
    final List<Journal.Entry> journalEntries = entries.stream().filter(Objects::nonNull).map(e -> new Journal.Entry(e.getMessageIdBytes(), e.getEncodedRawMessage())).peek(e -> msgBytes.addAndGet(e.getMessageBytes().length)).collect(Collectors.toList());
    try {
        writeToJournal(journalEntries);
    } catch (Exception e) {
        LOG.error("Unable to write to journal - retrying", e);
        // Use retryer with exponential back-off to avoid spamming the logs.
        try {
            writeRetryer.call(() -> {
                writeToJournal(journalEntries);
                return null;
            });
        } catch (ExecutionException | RetryException ex) {
            throw new MessageQueueException("Retryer exception", ex);
        }
    }
    metrics.writtenMessages().mark(journalEntries.size());
    metrics.writtenBytes().mark(msgBytes.get());
}
Also used : WaitStrategies(com.github.rholder.retry.WaitStrategies) Logger(org.slf4j.Logger) RawMessageEvent(org.graylog2.shared.buffers.RawMessageEvent) Retryer(com.github.rholder.retry.Retryer) RetryerBuilder(com.github.rholder.retry.RetryerBuilder) Semaphore(java.util.concurrent.Semaphore) RetryException(com.github.rholder.retry.RetryException) LoggerFactory(org.slf4j.LoggerFactory) Journal(org.graylog2.shared.journal.Journal) MessageQueueException(org.graylog2.shared.messageq.MessageQueueException) Singleton(javax.inject.Singleton) Collectors(java.util.stream.Collectors) LocalKafkaJournal(org.graylog2.shared.journal.LocalKafkaJournal) Inject(javax.inject.Inject) Objects(java.util.Objects) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) AtomicLong(java.util.concurrent.atomic.AtomicLong) List(java.util.List) MessageQueueWriter(org.graylog2.shared.messageq.MessageQueueWriter) StopStrategies(com.github.rholder.retry.StopStrategies) AbstractIdleService(com.google.common.util.concurrent.AbstractIdleService) Named(javax.inject.Named) AtomicLong(java.util.concurrent.atomic.AtomicLong) MessageQueueException(org.graylog2.shared.messageq.MessageQueueException) Journal(org.graylog2.shared.journal.Journal) LocalKafkaJournal(org.graylog2.shared.journal.LocalKafkaJournal) RetryException(com.github.rholder.retry.RetryException) MessageQueueException(org.graylog2.shared.messageq.MessageQueueException) ExecutionException(java.util.concurrent.ExecutionException)

Aggregations

RetryException (com.github.rholder.retry.RetryException)1 Retryer (com.github.rholder.retry.Retryer)1 RetryerBuilder (com.github.rholder.retry.RetryerBuilder)1 StopStrategies (com.github.rholder.retry.StopStrategies)1 WaitStrategies (com.github.rholder.retry.WaitStrategies)1 AbstractIdleService (com.google.common.util.concurrent.AbstractIdleService)1 List (java.util.List)1 Objects (java.util.Objects)1 ExecutionException (java.util.concurrent.ExecutionException)1 Semaphore (java.util.concurrent.Semaphore)1 TimeUnit (java.util.concurrent.TimeUnit)1 AtomicLong (java.util.concurrent.atomic.AtomicLong)1 Collectors (java.util.stream.Collectors)1 Inject (javax.inject.Inject)1 Named (javax.inject.Named)1 Singleton (javax.inject.Singleton)1 RawMessageEvent (org.graylog2.shared.buffers.RawMessageEvent)1 Journal (org.graylog2.shared.journal.Journal)1 LocalKafkaJournal (org.graylog2.shared.journal.LocalKafkaJournal)1 MessageQueueException (org.graylog2.shared.messageq.MessageQueueException)1