Search in sources :

Example 1 with Log

use of kafka.log.Log in project graylog2-server by Graylog2.

the class KafkaJournal method flushDirtyLogs.

/**
     * A Java transliteration of what the scala implementation does, which unfortunately is declared as private
     */
protected void flushDirtyLogs() {
    LOG.debug("Checking for dirty logs to flush...");
    final Set<Map.Entry<TopicAndPartition, Log>> entries = JavaConversions.mapAsJavaMap(logManager.logsByTopicPartition()).entrySet();
    for (final Map.Entry<TopicAndPartition, Log> topicAndPartitionLogEntry : entries) {
        final TopicAndPartition topicAndPartition = topicAndPartitionLogEntry.getKey();
        final Log kafkaLog = topicAndPartitionLogEntry.getValue();
        final long timeSinceLastFlush = JODA_TIME.milliseconds() - kafkaLog.lastFlushTime();
        try {
            LOG.debug("Checking if flush is needed on {} flush interval {} last flushed {} time since last flush: {}", topicAndPartition.topic(), kafkaLog.config().flushInterval(), kafkaLog.lastFlushTime(), timeSinceLastFlush);
            if (timeSinceLastFlush >= kafkaLog.config().flushMs()) {
                kafkaLog.flush();
            }
        } catch (Exception e) {
            LOG.error("Error flushing topic " + topicAndPartition.topic(), e);
        }
    }
}
Also used : Log(kafka.log.Log) TopicAndPartition(kafka.common.TopicAndPartition) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) SortedMap(java.util.SortedMap) ClosedByInterruptException(java.nio.channels.ClosedByInterruptException) SyncFailedException(java.io.SyncFailedException) AccessDeniedException(java.nio.file.AccessDeniedException) KafkaException(kafka.common.KafkaException) OffsetOutOfRangeException(kafka.common.OffsetOutOfRangeException) IOException(java.io.IOException)

Aggregations

ImmutableMap (com.google.common.collect.ImmutableMap)1 IOException (java.io.IOException)1 SyncFailedException (java.io.SyncFailedException)1 ClosedByInterruptException (java.nio.channels.ClosedByInterruptException)1 AccessDeniedException (java.nio.file.AccessDeniedException)1 Map (java.util.Map)1 SortedMap (java.util.SortedMap)1 KafkaException (kafka.common.KafkaException)1 OffsetOutOfRangeException (kafka.common.OffsetOutOfRangeException)1 TopicAndPartition (kafka.common.TopicAndPartition)1 Log (kafka.log.Log)1