Search in sources :

Example 1 with LogMessage

use of io.cdap.cdap.logging.appender.LogMessage in project cdap by caskdata.

the class LogAppenderLogProcessor method process.

@Override
public void process(Iterator<byte[]> loggingEventBytes) {
    LoggingEventSerializer serializer = LOGGING_EVENT_SERIALIZER.get();
    loggingEventBytes.forEachRemaining(bytes -> {
        try {
            ILoggingEvent iLoggingEvent = serializer.fromBytes(ByteBuffer.wrap(bytes));
            LoggingContext loggingContext = LoggingContextHelper.getLoggingContext(iLoggingEvent.getMDCPropertyMap());
            if (loggingContext == null) {
                // This shouldn't happen
                LOG.debug("Ignore logging event due to missing logging context: {}", iLoggingEvent);
                return;
            }
            logAppender.append(new LogMessage(iLoggingEvent, loggingContext));
        } catch (IOException e) {
            LOG.warn("Ignore logging event due to decode failure: {}", e.getMessage());
            LOG.debug("Ignore logging event stack trace", e);
        }
    });
}
Also used : LoggingContext(io.cdap.cdap.common.logging.LoggingContext) LogMessage(io.cdap.cdap.logging.appender.LogMessage) LoggingEventSerializer(io.cdap.cdap.logging.serialize.LoggingEventSerializer) IOException(java.io.IOException) ILoggingEvent(ch.qos.logback.classic.spi.ILoggingEvent)

Example 2 with LogMessage

use of io.cdap.cdap.logging.appender.LogMessage in project cdap by caskdata.

the class LogBufferRecoveryServiceTest method createLoggingEvent.

private ILoggingEvent createLoggingEvent(String loggerName, Level level, String message, long timestamp, LoggingContext loggingContext) {
    LoggingEvent event = new LoggingEvent();
    event.setLevel(level);
    event.setLoggerName(loggerName);
    event.setMessage(message);
    event.setTimeStamp(timestamp);
    return new LogMessage(event, loggingContext);
}
Also used : ILoggingEvent(ch.qos.logback.classic.spi.ILoggingEvent) LoggingEvent(ch.qos.logback.classic.spi.LoggingEvent) LogMessage(io.cdap.cdap.logging.appender.LogMessage)

Example 3 with LogMessage

use of io.cdap.cdap.logging.appender.LogMessage in project cdap by caskdata.

the class KafkaLogProcessorPipelineTest method publishLog.

private void publishLog(String topic, Iterable<ILoggingEvent> events, LoggingContext context) {
    KafkaPublisher.Preparer preparer = KAFKA_TESTER.getKafkaClient().getPublisher(KafkaPublisher.Ack.LEADER_RECEIVED, Compression.NONE).prepare(topic);
    LoggingEventSerializer serializer = new LoggingEventSerializer();
    for (ILoggingEvent event : events) {
        preparer.add(ByteBuffer.wrap(serializer.toBytes(new LogMessage(event, context))), context.getLogPartition());
    }
    preparer.send();
}
Also used : LogMessage(io.cdap.cdap.logging.appender.LogMessage) LoggingEventSerializer(io.cdap.cdap.logging.serialize.LoggingEventSerializer) ILoggingEvent(ch.qos.logback.classic.spi.ILoggingEvent) KafkaPublisher(org.apache.twill.kafka.client.KafkaPublisher)

Example 4 with LogMessage

use of io.cdap.cdap.logging.appender.LogMessage in project cdap by caskdata.

the class LogBufferReaderTest method createLoggingEvent.

private ILoggingEvent createLoggingEvent(String loggerName, Level level, String message, long timestamp, LoggingContext loggingContext) {
    LoggingEvent event = new LoggingEvent();
    event.setLevel(level);
    event.setLoggerName(loggerName);
    event.setMessage(message);
    event.setTimeStamp(timestamp);
    return new LogMessage(event, loggingContext);
}
Also used : ILoggingEvent(ch.qos.logback.classic.spi.ILoggingEvent) LoggingEvent(ch.qos.logback.classic.spi.LoggingEvent) LogMessage(io.cdap.cdap.logging.appender.LogMessage)

Example 5 with LogMessage

use of io.cdap.cdap.logging.appender.LogMessage in project cdap by caskdata.

the class DistributedLogFrameworkTest method publishLog.

/**
 * Publishes multiple log events.
 */
private void publishLog(String topic, LoggingContext context, Iterable<ILoggingEvent> events) {
    KafkaPublisher.Preparer preparer = KAFKA_TESTER.getKafkaClient().getPublisher(KafkaPublisher.Ack.LEADER_RECEIVED, Compression.NONE).prepare(topic);
    LoggingEventSerializer serializer = new LoggingEventSerializer();
    for (ILoggingEvent event : events) {
        preparer.add(ByteBuffer.wrap(serializer.toBytes(new LogMessage(event, context))), context.getLogPartition());
    }
    preparer.send();
}
Also used : LogMessage(io.cdap.cdap.logging.appender.LogMessage) LoggingEventSerializer(io.cdap.cdap.logging.serialize.LoggingEventSerializer) ILoggingEvent(ch.qos.logback.classic.spi.ILoggingEvent) KafkaPublisher(org.apache.twill.kafka.client.KafkaPublisher)

Aggregations

ILoggingEvent (ch.qos.logback.classic.spi.ILoggingEvent)15 LogMessage (io.cdap.cdap.logging.appender.LogMessage)15 LoggingEvent (ch.qos.logback.classic.spi.LoggingEvent)6 Test (org.junit.Test)6 Logger (org.slf4j.Logger)4 LoggingEventSerializer (io.cdap.cdap.logging.serialize.LoggingEventSerializer)3 LoggerContextVO (ch.qos.logback.classic.spi.LoggerContextVO)2 IOException (java.io.IOException)2 KafkaPublisher (org.apache.twill.kafka.client.KafkaPublisher)2 LoggerContext (ch.qos.logback.classic.LoggerContext)1 IThrowableProxy (ch.qos.logback.classic.spi.IThrowableProxy)1 ThrowableProxy (ch.qos.logback.classic.spi.ThrowableProxy)1 HttpExceptionHandler (io.cdap.cdap.common.HttpExceptionHandler)1 CConfiguration (io.cdap.cdap.common.conf.CConfiguration)1 LoggingContext (io.cdap.cdap.common.logging.LoggingContext)1 RemoteLogAppender (io.cdap.cdap.logging.appender.remote.RemoteLogAppender)1 WorkerLoggingContext (io.cdap.cdap.logging.context.WorkerLoggingContext)1 ConcurrentLogBufferWriter (io.cdap.cdap.logging.logbuffer.ConcurrentLogBufferWriter)1 MockAppender (io.cdap.cdap.logging.pipeline.MockAppender)1 LogBufferProcessorPipeline (io.cdap.cdap.logging.pipeline.logbuffer.LogBufferProcessorPipeline)1