Examples with SyslogdConfigFactory org.opennms.netmgt.config.SyslogdConfigFactory used on opensource projects

Search in sources :

Example 11 with SyslogdConfigFactory

use of org.opennms.netmgt.config.SyslogdConfigFactory in project opennms by OpenNMS.

the class Nms4335IT method setUp.

@Before
public void setUp() throws Exception {
    MockLogAppender.setupLogging(true, "TRACE");
    InputStream stream = null;
    try {
        final String config = "<?xml version=\"1.0\"?> \n" + "<syslogd-configuration> \n" + "    <configuration \n" + "            syslog-port=\"10514\" \n" + "            listen-address=\"127.0.0.1\" \n" + "            new-suspect-on-message=\"false\" \n" + "            forwarding-regexp=\"^((.+?) (.*))\\n?$\" \n" + "            matching-group-host=\"2\" \n" + "            matching-group-message=\"3\" \n" + "            discard-uei=\"DISCARD-MATCHING-MESSAGES\" \n" + "            /> \n" + "\n" + "    <!-- Use the following to convert UEI ad-hoc --> \n" + "    <ueiList> \n" + "        <ueiMatch> \n" + "            <match type=\"substr\" expression=\"CRISCO\"/> \n" + "            <uei>CISCO</uei> \n" + "        </ueiMatch> \n" + "        <ueiMatch> \n" + "            <match type=\"regex\" expression=\".*su:auth.*authentication failure.*\"/> \n" + "            <uei>uei.opennms.org/syslog/pam/su/suFailure</uei> \n" + "        </ueiMatch> \n" + "        <!-- Use the following to discard a syslog message without ever creating an event for it. \n" + "             If you change the value of \"discard-uei\" above, you must change the UEI used here to match. --> \n" + "        <ueiMatch> \n" + "            <match type=\"substr\" expression=\"JUNK\"/> \n" + "            <uei>DISCARD-MATCHING-MESSAGES</uei> \n" + "        </ueiMatch> \n" + "    </ueiList> \n" + "\n" + "    <!-- Use the following to remove a syslog message from the event-trail --> \n" + "\n" + "    <hideMessage> \n" + "        <hideMatch> \n" + "            <match type=\"substr\" expression=\"SECRET\"/> \n" + "        </hideMatch> \n" + "        <hideMatch> \n" + "            <match type=\"regex\" expression=\".*(double|triple)secret.*\"/> \n" + "        </hideMatch> \n" + "    </hideMessage> \n" + "\n" + "</syslogd-configuration>\n";
        stream = new ByteArrayInputStream(config.getBytes());
        m_config = new SyslogdConfigFactory(stream);
        m_syslogd = new Syslogd();
        SyslogReceiverJavaNetImpl receiver = new SyslogReceiverJavaNetImpl(m_config);
        receiver.setDistPollerDao(m_distPollerDao);
        receiver.setMessageDispatcherFactory(new MockMessageDispatcherFactory<>());
        m_syslogd.setSyslogReceiver(receiver);
        m_syslogd.init();
    } finally {
        if (stream != null) {
            IOUtils.closeQuietly(stream);
        }
    }
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) SyslogdConfigFactory(org.opennms.netmgt.config.SyslogdConfigFactory) Before(org.junit.Before)

Example 12 with SyslogdConfigFactory

use of org.opennms.netmgt.config.SyslogdConfigFactory in project opennms by OpenNMS.

the class BufferParserTest method testDifferentImplementations.

@Test
public void testDifferentImplementations() throws Exception {
    MockLogAppender.setupLogging(true, "INFO");
    final String abc = "<190>Mar 11 08:35:17 127.0.0.1 30128311[4]: Mar 11 08:35:16.844 CST: %SEC-6-IPACCESSLOGP: list in110 denied tcp 192.168.10.100(63923) -> 192.168.11.128(1521), 1 packet";
    // String abc = "<190>Mar 11 08:35:17 127.0.0.1 30128311: Mar 11 08:35:16.844 CST: %SEC-6-IPACCESSLOGP: list in110 denied tcp 192.168.10.100(63923) -> 192.168.11.128(1521), 1 packet";
    final ByteBuffer incoming = ByteBuffer.wrap(abc.getBytes());
    // final List<ParserStage> grokStages = GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{MONTH:month} %{INT:day} %{STRING:timestamp} %{STRING:timezone} \\%%{STRING:facility}-%{INT:priority}-%{STRING:mnemonic}: %{STRING:message}");
    final List<ParserStage> grokStages = GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}");
    // BufferParserFactory grokFactory = parseGrok("<%{INT:facilityPriority}> %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}: %{MONTH:month} %{INT:day} %{STRING:timestamp} %{STRING:timezone} \\%%{STRING:facility}-%{INT:priority}-%{STRING:mnemonic}: %{STRING:message}");
    final ByteBufferParser<SyslogMessage> grokParser = new SingleSequenceParser(grokStages);
    // SyslogNG format
    final List<ParserStage> parserStages = new ParserStageSequenceBuilder().intBetweenDelimiters('<', '>', (s, v) -> {
        SyslogFacility facility = SyslogFacility.getFacilityForCode(v);
        SyslogSeverity priority = SyslogSeverity.getSeverityForCode(v);
        s.message.setFacility(facility);
        s.message.setSeverity(priority);
    }).whitespace().monthString((s, v) -> {
        s.message.setMonth(v);
    }).whitespace().integer((s, v) -> {
        s.message.setDayOfMonth(v);
    }).whitespace().integer((s, v) -> {
        s.message.setHourOfDay(v);
    }).character(':').integer((s, v) -> {
        s.message.setMinute(v);
    }).character(':').integer((s, v) -> {
        s.message.setSecond(v);
    }).whitespace().stringUntilWhitespace((s, v) -> {
        s.message.setHostName(v);
    }).whitespace().stringUntil("\\s[:", (s, v) -> {
        s.message.setProcessName(v);
    }).optional().character('[').optional().integer((s, v) -> {
        s.message.setProcessId(String.valueOf(v));
    }).optional().character(']').optional().character(':').whitespace().stringUntilWhitespace(// Original month
    null).whitespace().integer(// Original day
    null).whitespace().stringUntilWhitespace(// Original timestamp
    null).whitespace().stringUntilWhitespace(// Original time zone
    null).whitespace().character('%').stringUntilChar('-', null).character('-').stringUntilChar('-', null).character('-').stringUntilChar(':', null).character(':').whitespace().terminal().string((s, v) -> {
        s.message.setMessage(v);
    }).getStages();
    final ByteBufferParser<SyslogMessage> parser = new SingleSequenceParser(parserStages);
    final RadixTreeParser radixParser = new RadixTreeParser();
    // radixParser.teach(grokStages.toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{INT:version} %{STRING:isotimestamp} %{STRING:hostname} %{STRING:processName} %{STRING:processId} %{STRING:messageId} [%{STRING:structureddata}][%{STRING:structureddata}] %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{INT:version} %{STRING:isotimestamp} %{STRING:hostname} %{STRING:processName} %{STRING:processId} %{STRING:messageId} [%{STRING:structureddata}][%{STRING:structureddata}]").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{INT:version} %{STRING:isotimestamp} %{STRING:hostname} %{STRING:processName} %{STRING:processId} %{STRING:messageId} [%{STRING:structureddata}] %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{INT:version} %{STRING:isotimestamp} %{STRING:hostname} %{STRING:processName} %{STRING:processId} %{STRING:messageId} %{STRING:structureddata} %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} [%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} [%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{NOSPACE:processName}: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{STRING:repeatedmonth} %{INT:repeatedday} %{INT:repeatedhour}:%{INT:repeatedminute}:%{INT:repeatedsecond} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{MONTH:month} %{INT:day} %{INT:hour}:%{INT:minute}:%{INT:second} %{STRING:hostname} %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{NOSPACE:processName}[%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} [%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{NOSPACE:processName}: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} [%{INT:processId}]: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{NOSPACE:processName}: %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}>%{NOSPACE:messageId}: %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{STRING:message}").toArray(new ParserStage[0]));
    radixParser.teach(GrokParserStageSequenceBuilder.parseGrok("<%{INT:facilityPriority}> %{INT:year}-%{INT:month}-%{INT:day} %{STRING:hostname} %{STRING:message}").toArray(new ParserStage[0]));
    final int iterations = 100000;
    {
        CompletableFuture<SyslogMessage> event = null;
        Event lastEvent = null;
        long start = System.currentTimeMillis();
        for (int i = 0; i < iterations; i++) {
            event = radixParser.parse(incoming.asReadOnlyBuffer());
            event.whenComplete((e, ex) -> {
                if (ex == null) {
                // System.out.println(e.toString());
                } else {
                    ex.printStackTrace();
                }
            });
        }
        // Wait for the last future to complete
        try {
            event.get();
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }
        long end = System.currentTimeMillis();
        System.out.println("RADIX: " + (end - start) + "ms");
    // System.out.println(lastEvent.toString());
    }
    {
        CompletableFuture<SyslogMessage> event = null;
        long start = System.currentTimeMillis();
        for (int i = 0; i < iterations; i++) {
            event = parser.parse(incoming.asReadOnlyBuffer());
            event.whenComplete((e, ex) -> {
                if (ex == null) {
                // System.out.println(e.toString());
                } else {
                    ex.printStackTrace();
                }
            });
        }
        // Wait for the last future to complete
        try {
            event.get();
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }
        long end = System.currentTimeMillis();
        System.out.println("NEW: " + (end - start) + "ms");
    }
    {
        CompletableFuture<SyslogMessage> event = null;
        long start = System.currentTimeMillis();
        for (int i = 0; i < iterations; i++) {
            event = grokParser.parse(incoming.asReadOnlyBuffer());
            event.whenComplete((e, ex) -> {
                if (ex == null) {
                // System.out.println(e.toString());
                } else {
                    ex.printStackTrace();
                }
            });
        }
        // Wait for the last future to complete
        try {
            event.get();
        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }
        long end = System.currentTimeMillis();
        System.out.println("GROK: " + (end - start) + "ms");
    }
    {
        InputStream stream = ConfigurationTestUtils.getInputStreamForResource(this, "/etc/syslogd-syslogng-configuration.xml");
        SyslogdConfig config = new SyslogdConfigFactory(stream);
        long start = System.currentTimeMillis();
        for (int i = 0; i < iterations; i++) {
            ConvertToEvent convertToEvent = new ConvertToEvent(DistPollerDao.DEFAULT_DIST_POLLER_ID, MonitoringLocationDao.DEFAULT_MONITORING_LOCATION_ID, InetAddressUtils.ONE_TWENTY_SEVEN, 9999, incoming, config);
            Event convertedEvent = convertToEvent.getEvent();
        }
        long end = System.currentTimeMillis();
        System.out.println("OLD: " + (end - start) + "ms");
    }
    {
        InputStream stream = ConfigurationTestUtils.getInputStreamForResource(this, "/etc/syslogd-radix-configuration.xml");
        SyslogdConfig config = new SyslogdConfigFactory(stream);
        long start = System.currentTimeMillis();
        for (int i = 0; i < iterations; i++) {
            ConvertToEvent convertToEvent = new ConvertToEvent(DistPollerDao.DEFAULT_DIST_POLLER_ID, MonitoringLocationDao.DEFAULT_MONITORING_LOCATION_ID, InetAddressUtils.ONE_TWENTY_SEVEN, 9999, incoming, config);
            Event convertedEvent = convertToEvent.getEvent();
        }
        long end = System.currentTimeMillis();
        System.out.println("RADIX CONVERT: " + (end - start) + "ms");
    }
}
Also used : ZonedDateTimeBuilder(org.opennms.core.time.ZonedDateTimeBuilder) SyslogdConfig(org.opennms.netmgt.config.SyslogdConfig) Logger(org.slf4j.Logger) Assert.assertNotNull(org.junit.Assert.assertNotNull) InetAddressUtils(org.opennms.core.utils.InetAddressUtils) MatchChar(org.opennms.netmgt.syslogd.ParserStageSequenceBuilder.MatchChar) LoggerFactory(org.slf4j.LoggerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) Test(org.junit.Test) ByteBuffer(java.nio.ByteBuffer) StandardCharsets(java.nio.charset.StandardCharsets) MockLogAppender(org.opennms.core.test.MockLogAppender) ExecutionException(java.util.concurrent.ExecutionException) List(java.util.List) Assert.assertNull(org.junit.Assert.assertNull) ConfigurationTestUtils(org.opennms.core.test.ConfigurationTestUtils) DistPollerDao(org.opennms.netmgt.dao.api.DistPollerDao) Event(org.opennms.netmgt.xml.event.Event) MonitoringLocationDao(org.opennms.netmgt.dao.api.MonitoringLocationDao) SyslogdConfigFactory(org.opennms.netmgt.config.SyslogdConfigFactory) MatchMonth(org.opennms.netmgt.syslogd.ParserStageSequenceBuilder.MatchMonth) Assert.assertEquals(org.junit.Assert.assertEquals) InputStream(java.io.InputStream) InputStream(java.io.InputStream) SyslogdConfig(org.opennms.netmgt.config.SyslogdConfig) ByteBuffer(java.nio.ByteBuffer) CompletableFuture(java.util.concurrent.CompletableFuture) Event(org.opennms.netmgt.xml.event.Event) SyslogdConfigFactory(org.opennms.netmgt.config.SyslogdConfigFactory) Test(org.junit.Test)

Aggregations

SyslogdConfigFactory (org.opennms.netmgt.config.SyslogdConfigFactory)12 InputStream (java.io.InputStream)11 Test (org.junit.Test)6 SyslogdConfig (org.opennms.netmgt.config.SyslogdConfig)4 ByteArrayInputStream (java.io.ByteArrayInputStream)3 Before (org.junit.Before)3 MetricRegistry (com.codahale.metrics.MetricRegistry)2 ByteBuffer (java.nio.ByteBuffer)2 Calendar (java.util.Calendar)2 GregorianCalendar (java.util.GregorianCalendar)2 DatagramPacket (java.net.DatagramPacket)1 StandardCharsets (java.nio.charset.StandardCharsets)1 Date (java.util.Date)1 List (java.util.List)1 Locale (java.util.Locale)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 ExecutionException (java.util.concurrent.ExecutionException)1 Assert.assertEquals (org.junit.Assert.assertEquals)1 Assert.assertNotNull (org.junit.Assert.assertNotNull)1 Assert.assertNull (org.junit.Assert.assertNull)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial