Search in sources :

Example 1 with NoOpMetricsCollectionService

use of co.cask.cdap.common.metrics.NoOpMetricsCollectionService in project cdap by caskdata.

the class RollingLocationLogAppenderTest method testFileClose.

@Test
public void testFileClose() throws Exception {
    // assume SLF4J is bound to logback in the current environment
    AppenderContext appenderContext = new LocalAppenderContext(injector.getInstance(DatasetFramework.class), injector.getInstance(TransactionSystemClient.class), injector.getInstance(LocationFactory.class), new NoOpMetricsCollectionService());
    JoranConfigurator configurator = new JoranConfigurator();
    configurator.setContext(appenderContext);
    // Call context.reset() to clear any previous configuration, e.g. default
    // configuration. For multi-step configuration, omit calling context.reset().
    appenderContext.reset();
    configurator.doConfigure(getClass().getResourceAsStream("/rolling-appender-logback-test.xml"));
    StatusPrinter.printInCaseOfErrorsOrWarnings(appenderContext);
    RollingLocationLogAppender rollingAppender = (RollingLocationLogAppender) appenderContext.getLogger(RollingLocationLogAppenderTest.class).getAppender("rollingAppender");
    addTagsToMdc("testNs", "testApp");
    Logger logger = appenderContext.getLogger(RollingLocationLogAppenderTest.class);
    ingestLogs(logger, 20);
    // wait for 500 ms so that file is eligible for closing
    Thread.sleep(500);
    // flush to make sure file is closed
    rollingAppender.flush();
    Assert.assertEquals(0, rollingAppender.getLocationManager().getActiveLocations().size());
}
Also used : DatasetFramework(co.cask.cdap.data2.dataset2.DatasetFramework) TransactionSystemClient(org.apache.tephra.TransactionSystemClient) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) JoranConfigurator(ch.qos.logback.classic.joran.JoranConfigurator) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) AppenderContext(co.cask.cdap.api.logging.AppenderContext) NoOpMetricsCollectionService(co.cask.cdap.common.metrics.NoOpMetricsCollectionService) Logger(org.slf4j.Logger) LocationFactory(org.apache.twill.filesystem.LocationFactory) Test(org.junit.Test)

Example 2 with NoOpMetricsCollectionService

use of co.cask.cdap.common.metrics.NoOpMetricsCollectionService in project cdap by caskdata.

the class RollingLocationLogAppenderTest method testRollingLocationLogAppender.

@Test
public void testRollingLocationLogAppender() throws Exception {
    // assume SLF4J is bound to logback in the current environment
    AppenderContext appenderContext = new LocalAppenderContext(injector.getInstance(DatasetFramework.class), injector.getInstance(TransactionSystemClient.class), injector.getInstance(LocationFactory.class), new NoOpMetricsCollectionService());
    JoranConfigurator configurator = new JoranConfigurator();
    configurator.setContext(appenderContext);
    // Call context.reset() to clear any previous configuration, e.g. default
    // configuration. For multi-step configuration, omit calling context.reset().
    appenderContext.reset();
    configurator.doConfigure(getClass().getResourceAsStream("/rolling-appender-logback-test.xml"));
    StatusPrinter.printInCaseOfErrorsOrWarnings(appenderContext);
    RollingLocationLogAppender rollingAppender = (RollingLocationLogAppender) appenderContext.getLogger(RollingLocationLogAppenderTest.class).getAppender("rollingAppender");
    addTagsToMdc("testNamespace", "testApp");
    Logger logger = appenderContext.getLogger(RollingLocationLogAppenderTest.class);
    ingestLogs(logger, 5);
    Map<LocationIdentifier, LocationOutputStream> activeFiles = rollingAppender.getLocationManager().getActiveLocations();
    Assert.assertEquals(1, activeFiles.size());
    verifyFileOutput(activeFiles, 5);
    // different program should go to different directory
    addTagsToMdc("testNamespace", "testApp1");
    ingestLogs(logger, 5);
    activeFiles = rollingAppender.getLocationManager().getActiveLocations();
    Assert.assertEquals(2, activeFiles.size());
    verifyFileOutput(activeFiles, 5);
    // different program should go to different directory because namespace is different
    addTagsToMdc("testNamespace1", "testApp1");
    ingestLogs(logger, 5);
    activeFiles = rollingAppender.getLocationManager().getActiveLocations();
    Assert.assertEquals(3, activeFiles.size());
    verifyFileOutput(activeFiles, 5);
}
Also used : DatasetFramework(co.cask.cdap.data2.dataset2.DatasetFramework) TransactionSystemClient(org.apache.tephra.TransactionSystemClient) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) JoranConfigurator(ch.qos.logback.classic.joran.JoranConfigurator) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) AppenderContext(co.cask.cdap.api.logging.AppenderContext) NoOpMetricsCollectionService(co.cask.cdap.common.metrics.NoOpMetricsCollectionService) Logger(org.slf4j.Logger) LocationFactory(org.apache.twill.filesystem.LocationFactory) Test(org.junit.Test)

Example 3 with NoOpMetricsCollectionService

use of co.cask.cdap.common.metrics.NoOpMetricsCollectionService in project cdap by caskdata.

the class CDAPLogAppenderTest method testCDAPLogAppender.

@Test
public void testCDAPLogAppender() throws Exception {
    int syncInterval = 1024 * 1024;
    CDAPLogAppender cdapLogAppender = new CDAPLogAppender();
    cdapLogAppender.setSyncIntervalBytes(syncInterval);
    cdapLogAppender.setMaxFileLifetimeMs(TimeUnit.DAYS.toMillis(1));
    cdapLogAppender.setMaxFileSizeInBytes(104857600);
    cdapLogAppender.setDirPermissions("700");
    cdapLogAppender.setFilePermissions("600");
    cdapLogAppender.setFileRetentionDurationDays(1);
    cdapLogAppender.setLogCleanupIntervalMins(10);
    cdapLogAppender.setFileCleanupTransactionTimeout(30);
    AppenderContext context = new LocalAppenderContext(injector.getInstance(DatasetFramework.class), injector.getInstance(TransactionSystemClient.class), injector.getInstance(LocationFactory.class), new NoOpMetricsCollectionService());
    context.start();
    cdapLogAppender.setContext(context);
    cdapLogAppender.start();
    FileMetaDataReader fileMetaDataReader = injector.getInstance(FileMetaDataReader.class);
    LoggingEvent event = new LoggingEvent("co.cask.Test", (ch.qos.logback.classic.Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME), Level.ERROR, "test message", null, null);
    Map<String, String> properties = new HashMap<>();
    properties.put(NamespaceLoggingContext.TAG_NAMESPACE_ID, "default");
    properties.put(ApplicationLoggingContext.TAG_APPLICATION_ID, "testApp");
    properties.put(FlowletLoggingContext.TAG_FLOW_ID, "testFlow");
    properties.put(FlowletLoggingContext.TAG_FLOWLET_ID, "testFlowlet");
    event.setMDCPropertyMap(properties);
    cdapLogAppender.doAppend(event);
    cdapLogAppender.stop();
    context.stop();
    try {
        List<LogLocation> files = fileMetaDataReader.listFiles(cdapLogAppender.getLoggingPath(properties), 0, Long.MAX_VALUE);
        Assert.assertEquals(1, files.size());
        LogLocation logLocation = files.get(0);
        Assert.assertEquals(LogLocation.VERSION_1, logLocation.getFrameworkVersion());
        Assert.assertTrue(logLocation.getLocation().exists());
        CloseableIterator<LogEvent> logEventCloseableIterator = logLocation.readLog(Filter.EMPTY_FILTER, 0, Long.MAX_VALUE, Integer.MAX_VALUE);
        int logCount = 0;
        while (logEventCloseableIterator.hasNext()) {
            logCount++;
            LogEvent logEvent = logEventCloseableIterator.next();
            Assert.assertEquals(event.getMessage(), logEvent.getLoggingEvent().getMessage());
        }
        logEventCloseableIterator.close();
        Assert.assertEquals(1, logCount);
        // checking permission
        String expectedPermissions = "rw-------";
        for (LogLocation file : files) {
            Location location = file.getLocation();
            Assert.assertEquals(expectedPermissions, location.getPermissions());
        }
    } catch (Exception e) {
        Assert.fail();
    }
}
Also used : HashMap(java.util.HashMap) LogEvent(co.cask.cdap.logging.read.LogEvent) NoOpMetricsCollectionService(co.cask.cdap.common.metrics.NoOpMetricsCollectionService) IOException(java.io.IOException) LocationFactory(org.apache.twill.filesystem.LocationFactory) DatasetFramework(co.cask.cdap.data2.dataset2.DatasetFramework) LoggingEvent(ch.qos.logback.classic.spi.LoggingEvent) TransactionSystemClient(org.apache.tephra.TransactionSystemClient) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) LogLocation(co.cask.cdap.logging.write.LogLocation) LocalAppenderContext(co.cask.cdap.logging.framework.LocalAppenderContext) AppenderContext(co.cask.cdap.api.logging.AppenderContext) FileMetaDataReader(co.cask.cdap.logging.meta.FileMetaDataReader) Location(org.apache.twill.filesystem.Location) LogLocation(co.cask.cdap.logging.write.LogLocation) Test(org.junit.Test)

Example 4 with NoOpMetricsCollectionService

use of co.cask.cdap.common.metrics.NoOpMetricsCollectionService in project cdap by caskdata.

the class DefaultServiceConfigurer method createHttpHandler.

private <T extends HttpServiceHandler> HttpHandler createHttpHandler(T handler) {
    MetricsContext noOpsMetricsContext = new NoOpMetricsCollectionService().getContext(new HashMap<String, String>());
    HttpHandlerFactory factory = new HttpHandlerFactory("", noOpsMetricsContext);
    @SuppressWarnings("unchecked") TypeToken<T> type = (TypeToken<T>) TypeToken.of(handler.getClass());
    return factory.createHttpHandler(type, new VerificationDelegateContext<>(handler));
}
Also used : HttpHandlerFactory(co.cask.cdap.internal.app.runtime.service.http.HttpHandlerFactory) TypeToken(com.google.common.reflect.TypeToken) MetricsContext(co.cask.cdap.api.metrics.MetricsContext) NoOpMetricsCollectionService(co.cask.cdap.common.metrics.NoOpMetricsCollectionService)

Example 5 with NoOpMetricsCollectionService

use of co.cask.cdap.common.metrics.NoOpMetricsCollectionService in project cdap by caskdata.

the class MessagingHttpServiceTest method init.

@BeforeClass
public static void init() throws IOException {
    cConf = CConfiguration.create();
    cConf.set(Constants.CFG_LOCAL_DATA_DIR, TEMP_FOLDER.newFolder().getAbsolutePath());
    cConf.setInt(Constants.MessagingSystem.HTTP_SERVER_CONSUME_CHUNK_SIZE, 128);
    // Set max life time to a high value so that dummy tx ids that we create in the tests still work
    cConf.setLong(TxConstants.Manager.CFG_TX_MAX_LIFETIME, 10000000000L);
    Injector injector = Guice.createInjector(new ConfigModule(cConf), new DiscoveryRuntimeModule().getInMemoryModules(), new MessagingServerRuntimeModule().getInMemoryModules(), new AbstractModule() {

        @Override
        protected void configure() {
            bind(MetricsCollectionService.class).toInstance(new NoOpMetricsCollectionService());
        }
    });
    httpService = injector.getInstance(MessagingHttpService.class);
    httpService.startAndWait();
    client = new ClientMessagingService(injector.getInstance(DiscoveryServiceClient.class));
}
Also used : Injector(com.google.inject.Injector) ConfigModule(co.cask.cdap.common.guice.ConfigModule) MessagingServerRuntimeModule(co.cask.cdap.messaging.guice.MessagingServerRuntimeModule) ClientMessagingService(co.cask.cdap.messaging.client.ClientMessagingService) NoOpMetricsCollectionService(co.cask.cdap.common.metrics.NoOpMetricsCollectionService) DiscoveryRuntimeModule(co.cask.cdap.common.guice.DiscoveryRuntimeModule) AbstractModule(com.google.inject.AbstractModule) BeforeClass(org.junit.BeforeClass)

Aggregations

NoOpMetricsCollectionService (co.cask.cdap.common.metrics.NoOpMetricsCollectionService)10 DatasetFramework (co.cask.cdap.data2.dataset2.DatasetFramework)7 AppenderContext (co.cask.cdap.api.logging.AppenderContext)6 LocalAppenderContext (co.cask.cdap.logging.framework.LocalAppenderContext)6 TransactionSystemClient (org.apache.tephra.TransactionSystemClient)6 LocationFactory (org.apache.twill.filesystem.LocationFactory)6 Test (org.junit.Test)6 JoranConfigurator (ch.qos.logback.classic.joran.JoranConfigurator)3 LoggingEvent (ch.qos.logback.classic.spi.LoggingEvent)3 ConfigModule (co.cask.cdap.common.guice.ConfigModule)3 DiscoveryRuntimeModule (co.cask.cdap.common.guice.DiscoveryRuntimeModule)3 FileMetaDataReader (co.cask.cdap.logging.meta.FileMetaDataReader)3 LogLocation (co.cask.cdap.logging.write.LogLocation)3 AbstractModule (com.google.inject.AbstractModule)3 IOException (java.io.IOException)3 HashMap (java.util.HashMap)3 Location (org.apache.twill.filesystem.Location)3 MessagingServerRuntimeModule (co.cask.cdap.messaging.guice.MessagingServerRuntimeModule)2 Logger (org.slf4j.Logger)2 MetricsCollectionService (co.cask.cdap.api.metrics.MetricsCollectionService)1