Search in sources :

Example 6 with TaskLogs

use of io.druid.tasklogs.TaskLogs in project druid by druid-io.

the class FileTaskLogsTest method testKill.

@Test
public void testKill() throws Exception {
    final File tmpDir = temporaryFolder.newFolder();
    final File logDir = new File(tmpDir, "logs");
    final File logFile = new File(tmpDir, "log");
    final TaskLogs taskLogs = new FileTaskLogs(new FileTaskLogsConfig(logDir));
    Files.write("log1content", logFile, Charsets.UTF_8);
    taskLogs.pushTaskLog("log1", logFile);
    Assert.assertEquals("log1content", readLog(taskLogs, "log1", 0));
    //File modification timestamp is only maintained to seconds resolution, so artificial delay
    //is necessary to separate 2 file creations by a timestamp that would result in only one
    //of them getting deleted
    Thread.sleep(1500);
    long time = (System.currentTimeMillis() / 1000) * 1000;
    Assert.assertTrue(new File(logDir, "log1.log").lastModified() < time);
    Files.write("log2content", logFile, Charsets.UTF_8);
    taskLogs.pushTaskLog("log2", logFile);
    Assert.assertEquals("log2content", readLog(taskLogs, "log2", 0));
    Assert.assertTrue(new File(logDir, "log2.log").lastModified() >= time);
    taskLogs.killOlderThan(time);
    Assert.assertFalse(taskLogs.streamTaskLog("log1", 0).isPresent());
    Assert.assertEquals("log2content", readLog(taskLogs, "log2", 0));
}
Also used : TaskLogs(io.druid.tasklogs.TaskLogs) File(java.io.File) FileTaskLogsConfig(io.druid.indexing.common.config.FileTaskLogsConfig) Test(org.junit.Test)

Aggregations

TaskLogs (io.druid.tasklogs.TaskLogs)6 File (java.io.File)6 Test (org.junit.Test)6 FileTaskLogsConfig (io.druid.indexing.common.config.FileTaskLogsConfig)3 HdfsTaskLogs (io.druid.storage.hdfs.tasklog.HdfsTaskLogs)3 HdfsTaskLogsConfig (io.druid.storage.hdfs.tasklog.HdfsTaskLogsConfig)3 Configuration (org.apache.hadoop.conf.Configuration)3 ImmutableMap (com.google.common.collect.ImmutableMap)2 Map (java.util.Map)2 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1