Search in sources :

Example 1 with NoopTaskLogs

use of org.apache.druid.tasklogs.NoopTaskLogs in project druid by druid-io.

the class ThreadingTaskRunnerTest method testTaskStatusWhenTaskThrowsExceptionWhileRunning.

@Test
public void testTaskStatusWhenTaskThrowsExceptionWhileRunning() throws ExecutionException, InterruptedException {
    ThreadingTaskRunner runner = new ThreadingTaskRunner(mockTaskToolboxFactory(), new TaskConfig(null, null, null, null, ImmutableList.of(), false, new Period("PT0S"), new Period("PT10S"), ImmutableList.of(), false, false, TaskConfig.BATCH_PROCESSING_MODE_DEFAULT.name()), new WorkerConfig(), new NoopTaskLogs(), new DefaultObjectMapper(), new TestAppenderatorsManager(), new MultipleFileTaskReportFileWriter(), new DruidNode("middleManager", "host", false, 8091, null, true, false));
    Future<TaskStatus> statusFuture = runner.run(new AbstractTask("id", "datasource", null) {

        @Override
        public String getType() {
            return "test";
        }

        @Override
        public boolean isReady(TaskActionClient taskActionClient) {
            return true;
        }

        @Override
        public void stopGracefully(TaskConfig taskConfig) {
        }

        @Override
        public TaskStatus run(TaskToolbox toolbox) {
            throw new RuntimeException("Task failure test");
        }
    });
    TaskStatus status = statusFuture.get();
    Assert.assertEquals(TaskState.FAILED, status.getStatusCode());
    Assert.assertEquals("Failed with an exception. See indexer logs for more details.", status.getErrorMsg());
}
Also used : NoopTaskLogs(org.apache.druid.tasklogs.NoopTaskLogs) AbstractTask(org.apache.druid.indexing.common.task.AbstractTask) Period(org.joda.time.Period) TaskConfig(org.apache.druid.indexing.common.config.TaskConfig) TaskStatus(org.apache.druid.indexer.TaskStatus) MultipleFileTaskReportFileWriter(org.apache.druid.indexing.common.MultipleFileTaskReportFileWriter) TaskToolbox(org.apache.druid.indexing.common.TaskToolbox) TaskActionClient(org.apache.druid.indexing.common.actions.TaskActionClient) WorkerConfig(org.apache.druid.indexing.worker.config.WorkerConfig) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) DruidNode(org.apache.druid.server.DruidNode) TestAppenderatorsManager(org.apache.druid.indexing.common.task.TestAppenderatorsManager) Test(org.junit.Test)

Example 2 with NoopTaskLogs

use of org.apache.druid.tasklogs.NoopTaskLogs in project druid by druid-io.

the class ForkingTaskRunnerTest method testTaskStatusWhenTaskProcessSucceedsTaskFails.

@Test
public void testTaskStatusWhenTaskProcessSucceedsTaskFails() throws ExecutionException, InterruptedException {
    ObjectMapper mapper = new DefaultObjectMapper();
    Task task = NoopTask.create();
    ForkingTaskRunner forkingTaskRunner = new ForkingTaskRunner(new ForkingTaskRunnerConfig(), new TaskConfig(null, null, null, null, ImmutableList.of(), false, new Period("PT0S"), new Period("PT10S"), ImmutableList.of(), false, false, TaskConfig.BATCH_PROCESSING_MODE_DEFAULT.name()), new WorkerConfig(), new Properties(), new NoopTaskLogs(), mapper, new DruidNode("middleManager", "host", false, 8091, null, true, false), new StartupLoggingConfig()) {

        @Override
        ProcessHolder runTaskProcess(List<String> command, File logFile, TaskLocation taskLocation) throws IOException {
            ProcessHolder processHolder = Mockito.mock(ProcessHolder.class);
            Mockito.doNothing().when(processHolder).registerWithCloser(ArgumentMatchers.any());
            Mockito.doNothing().when(processHolder).shutdown();
            for (String param : command) {
                if (param.endsWith("status.json")) {
                    mapper.writeValue(new File(param), TaskStatus.failure(task.getId(), "task failure test"));
                    break;
                }
            }
            return processHolder;
        }

        @Override
        int waitForTaskProcessToComplete(Task task, ProcessHolder processHolder, File logFile, File reportsFile) {
            return 0;
        }
    };
    final TaskStatus status = forkingTaskRunner.run(task).get();
    Assert.assertEquals(TaskState.FAILED, status.getStatusCode());
    Assert.assertEquals("task failure test", status.getErrorMsg());
}
Also used : NoopTaskLogs(org.apache.druid.tasklogs.NoopTaskLogs) Task(org.apache.druid.indexing.common.task.Task) NoopTask(org.apache.druid.indexing.common.task.NoopTask) Period(org.joda.time.Period) TaskConfig(org.apache.druid.indexing.common.config.TaskConfig) Properties(java.util.Properties) TaskStatus(org.apache.druid.indexer.TaskStatus) TaskLocation(org.apache.druid.indexer.TaskLocation) StartupLoggingConfig(org.apache.druid.server.log.StartupLoggingConfig) ForkingTaskRunnerConfig(org.apache.druid.indexing.overlord.config.ForkingTaskRunnerConfig) WorkerConfig(org.apache.druid.indexing.worker.config.WorkerConfig) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) DruidNode(org.apache.druid.server.DruidNode) File(java.io.File) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) Test(org.junit.Test)

Example 3 with NoopTaskLogs

use of org.apache.druid.tasklogs.NoopTaskLogs in project druid by druid-io.

the class ForkingTaskRunnerTest method testTaskStatusWhenTaskProcessSucceedsTaskSucceeds.

@Test
public void testTaskStatusWhenTaskProcessSucceedsTaskSucceeds() throws ExecutionException, InterruptedException {
    ObjectMapper mapper = new DefaultObjectMapper();
    Task task = NoopTask.create();
    ForkingTaskRunner forkingTaskRunner = new ForkingTaskRunner(new ForkingTaskRunnerConfig(), new TaskConfig(null, null, null, null, ImmutableList.of(), false, new Period("PT0S"), new Period("PT10S"), ImmutableList.of(), false, false, TaskConfig.BATCH_PROCESSING_MODE_DEFAULT.name()), new WorkerConfig(), new Properties(), new NoopTaskLogs(), mapper, new DruidNode("middleManager", "host", false, 8091, null, true, false), new StartupLoggingConfig()) {

        @Override
        ProcessHolder runTaskProcess(List<String> command, File logFile, TaskLocation taskLocation) throws IOException {
            ProcessHolder processHolder = Mockito.mock(ProcessHolder.class);
            Mockito.doNothing().when(processHolder).registerWithCloser(ArgumentMatchers.any());
            Mockito.doNothing().when(processHolder).shutdown();
            for (String param : command) {
                if (param.endsWith("status.json")) {
                    mapper.writeValue(new File(param), TaskStatus.success(task.getId()));
                    break;
                }
            }
            return processHolder;
        }

        @Override
        int waitForTaskProcessToComplete(Task task, ProcessHolder processHolder, File logFile, File reportsFile) {
            return 0;
        }
    };
    final TaskStatus status = forkingTaskRunner.run(task).get();
    Assert.assertEquals(TaskState.SUCCESS, status.getStatusCode());
    Assert.assertNull(status.getErrorMsg());
}
Also used : NoopTaskLogs(org.apache.druid.tasklogs.NoopTaskLogs) Task(org.apache.druid.indexing.common.task.Task) NoopTask(org.apache.druid.indexing.common.task.NoopTask) Period(org.joda.time.Period) TaskConfig(org.apache.druid.indexing.common.config.TaskConfig) Properties(java.util.Properties) TaskStatus(org.apache.druid.indexer.TaskStatus) TaskLocation(org.apache.druid.indexer.TaskLocation) StartupLoggingConfig(org.apache.druid.server.log.StartupLoggingConfig) ForkingTaskRunnerConfig(org.apache.druid.indexing.overlord.config.ForkingTaskRunnerConfig) WorkerConfig(org.apache.druid.indexing.worker.config.WorkerConfig) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) DruidNode(org.apache.druid.server.DruidNode) File(java.io.File) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) Test(org.junit.Test)

Example 4 with NoopTaskLogs

use of org.apache.druid.tasklogs.NoopTaskLogs in project druid by druid-io.

the class ForkingTaskRunnerTest method testTaskStatusWhenTaskProcessFails.

@Test
public void testTaskStatusWhenTaskProcessFails() throws ExecutionException, InterruptedException {
    ForkingTaskRunner forkingTaskRunner = new ForkingTaskRunner(new ForkingTaskRunnerConfig(), new TaskConfig(null, null, null, null, ImmutableList.of(), false, new Period("PT0S"), new Period("PT10S"), ImmutableList.of(), false, false, TaskConfig.BATCH_PROCESSING_MODE_DEFAULT.name()), new WorkerConfig(), new Properties(), new NoopTaskLogs(), new DefaultObjectMapper(), new DruidNode("middleManager", "host", false, 8091, null, true, false), new StartupLoggingConfig()) {

        @Override
        ProcessHolder runTaskProcess(List<String> command, File logFile, TaskLocation taskLocation) {
            ProcessHolder processHolder = Mockito.mock(ProcessHolder.class);
            Mockito.doNothing().when(processHolder).registerWithCloser(ArgumentMatchers.any());
            Mockito.doNothing().when(processHolder).shutdown();
            return processHolder;
        }

        @Override
        int waitForTaskProcessToComplete(Task task, ProcessHolder processHolder, File logFile, File reportsFile) {
            // Emulate task process failure
            return 1;
        }
    };
    final TaskStatus status = forkingTaskRunner.run(NoopTask.create()).get();
    Assert.assertEquals(TaskState.FAILED, status.getStatusCode());
    Assert.assertEquals("Task execution process exited unsuccessfully with code[1]. See middleManager logs for more details.", status.getErrorMsg());
}
Also used : NoopTaskLogs(org.apache.druid.tasklogs.NoopTaskLogs) Task(org.apache.druid.indexing.common.task.Task) NoopTask(org.apache.druid.indexing.common.task.NoopTask) Period(org.joda.time.Period) TaskConfig(org.apache.druid.indexing.common.config.TaskConfig) Properties(java.util.Properties) TaskStatus(org.apache.druid.indexer.TaskStatus) TaskLocation(org.apache.druid.indexer.TaskLocation) StartupLoggingConfig(org.apache.druid.server.log.StartupLoggingConfig) ForkingTaskRunnerConfig(org.apache.druid.indexing.overlord.config.ForkingTaskRunnerConfig) WorkerConfig(org.apache.druid.indexing.worker.config.WorkerConfig) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) DruidNode(org.apache.druid.server.DruidNode) File(java.io.File) Test(org.junit.Test)

Aggregations

TaskStatus (org.apache.druid.indexer.TaskStatus)4 TaskConfig (org.apache.druid.indexing.common.config.TaskConfig)4 WorkerConfig (org.apache.druid.indexing.worker.config.WorkerConfig)4 DefaultObjectMapper (org.apache.druid.jackson.DefaultObjectMapper)4 DruidNode (org.apache.druid.server.DruidNode)4 NoopTaskLogs (org.apache.druid.tasklogs.NoopTaskLogs)4 Period (org.joda.time.Period)4 Test (org.junit.Test)4 ImmutableList (com.google.common.collect.ImmutableList)3 File (java.io.File)3 List (java.util.List)3 Properties (java.util.Properties)3 TaskLocation (org.apache.druid.indexer.TaskLocation)3 NoopTask (org.apache.druid.indexing.common.task.NoopTask)3 Task (org.apache.druid.indexing.common.task.Task)3 ForkingTaskRunnerConfig (org.apache.druid.indexing.overlord.config.ForkingTaskRunnerConfig)3 StartupLoggingConfig (org.apache.druid.server.log.StartupLoggingConfig)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 MultipleFileTaskReportFileWriter (org.apache.druid.indexing.common.MultipleFileTaskReportFileWriter)1 TaskToolbox (org.apache.druid.indexing.common.TaskToolbox)1