Search in sources :

Example 16 with NoopTask

use of org.apache.druid.indexing.common.task.NoopTask in project druid by druid-io.

the class FillCapacityWithAffinityWorkerSelectStrategyTest method testIsolation.

@Test
public void testIsolation() {
    FillCapacityWorkerSelectStrategy strategy = new FillCapacityWithAffinityWorkerSelectStrategy(new AffinityConfig(ImmutableMap.of("foo", ImmutableSet.of("localhost")), false));
    ImmutableWorkerInfo worker = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), ImmutableMap.of("localhost", new ImmutableWorkerInfo(new Worker("http", "localhost", "localhost", 1, "v1", WorkerConfig.DEFAULT_CATEGORY), 0, new HashSet<>(), new HashSet<>(), DateTimes.nowUtc())), new NoopTask(null, null, null, 1, 0, null, null, null));
    Assert.assertNull(worker);
}
Also used : Worker(org.apache.druid.indexing.worker.Worker) NoopTask(org.apache.druid.indexing.common.task.NoopTask) RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 17 with NoopTask

use of org.apache.druid.indexing.common.task.NoopTask in project druid by druid-io.

the class OverlordResourceTest method testGetTaskPayload.

@Test
public void testGetTaskPayload() throws Exception {
    // This is disabled since OverlordResource.getTaskStatus() is annotated with TaskResourceFilter which is supposed to
    // set authorization token properly, but isn't called in this test.
    // This should be fixed in https://github.com/apache/druid/issues/6685.
    // expectAuthorizationTokenCheck();
    final NoopTask task = NoopTask.create("mydatasource");
    EasyMock.expect(taskStorageQueryAdapter.getTask("mytask")).andReturn(Optional.of(task));
    EasyMock.expect(taskStorageQueryAdapter.getTask("othertask")).andReturn(Optional.absent());
    EasyMock.replay(taskRunner, taskMaster, taskStorageQueryAdapter, indexerMetadataStorageAdapter, req, workerTaskRunnerQueryAdapter);
    final Response response1 = overlordResource.getTaskPayload("mytask");
    final TaskPayloadResponse taskPayloadResponse1 = TestHelper.makeJsonMapper().readValue(TestHelper.makeJsonMapper().writeValueAsString(response1.getEntity()), TaskPayloadResponse.class);
    Assert.assertEquals(new TaskPayloadResponse("mytask", task), taskPayloadResponse1);
    final Response response2 = overlordResource.getTaskPayload("othertask");
    final TaskPayloadResponse taskPayloadResponse2 = TestHelper.makeJsonMapper().readValue(TestHelper.makeJsonMapper().writeValueAsString(response2.getEntity()), TaskPayloadResponse.class);
    Assert.assertEquals(new TaskPayloadResponse("othertask", null), taskPayloadResponse2);
}
Also used : Response(javax.ws.rs.core.Response) NoopTask(org.apache.druid.indexing.common.task.NoopTask) Test(org.junit.Test)

Example 18 with NoopTask

use of org.apache.druid.indexing.common.task.NoopTask in project druid by druid-io.

the class WorkerTaskManagerTest method testTaskStatusWhenTaskRunnerFutureThrowsException.

@Test(timeout = 30_000L)
public void testTaskStatusWhenTaskRunnerFutureThrowsException() throws Exception {
    Task task = new NoopTask("id", null, null, 100, 0, null, null, ImmutableMap.of(Tasks.PRIORITY_KEY, 0)) {

        @Override
        public TaskStatus run(TaskToolbox toolbox) {
            throw new Error("task failure test");
        }
    };
    workerTaskManager.start();
    workerTaskManager.assignTask(task);
    Map<String, TaskAnnouncement> completeTasks;
    do {
        completeTasks = workerTaskManager.getCompletedTasks();
    } while (completeTasks.isEmpty());
    Assert.assertEquals(1, completeTasks.size());
    TaskAnnouncement announcement = completeTasks.get(task.getId());
    Assert.assertNotNull(announcement);
    Assert.assertEquals(TaskState.FAILED, announcement.getStatus());
    Assert.assertEquals("Failed to run task with an exception. See middleManager or indexer logs for more details.", announcement.getTaskStatus().getErrorMsg());
}
Also used : TaskToolbox(org.apache.druid.indexing.common.TaskToolbox) Task(org.apache.druid.indexing.common.task.Task) NoopTask(org.apache.druid.indexing.common.task.NoopTask) NoopTask(org.apache.druid.indexing.common.task.NoopTask) Test(org.junit.Test)

Example 19 with NoopTask

use of org.apache.druid.indexing.common.task.NoopTask in project druid by druid-io.

the class EqualDistributionWithAffinityWorkerSelectStrategyTest method testFindWorkerForTaskWithNulls.

@Test
public void testFindWorkerForTaskWithNulls() {
    EqualDistributionWorkerSelectStrategy strategy = new EqualDistributionWithAffinityWorkerSelectStrategy(new AffinityConfig(ImmutableMap.of("foo", ImmutableSet.of("localhost")), false));
    ImmutableWorkerInfo worker = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), ImmutableMap.of("lhost", new ImmutableWorkerInfo(new Worker("http", "lhost", "lhost", 1, "v1", WorkerConfig.DEFAULT_CATEGORY), 0, new HashSet<>(), new HashSet<>(), DateTimes.nowUtc()), "localhost", new ImmutableWorkerInfo(new Worker("http", "localhost", "localhost", 1, "v1", WorkerConfig.DEFAULT_CATEGORY), 0, new HashSet<>(), new HashSet<>(), DateTimes.nowUtc())), new NoopTask(null, null, null, 1, 0, null, null, null));
    Assert.assertEquals("lhost", worker.getWorker().getHost());
}
Also used : Worker(org.apache.druid.indexing.worker.Worker) NoopTask(org.apache.druid.indexing.common.task.NoopTask) RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 20 with NoopTask

use of org.apache.druid.indexing.common.task.NoopTask in project druid by druid-io.

the class FillCapacityWithCategorySpecWorkerSelectStrategyTest method selectWorker.

private ImmutableWorkerInfo selectWorker(WorkerCategorySpec workerCategorySpec) {
    final FillCapacityWithCategorySpecWorkerSelectStrategy strategy = new FillCapacityWithCategorySpecWorkerSelectStrategy(workerCategorySpec);
    ImmutableWorkerInfo worker = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_TIER_TESTS, new NoopTask(null, null, "ds1", 1, 0, null, null, null));
    return worker;
}
Also used : NoopTask(org.apache.druid.indexing.common.task.NoopTask) RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo)

Aggregations

NoopTask (org.apache.druid.indexing.common.task.NoopTask)20 Test (org.junit.Test)18 ImmutableWorkerInfo (org.apache.druid.indexing.overlord.ImmutableWorkerInfo)12 RemoteTaskRunnerConfig (org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig)12 Worker (org.apache.druid.indexing.worker.Worker)10 TaskToolbox (org.apache.druid.indexing.common.TaskToolbox)3 Response (javax.ws.rs.core.Response)2 TaskStatus (org.apache.druid.indexer.TaskStatus)2 TaskActionClient (org.apache.druid.indexing.common.actions.TaskActionClient)2 ImmutableList (com.google.common.collect.ImmutableList)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 AtomicReference (java.util.concurrent.atomic.AtomicReference)1 TaskLocation (org.apache.druid.indexer.TaskLocation)1 TaskStatusPlus (org.apache.druid.indexer.TaskStatusPlus)1 Task (org.apache.druid.indexing.common.task.Task)1 IndexerMetadataStorageAdapter (org.apache.druid.indexing.overlord.IndexerMetadataStorageAdapter)1 TaskStorageQueryAdapter (org.apache.druid.indexing.overlord.TaskStorageQueryAdapter)1 WorkerTaskRunnerQueryAdapter (org.apache.druid.indexing.overlord.WorkerTaskRunnerQueryAdapter)1