Search in sources :

Example 16 with TaskLock

use of org.apache.druid.indexing.common.TaskLock in project druid by druid-io.

the class TaskLocksTest method testCheckSegmentLockCoversSegments.

@Test
public void testCheckSegmentLockCoversSegments() {
    final Set<DataSegment> segments = createNumberedPartitionedSegments();
    final Interval interval = Intervals.of("2017-01-01/2017-01-02");
    final String version = DateTimes.nowUtc().toString();
    final List<TaskLock> locks = IntStream.range(0, 5).mapToObj(partitionId -> {
        final TaskLock lock = trySegmentLock(task, interval, version, partitionId).getTaskLock();
        Assert.assertNotNull(lock);
        return lock;
    }).collect(Collectors.toList());
    Assert.assertEquals(5, locks.size());
    Assert.assertTrue(TaskLocks.isLockCoversSegments(task, lockbox, segments));
}
Also used : IntStream(java.util.stream.IntStream) Intervals(org.apache.druid.java.util.common.Intervals) TaskStorageConfig(org.apache.druid.indexing.common.config.TaskStorageConfig) Function(java.util.function.Function) SpecificSegmentLockRequest(org.apache.druid.indexing.overlord.SpecificSegmentLockRequest) Interval(org.joda.time.Interval) ImmutableList(com.google.common.collect.ImmutableList) TimeChunkLock(org.apache.druid.indexing.common.TimeChunkLock) Task(org.apache.druid.indexing.common.task.Task) Map(java.util.Map) TaskLock(org.apache.druid.indexing.common.TaskLock) Before(org.junit.Before) DateTimes(org.apache.druid.java.util.common.DateTimes) ImmutableSet(com.google.common.collect.ImmutableSet) TaskLockbox(org.apache.druid.indexing.overlord.TaskLockbox) TestIndexerMetadataStorageCoordinator(org.apache.druid.indexing.test.TestIndexerMetadataStorageCoordinator) NumberedShardSpec(org.apache.druid.timeline.partition.NumberedShardSpec) Set(java.util.Set) Test(org.junit.Test) Collectors(java.util.stream.Collectors) NoopTask(org.apache.druid.indexing.common.task.NoopTask) List(java.util.List) HeapMemoryTaskStorage(org.apache.druid.indexing.overlord.HeapMemoryTaskStorage) SegmentLock(org.apache.druid.indexing.common.SegmentLock) TaskLockType(org.apache.druid.indexing.common.TaskLockType) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) LockResult(org.apache.druid.indexing.overlord.LockResult) TimeChunkLockRequest(org.apache.druid.indexing.overlord.TimeChunkLockRequest) Assert(org.junit.Assert) TaskLock(org.apache.druid.indexing.common.TaskLock) DataSegment(org.apache.druid.timeline.DataSegment) Interval(org.joda.time.Interval) Test(org.junit.Test)

Example 17 with TaskLock

use of org.apache.druid.indexing.common.TaskLock in project druid by druid-io.

the class TimeChunkLockTryAcquireActionTest method testWithLockType.

@Test(timeout = 60_000L)
public void testWithLockType() {
    final Task task = NoopTask.create();
    final TimeChunkLockTryAcquireAction action = new TimeChunkLockTryAcquireAction(TaskLockType.EXCLUSIVE, Intervals.of("2017-01-01/2017-01-02"));
    actionTestKit.getTaskLockbox().add(task);
    final TaskLock lock = action.perform(task, actionTestKit.getTaskActionToolbox());
    Assert.assertNotNull(lock);
}
Also used : NoopTask(org.apache.druid.indexing.common.task.NoopTask) Task(org.apache.druid.indexing.common.task.Task) TaskLock(org.apache.druid.indexing.common.TaskLock) Test(org.junit.Test)

Example 18 with TaskLock

use of org.apache.druid.indexing.common.TaskLock in project druid by druid-io.

the class TimeChunkLockAcquireActionTest method testWithLockType.

@Test(timeout = 60_000L)
public void testWithLockType() {
    final Task task = NoopTask.create();
    final TimeChunkLockAcquireAction action = new TimeChunkLockAcquireAction(TaskLockType.EXCLUSIVE, Intervals.of("2017-01-01/2017-01-02"), 1000);
    actionTestKit.getTaskLockbox().add(task);
    final TaskLock lock = action.perform(task, actionTestKit.getTaskActionToolbox());
    Assert.assertNotNull(lock);
}
Also used : NoopTask(org.apache.druid.indexing.common.task.NoopTask) Task(org.apache.druid.indexing.common.task.Task) TaskLock(org.apache.druid.indexing.common.TaskLock) Test(org.junit.Test)

Example 19 with TaskLock

use of org.apache.druid.indexing.common.TaskLock in project druid by druid-io.

the class TimeChunkLockAcquireActionTest method testWithoutLockType.

@Test(timeout = 60_000L)
public void testWithoutLockType() {
    final Task task = NoopTask.create();
    final TimeChunkLockAcquireAction action = new TimeChunkLockAcquireAction(null, Intervals.of("2017-01-01/2017-01-02"), 1000);
    actionTestKit.getTaskLockbox().add(task);
    final TaskLock lock = action.perform(task, actionTestKit.getTaskActionToolbox());
    Assert.assertNotNull(lock);
}
Also used : NoopTask(org.apache.druid.indexing.common.task.NoopTask) Task(org.apache.druid.indexing.common.task.Task) TaskLock(org.apache.druid.indexing.common.TaskLock) Test(org.junit.Test)

Example 20 with TaskLock

use of org.apache.druid.indexing.common.TaskLock in project druid by druid-io.

the class HashPartitionCachingLocalSegmentAllocatorTest method createTaskActionClient.

private static TaskActionClient createTaskActionClient() {
    List<TaskLock> taskLocks = Collections.singletonList(createTaskLock());
    try {
        TaskActionClient taskActionClient = EasyMock.mock(TaskActionClient.class);
        EasyMock.expect(taskActionClient.submit(EasyMock.anyObject(LockListAction.class))).andStubReturn(taskLocks);
        EasyMock.replay(taskActionClient);
        return taskActionClient;
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    }
}
Also used : LockListAction(org.apache.druid.indexing.common.actions.LockListAction) TaskLock(org.apache.druid.indexing.common.TaskLock) TaskActionClient(org.apache.druid.indexing.common.actions.TaskActionClient) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException)

Aggregations

TaskLock (org.apache.druid.indexing.common.TaskLock)49 Task (org.apache.druid.indexing.common.task.Task)29 Test (org.junit.Test)28 NoopTask (org.apache.druid.indexing.common.task.NoopTask)22 Interval (org.joda.time.Interval)22 ISE (org.apache.druid.java.util.common.ISE)21 DataSegment (org.apache.druid.timeline.DataSegment)16 List (java.util.List)15 ArrayList (java.util.ArrayList)13 Map (java.util.Map)13 SegmentLock (org.apache.druid.indexing.common.SegmentLock)13 HashMap (java.util.HashMap)12 Collectors (java.util.stream.Collectors)12 TaskStatus (org.apache.druid.indexer.TaskStatus)12 AbstractTask (org.apache.druid.indexing.common.task.AbstractTask)12 LockGranularity (org.apache.druid.indexing.common.LockGranularity)11 TaskToolbox (org.apache.druid.indexing.common.TaskToolbox)11 TimeChunkLock (org.apache.druid.indexing.common.TimeChunkLock)11 DateTimes (org.apache.druid.java.util.common.DateTimes)11 HashSet (java.util.HashSet)10