Search in sources :

Example 36 with ImmutableWorkerInfo

use of org.apache.druid.indexing.overlord.ImmutableWorkerInfo in project druid by druid-io.

the class EqualDistributionWithAffinityWorkerSelectStrategyTest method testFindWorkerForTaskWithNulls.

@Test
public void testFindWorkerForTaskWithNulls() {
    EqualDistributionWorkerSelectStrategy strategy = new EqualDistributionWithAffinityWorkerSelectStrategy(new AffinityConfig(ImmutableMap.of("foo", ImmutableSet.of("localhost")), false));
    ImmutableWorkerInfo worker = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), ImmutableMap.of("lhost", new ImmutableWorkerInfo(new Worker("http", "lhost", "lhost", 1, "v1", WorkerConfig.DEFAULT_CATEGORY), 0, new HashSet<>(), new HashSet<>(), DateTimes.nowUtc()), "localhost", new ImmutableWorkerInfo(new Worker("http", "localhost", "localhost", 1, "v1", WorkerConfig.DEFAULT_CATEGORY), 0, new HashSet<>(), new HashSet<>(), DateTimes.nowUtc())), new NoopTask(null, null, null, 1, 0, null, null, null));
    Assert.assertEquals("lhost", worker.getWorker().getHost());
}
Also used : Worker(org.apache.druid.indexing.worker.Worker) NoopTask(org.apache.druid.indexing.common.task.NoopTask) RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 37 with ImmutableWorkerInfo

use of org.apache.druid.indexing.overlord.ImmutableWorkerInfo in project druid by druid-io.

the class EqualDistributionWorkerSelectStrategyTest method testStrongAffinity.

@Test
public void testStrongAffinity() {
    EqualDistributionWorkerSelectStrategy strategy = new EqualDistributionWorkerSelectStrategy(new AffinityConfig(ImmutableMap.of("foo", ImmutableSet.of("localhost1", "localhost2", "localhost3"), "bar", ImmutableSet.of("nonexistent-worker")), true));
    ImmutableWorkerInfo workerFoo = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("foo"));
    Assert.assertEquals("localhost1", workerFoo.getWorker().getHost());
    // With strong affinity, no workers can be found for bar.
    ImmutableWorkerInfo workerBar = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("bar"));
    Assert.assertNull(workerBar);
    ImmutableWorkerInfo workerBaz = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("baz"));
    Assert.assertEquals("localhost0", workerBaz.getWorker().getHost());
}
Also used : RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 38 with ImmutableWorkerInfo

use of org.apache.druid.indexing.overlord.ImmutableWorkerInfo in project druid by druid-io.

the class EqualDistributionWorkerSelectStrategyTest method testWeakAffinity.

@Test
public void testWeakAffinity() {
    EqualDistributionWorkerSelectStrategy strategy = new EqualDistributionWorkerSelectStrategy(new AffinityConfig(ImmutableMap.of("foo", ImmutableSet.of("localhost1", "localhost2", "localhost3"), "bar", ImmutableSet.of("nonexistent-worker")), false));
    ImmutableWorkerInfo workerFoo = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("foo"));
    Assert.assertEquals("localhost1", workerFoo.getWorker().getHost());
    // With weak affinity, bar (which has no affinity workers available) can use a non-affinity worker.
    ImmutableWorkerInfo workerBar = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("bar"));
    Assert.assertEquals("localhost0", workerBar.getWorker().getHost());
    ImmutableWorkerInfo workerBaz = strategy.findWorkerForTask(new RemoteTaskRunnerConfig(), WORKERS_FOR_AFFINITY_TESTS, createDummyTask("baz"));
    Assert.assertEquals("localhost0", workerBaz.getWorker().getHost());
}
Also used : RemoteTaskRunnerConfig(org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig) ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 39 with ImmutableWorkerInfo

use of org.apache.druid.indexing.overlord.ImmutableWorkerInfo in project druid by druid-io.

the class FillCapacityWithCategorySpecWorkerSelectStrategyTest method testStrongTierSpec.

@Test
public void testStrongTierSpec() {
    final WorkerCategorySpec workerCategorySpec = new WorkerCategorySpec(ImmutableMap.of("noop", new WorkerCategorySpec.CategoryConfig("c1", ImmutableMap.of("ds1", "c3"))), true);
    ImmutableWorkerInfo worker = selectWorker(workerCategorySpec);
    Assert.assertNull(worker);
}
Also used : ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Example 40 with ImmutableWorkerInfo

use of org.apache.druid.indexing.overlord.ImmutableWorkerInfo in project druid by druid-io.

the class FillCapacityWithCategorySpecWorkerSelectStrategyTest method testFindWorkerForTaskWithNullWorkerTierSpec.

@Test
public void testFindWorkerForTaskWithNullWorkerTierSpec() {
    ImmutableWorkerInfo worker = selectWorker(null);
    Assert.assertEquals("localhost3", worker.getWorker().getHost());
}
Also used : ImmutableWorkerInfo(org.apache.druid.indexing.overlord.ImmutableWorkerInfo) Test(org.junit.Test)

Aggregations

ImmutableWorkerInfo (org.apache.druid.indexing.overlord.ImmutableWorkerInfo)42 Test (org.junit.Test)31 RemoteTaskRunnerConfig (org.apache.druid.indexing.overlord.config.RemoteTaskRunnerConfig)14 Worker (org.apache.druid.indexing.worker.Worker)14 NoopTask (org.apache.druid.indexing.common.task.NoopTask)12 TestRemoteTaskRunnerConfig (org.apache.druid.indexing.overlord.TestRemoteTaskRunnerConfig)5 Period (org.joda.time.Period)5 HashMap (java.util.HashMap)4 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)4 WorkerTaskRunner (org.apache.druid.indexing.overlord.WorkerTaskRunner)3 DefaultWorkerBehaviorConfig (org.apache.druid.indexing.overlord.setup.DefaultWorkerBehaviorConfig)3 WorkerBehaviorConfig (org.apache.druid.indexing.overlord.setup.WorkerBehaviorConfig)3 HashSet (java.util.HashSet)2 AtomicReference (java.util.concurrent.atomic.AtomicReference)2 Nullable (javax.annotation.Nullable)2 Response (javax.ws.rs.core.Response)2 ZkWorker (org.apache.druid.indexing.overlord.ZkWorker)2 AutoScaler (org.apache.druid.indexing.overlord.autoscaling.AutoScaler)2 ResourceFilters (com.sun.jersey.spi.container.ResourceFilters)1 GET (javax.ws.rs.GET)1