Search in sources :

Example 1 with LocalContainerFactory

use of org.apache.tez.dag.app.rm.LocalTaskSchedulerService.LocalContainerFactory in project tez by apache.

the class TestLocalTaskScheduler method maxTasksAllocationsCannotBeExceeded.

@Test(timeout = 5000)
public void maxTasksAllocationsCannotBeExceeded() {
    final int MAX_TASKS = 4;
    TezConfiguration tezConf = new TezConfiguration();
    tezConf.setInt(TezConfiguration.TEZ_AM_INLINE_TASK_EXECUTION_MAX_TASKS, MAX_TASKS);
    ApplicationId appId = ApplicationId.newInstance(2000, 1);
    ApplicationAttemptId appAttemptId = ApplicationAttemptId.newInstance(appId, 1);
    TaskSchedulerContext mockContext = TestTaskSchedulerHelpers.setupMockTaskSchedulerContext("", 0, "", true, appAttemptId, 1000l, null, new Configuration());
    LocalContainerFactory containerFactory = new LocalContainerFactory(appAttemptId, 1000);
    HashMap<Object, AllocatedTask> taskAllocations = new LinkedHashMap<>();
    LinkedBlockingQueue<SchedulerRequest> clientRequestQueue = new LinkedBlockingQueue<>();
    // Object under test
    AsyncDelegateRequestHandler requestHandler = new AsyncDelegateRequestHandler(clientRequestQueue, containerFactory, taskAllocations, mockContext, tezConf);
    // Allocate up to max tasks
    for (int i = 0; i < MAX_TASKS; i++) {
        Priority priority = Priority.newInstance(20);
        requestHandler.addAllocateTaskRequest(new Long(i), null, priority, null);
        requestHandler.dispatchRequest();
        requestHandler.allocateTask();
    }
    // Only MAX_TASKS number of tasks should have been allocated
    Assert.assertEquals("Wrong number of allocate tasks", MAX_TASKS, taskAllocations.size());
    Assert.assertTrue("Another allocation should not fit", !requestHandler.shouldProcess());
    // Deallocate down to zero
    for (int i = 0; i < MAX_TASKS; i++) {
        requestHandler.addDeallocateTaskRequest(new Long(i));
        requestHandler.dispatchRequest();
    }
    // All allocated tasks should have been removed
    Assert.assertEquals("Wrong number of allocate tasks", 0, taskAllocations.size());
}
Also used : TaskSchedulerContext(org.apache.tez.serviceplugins.api.TaskSchedulerContext) Configuration(org.apache.hadoop.conf.Configuration) TezConfiguration(org.apache.tez.dag.api.TezConfiguration) Priority(org.apache.hadoop.yarn.api.records.Priority) ApplicationAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId) AllocatedTask(org.apache.tez.dag.app.rm.LocalTaskSchedulerService.AllocatedTask) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) LocalContainerFactory(org.apache.tez.dag.app.rm.LocalTaskSchedulerService.LocalContainerFactory) LinkedHashMap(java.util.LinkedHashMap) AsyncDelegateRequestHandler(org.apache.tez.dag.app.rm.LocalTaskSchedulerService.AsyncDelegateRequestHandler) SchedulerRequest(org.apache.tez.dag.app.rm.LocalTaskSchedulerService.SchedulerRequest) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) TezConfiguration(org.apache.tez.dag.api.TezConfiguration) Test(org.junit.Test)

Aggregations

LinkedHashMap (java.util.LinkedHashMap)1 LinkedBlockingQueue (java.util.concurrent.LinkedBlockingQueue)1 Configuration (org.apache.hadoop.conf.Configuration)1 ApplicationAttemptId (org.apache.hadoop.yarn.api.records.ApplicationAttemptId)1 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)1 Priority (org.apache.hadoop.yarn.api.records.Priority)1 TezConfiguration (org.apache.tez.dag.api.TezConfiguration)1 AllocatedTask (org.apache.tez.dag.app.rm.LocalTaskSchedulerService.AllocatedTask)1 AsyncDelegateRequestHandler (org.apache.tez.dag.app.rm.LocalTaskSchedulerService.AsyncDelegateRequestHandler)1 LocalContainerFactory (org.apache.tez.dag.app.rm.LocalTaskSchedulerService.LocalContainerFactory)1 SchedulerRequest (org.apache.tez.dag.app.rm.LocalTaskSchedulerService.SchedulerRequest)1 TaskSchedulerContext (org.apache.tez.serviceplugins.api.TaskSchedulerContext)1 Test (org.junit.Test)1