Search in sources :

Example 1 with FSParentQueue

use of org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue in project hadoop by apache.

the class TestWorkPreservingRMRestart method checkFSQueue.

private void checkFSQueue(ResourceManager rm, SchedulerApplication schedulerApp, Resource usedResources, Resource availableResources) throws Exception {
    // waiting for RM's scheduling apps
    int retry = 0;
    Resource assumedFairShare = Resource.newInstance(8192, 8);
    while (true) {
        Thread.sleep(100);
        if (assumedFairShare.equals(((FairScheduler) rm.getResourceScheduler()).getQueueManager().getRootQueue().getFairShare())) {
            break;
        }
        retry++;
        if (retry > 30) {
            Assert.fail("Apps are not scheduled within assumed timeout");
        }
    }
    FairScheduler scheduler = (FairScheduler) rm.getResourceScheduler();
    FSParentQueue root = scheduler.getQueueManager().getRootQueue();
    // ************ check cluster used Resources ********
    assertTrue(root.getPolicy() instanceof DominantResourceFairnessPolicy);
    assertEquals(usedResources, root.getResourceUsage());
    // ************ check app headroom ****************
    FSAppAttempt schedulerAttempt = (FSAppAttempt) schedulerApp.getCurrentAppAttempt();
    assertEquals(availableResources, schedulerAttempt.getHeadroom());
    // ************ check queue metrics ****************
    QueueMetrics queueMetrics = scheduler.getRootQueueMetrics();
    assertMetrics(queueMetrics, 1, 0, 1, 0, 2, availableResources.getMemorySize(), availableResources.getVirtualCores(), usedResources.getMemorySize(), usedResources.getVirtualCores());
}
Also used : QueueMetrics(org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics) FairScheduler(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler) FSAppAttempt(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt) Resource(org.apache.hadoop.yarn.api.records.Resource) DominantResourceFairnessPolicy(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy) FSParentQueue(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue)

Example 2 with FSParentQueue

use of org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue in project hadoop by apache.

the class FairSchedulerPlanFollower method getPlanResources.

@Override
protected Resource getPlanResources(Plan plan, Queue queue, Resource clusterResources) {
    FSParentQueue planQueue = (FSParentQueue) queue;
    Resource planResources = planQueue.getSteadyFairShare();
    return planResources;
}
Also used : Resource(org.apache.hadoop.yarn.api.records.Resource) FSParentQueue(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue)

Aggregations

Resource (org.apache.hadoop.yarn.api.records.Resource)2 FSParentQueue (org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSParentQueue)2 QueueMetrics (org.apache.hadoop.yarn.server.resourcemanager.scheduler.QueueMetrics)1 FSAppAttempt (org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt)1 FairScheduler (org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler)1 DominantResourceFairnessPolicy (org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.policies.DominantResourceFairnessPolicy)1