Search in sources :

Example 1 with ReservationSystemTestUtil

use of org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSystemTestUtil in project hadoop by apache.

the class TestGreedyReservationAgent method testStress.

public void testStress(int numJobs) throws PlanningException, IOException {
    long timeWindow = 1000000L;
    Resource clusterCapacity = Resource.newInstance(500 * 100 * 1024, 500 * 32);
    step = 1000L;
    ReservationSystemTestUtil testUtil = new ReservationSystemTestUtil();
    CapacityScheduler scheduler = testUtil.mockCapacityScheduler(500 * 100);
    String reservationQ = ReservationSystemTestUtil.getFullReservationQueueName();
    float instConstraint = 100;
    float avgConstraint = 100;
    ReservationSchedulerConfiguration conf = ReservationSystemTestUtil.createConf(reservationQ, timeWindow, instConstraint, avgConstraint);
    CapacityOverTimePolicy policy = new CapacityOverTimePolicy();
    policy.init(reservationQ, conf);
    RMContext context = ReservationSystemTestUtil.createMockRMContext();
    plan = new InMemoryPlan(scheduler.getRootQueueMetrics(), policy, agent, clusterCapacity, step, res, minAlloc, maxAlloc, "dedicated", null, true, context);
    int acc = 0;
    List<ReservationDefinition> list = new ArrayList<ReservationDefinition>();
    for (long i = 0; i < numJobs; i++) {
        list.add(ReservationSystemTestUtil.generateRandomRR(rand, i));
    }
    long start = System.currentTimeMillis();
    for (int i = 0; i < numJobs; i++) {
        try {
            if (agent.createReservation(ReservationSystemTestUtil.getNewReservationId(), "u" + i % 100, plan, list.get(i))) {
                acc++;
            }
        } catch (PlanningException p) {
        // ignore exceptions
        }
    }
    long end = System.currentTimeMillis();
    System.out.println("Submitted " + numJobs + " jobs " + " accepted " + acc + " in " + (end - start) + "ms");
}
Also used : InMemoryPlan(org.apache.hadoop.yarn.server.resourcemanager.reservation.InMemoryPlan) CapacityOverTimePolicy(org.apache.hadoop.yarn.server.resourcemanager.reservation.CapacityOverTimePolicy) RMContext(org.apache.hadoop.yarn.server.resourcemanager.RMContext) ReservationDefinition(org.apache.hadoop.yarn.api.records.ReservationDefinition) Resource(org.apache.hadoop.yarn.api.records.Resource) ArrayList(java.util.ArrayList) ReservationSchedulerConfiguration(org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSchedulerConfiguration) PlanningException(org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException) ReservationSystemTestUtil(org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSystemTestUtil) CapacityScheduler(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler)

Aggregations

ArrayList (java.util.ArrayList)1 ReservationDefinition (org.apache.hadoop.yarn.api.records.ReservationDefinition)1 Resource (org.apache.hadoop.yarn.api.records.Resource)1 RMContext (org.apache.hadoop.yarn.server.resourcemanager.RMContext)1 CapacityOverTimePolicy (org.apache.hadoop.yarn.server.resourcemanager.reservation.CapacityOverTimePolicy)1 InMemoryPlan (org.apache.hadoop.yarn.server.resourcemanager.reservation.InMemoryPlan)1 ReservationSchedulerConfiguration (org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSchedulerConfiguration)1 ReservationSystemTestUtil (org.apache.hadoop.yarn.server.resourcemanager.reservation.ReservationSystemTestUtil)1 PlanningException (org.apache.hadoop.yarn.server.resourcemanager.reservation.exceptions.PlanningException)1 CapacityScheduler (org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler)1