Search in sources :

Example 1 with ConsumptionResult

use of com.netflix.titus.master.eviction.service.quota.ConsumptionResult in project titus-control-plane by Netflix.

the class JobQuotaController method consume.

@Override
public ConsumptionResult consume(String taskId) {
    if (isLegacy()) {
        return LEGACY;
    }
    Reference taskReference = Reference.task(taskId);
    StringBuilder rejectionResponseBuilder = new StringBuilder("MissingQuotas[");
    // Check quota trackers first
    boolean noQuota = false;
    for (QuotaTracker tracker : quotaTrackers) {
        EvictionQuota quotaStatus = tracker.getQuota(taskReference);
        if (quotaStatus.getQuota() <= 0) {
            noQuota = true;
            rejectionResponseBuilder.append(tracker.getClass().getSimpleName()).append('=').append(quotaStatus.getMessage()).append(", ");
        }
    }
    if (noQuota) {
        rejectionResponseBuilder.setLength(rejectionResponseBuilder.length() - 2);
        return ConsumptionResult.rejected(rejectionResponseBuilder.append(']').toString());
    }
    // Now controllers
    for (int i = 0; i < quotaControllers.size(); i++) {
        QuotaController<Job<?>> controller = quotaControllers.get(i);
        ConsumptionResult result = controller.consume(taskId);
        if (!result.isApproved()) {
            for (int j = 0; j < i; j++) {
                quotaControllers.get(j).giveBackConsumedQuota(taskId);
            }
            rejectionResponseBuilder.append(controller.getClass().getSimpleName()).append('=').append(result.getRejectionReason().orElse("no quota"));
            return ConsumptionResult.rejected(rejectionResponseBuilder.append(']').toString());
        }
    }
    return ConsumptionResult.approved();
}
Also used : TimeWindowQuotaTracker(com.netflix.titus.master.eviction.service.quota.TimeWindowQuotaTracker) QuotaTracker(com.netflix.titus.master.eviction.service.quota.QuotaTracker) EvictionQuota(com.netflix.titus.api.eviction.model.EvictionQuota) ConsumptionResult(com.netflix.titus.master.eviction.service.quota.ConsumptionResult) Reference(com.netflix.titus.api.model.reference.Reference) Job(com.netflix.titus.api.jobmanager.model.job.Job)

Example 2 with ConsumptionResult

use of com.netflix.titus.master.eviction.service.quota.ConsumptionResult in project titus-control-plane by Netflix.

the class JobQuotaControllerTest method testSelfManagedJobUsesInternalDisruptionBudget.

@Test
public void testSelfManagedJobUsesInternalDisruptionBudget() {
    Job<BatchJobExt> job = newBatchJob(10, budget(selfManagedPolicy(1_000), unlimitedRate(), Collections.emptyList()));
    com.netflix.titus.api.model.reference.Reference jobReference = com.netflix.titus.api.model.reference.Reference.job(job.getId());
    scheduleJob(job, 10);
    EffectiveJobDisruptionBudgetResolver budgetResolver = j -> budget(perTaskRelocationLimitPolicy(100), hourlyRatePercentage(5), singletonList(officeHourTimeWindow()));
    JobQuotaController jobController = new JobQuotaController(job, jobOperations, budgetResolver, containerHealthService, titusRuntime);
    assertThat(jobController.getQuota(jobReference).getQuota()).isEqualTo(1);
    Task task = jobOperations.getTasks(job.getId()).get(0);
    assertThat(jobController.consume(task.getId()).isApproved()).isTrue();
    assertThat(jobController.getQuota(jobReference).getQuota()).isEqualTo(0);
    ConsumptionResult failure = jobController.consume(task.getId());
    assertThat(failure.isApproved()).isFalse();
    assertThat(failure.getRejectionReason().get()).contains("JobPercentagePerHourRelocationRateController");
}
Also used : DisruptionBudgetGenerator.numberOfHealthyPolicy(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.numberOfHealthyPolicy) DisruptionBudgetGenerator.unlimitedRate(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.unlimitedRate) ConsumptionResult(com.netflix.titus.master.eviction.service.quota.ConsumptionResult) DisruptionBudgetGenerator.exceptBudget(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.exceptBudget) DisruptionBudgetGenerator.percentageOfHealthyPolicy(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.percentageOfHealthyPolicy) Task(com.netflix.titus.api.jobmanager.model.job.Task) DisruptionBudgetGenerator.budget(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.budget) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) DisruptionBudgetGenerator.exceptRate(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.exceptRate) TimeWindowQuotaTracker(com.netflix.titus.master.eviction.service.quota.TimeWindowQuotaTracker) DisruptionBudgetGenerator.newBatchJob(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.newBatchJob) JobQuotaController.buildQuotaControllers(com.netflix.titus.master.eviction.service.quota.job.JobQuotaController.buildQuotaControllers) DisruptionBudgetGenerator.hourlyRatePercentage(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.hourlyRatePercentage) Collections.singletonList(java.util.Collections.singletonList) DisruptionBudgetGenerator.perTaskRelocationLimitPolicy(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.perTaskRelocationLimitPolicy) TitusRuntimes(com.netflix.titus.common.runtime.TitusRuntimes) QuotaTracker(com.netflix.titus.master.eviction.service.quota.QuotaTracker) JobQuotaController.buildQuotaTrackers(com.netflix.titus.master.eviction.service.quota.job.JobQuotaController.buildQuotaTrackers) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) QuotaController(com.netflix.titus.master.eviction.service.quota.QuotaController) DisruptionBudgetGenerator.exceptPolicy(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.exceptPolicy) DisruptionBudgetGenerator.selfManagedPolicy(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.selfManagedPolicy) Clocks(com.netflix.titus.common.util.time.Clocks) Job(com.netflix.titus.api.jobmanager.model.job.Job) DisruptionBudgetGenerator.officeHourTimeWindow(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.officeHourTimeWindow) Set(java.util.Set) JobFunctions(com.netflix.titus.api.jobmanager.model.job.JobFunctions) Test(org.junit.Test) Collectors(java.util.stream.Collectors) TaskState(com.netflix.titus.api.jobmanager.model.job.TaskState) List(java.util.List) V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) Stream(java.util.stream.Stream) DayOfWeek(java.time.DayOfWeek) JobComponentStub(com.netflix.titus.testkit.model.job.JobComponentStub) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) TestClock(com.netflix.titus.common.util.time.TestClock) DisruptionBudgetGenerator.ratePerInterval(com.netflix.titus.testkit.model.eviction.DisruptionBudgetGenerator.ratePerInterval) ContainerHealthService(com.netflix.titus.api.containerhealth.service.ContainerHealthService) Collections(java.util.Collections) JobQuotaController.mergeQuotaControllers(com.netflix.titus.master.eviction.service.quota.job.JobQuotaController.mergeQuotaControllers) Task(com.netflix.titus.api.jobmanager.model.job.Task) ConsumptionResult(com.netflix.titus.master.eviction.service.quota.ConsumptionResult) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) Test(org.junit.Test)

Example 3 with ConsumptionResult

use of com.netflix.titus.master.eviction.service.quota.ConsumptionResult in project titus-control-plane by Netflix.

the class JobQuotaControllerTest method testConsume.

@Test
public void testConsume() {
    Job<BatchJobExt> job = newBatchJob(10, budget(percentageOfHealthyPolicy(80.0), hourlyRatePercentage(20), Collections.emptyList()));
    com.netflix.titus.api.model.reference.Reference jobReference = com.netflix.titus.api.model.reference.Reference.job(job.getId());
    scheduleJob(job, 10);
    JobQuotaController jobController = new JobQuotaController(job, jobOperations, SelfJobDisruptionBudgetResolver.getInstance(), containerHealthService, titusRuntime);
    assertThat(jobController.getQuota(jobReference).getQuota()).isEqualTo(2);
    Task task = jobOperations.getTasks(job.getId()).get(0);
    assertThat(jobController.consume(task.getId()).isApproved()).isTrue();
    assertThat(jobController.consume(task.getId()).isApproved()).isTrue();
    assertThat(jobController.getQuota(jobReference).getQuota()).isEqualTo(0);
    ConsumptionResult failure = jobController.consume(task.getId());
    assertThat(failure.isApproved()).isFalse();
    assertThat(failure.getRejectionReason().get()).contains("JobPercentagePerHourRelocationRateController");
}
Also used : Task(com.netflix.titus.api.jobmanager.model.job.Task) ConsumptionResult(com.netflix.titus.master.eviction.service.quota.ConsumptionResult) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) Test(org.junit.Test)

Aggregations

ConsumptionResult (com.netflix.titus.master.eviction.service.quota.ConsumptionResult)3 Job (com.netflix.titus.api.jobmanager.model.job.Job)2 Task (com.netflix.titus.api.jobmanager.model.job.Task)2 BatchJobExt (com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt)2 QuotaTracker (com.netflix.titus.master.eviction.service.quota.QuotaTracker)2 TimeWindowQuotaTracker (com.netflix.titus.master.eviction.service.quota.TimeWindowQuotaTracker)2 Test (org.junit.Test)2 ContainerHealthService (com.netflix.titus.api.containerhealth.service.ContainerHealthService)1 EvictionQuota (com.netflix.titus.api.eviction.model.EvictionQuota)1 JobFunctions (com.netflix.titus.api.jobmanager.model.job.JobFunctions)1 TaskState (com.netflix.titus.api.jobmanager.model.job.TaskState)1 V3JobOperations (com.netflix.titus.api.jobmanager.service.V3JobOperations)1 Reference (com.netflix.titus.api.model.reference.Reference)1 TitusRuntime (com.netflix.titus.common.runtime.TitusRuntime)1 TitusRuntimes (com.netflix.titus.common.runtime.TitusRuntimes)1 Clocks (com.netflix.titus.common.util.time.Clocks)1 TestClock (com.netflix.titus.common.util.time.TestClock)1 QuotaController (com.netflix.titus.master.eviction.service.quota.QuotaController)1 JobQuotaController.buildQuotaControllers (com.netflix.titus.master.eviction.service.quota.job.JobQuotaController.buildQuotaControllers)1 JobQuotaController.buildQuotaTrackers (com.netflix.titus.master.eviction.service.quota.job.JobQuotaController.buildQuotaTrackers)1