Search in sources :

Example 6 with ApplicationSLA

use of com.netflix.titus.api.model.ApplicationSLA in project titus-control-plane by Netflix.

the class ReservationUsageCalculator method buildCapacityGroupUsage.

public ReservationUsage buildCapacityGroupUsage(String capacityGroupName) {
    ApplicationSLA capacityGroup = capacityManagementService.getApplicationSLA(capacityGroupName);
    if (capacityGroup == null) {
        return ReservationUsage.none();
    }
    List<Pair<Job, List<Task>>> jobsAndTasks = jobOperations.getJobsAndTasks();
    ResourceAccumulator accumulator = new ResourceAccumulator();
    for (Pair<Job, List<Task>> jobAndTasks : jobsAndTasks) {
        Job job = jobAndTasks.getLeft();
        if (capacityGroupName.equals(job.getJobDescriptor().getCapacityGroup())) {
            processJob(accumulator, jobAndTasks);
        }
    }
    return accumulator.toReservationUsage();
}
Also used : Task(com.netflix.titus.api.jobmanager.model.job.Task) ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA) List(java.util.List) Job(com.netflix.titus.api.jobmanager.model.job.Job) Pair(com.netflix.titus.common.util.tuple.Pair)

Example 7 with ApplicationSLA

use of com.netflix.titus.api.model.ApplicationSLA in project titus-control-plane by Netflix.

the class BatchDifferenceResolver method createNewTaskAction.

private Optional<TitusChangeAction> createNewTaskAction(BatchJobView refJobView, int taskIndex, Optional<EntityHolder> previousTask, List<String> unassignedIpAllocations, List<String> ebsVolumeIds) {
    // Safety check
    long numberOfNotFinishedTasks = refJobView.getJobHolder().getChildren().stream().filter(holder -> TaskState.isRunning(((Task) holder.getEntity()).getStatus().getState())).count();
    if (numberOfNotFinishedTasks >= refJobView.getRequiredSize()) {
        titusRuntime.getCodeInvariants().inconsistent("Batch job reconciler attempts to create too many tasks: jobId=%s, requiredSize=%s, current=%s", refJobView.getJob().getId(), refJobView.getRequiredSize(), numberOfNotFinishedTasks);
        return Optional.empty();
    }
    Map<String, String> taskContext = getTaskContext(previousTask, unassignedIpAllocations, ebsVolumeIds);
    JobDescriptor jobDescriptor = refJobView.getJob().getJobDescriptor();
    ApplicationSLA capacityGroupDescriptor = JobManagerUtil.getCapacityGroupDescriptor(jobDescriptor, capacityGroupService);
    String resourcePool = capacityGroupDescriptor.getResourcePool();
    taskContext = CollectionsExt.copyAndAdd(taskContext, ImmutableMap.of(TaskAttributes.TASK_ATTRIBUTES_RESOURCE_POOL, resourcePool, TaskAttributes.TASK_ATTRIBUTES_TIER, capacityGroupDescriptor.getTier().name()));
    TitusChangeAction storeAction = storeWriteRetryInterceptor.apply(createOrReplaceTaskAction(runtime, jobStore, refJobView.getJobHolder(), taskIndex, versionSupplier, clock, taskContext));
    return Optional.of(storeAction);
}
Also used : JobServiceRuntime(com.netflix.titus.master.jobmanager.service.JobServiceRuntime) TitusChangeAction(com.netflix.titus.master.jobmanager.service.common.action.TitusChangeAction) Task(com.netflix.titus.api.jobmanager.model.job.Task) CollectionsExt(com.netflix.titus.common.util.CollectionsExt) LoggerFactory(org.slf4j.LoggerFactory) RetryActionInterceptor(com.netflix.titus.master.jobmanager.service.common.interceptor.RetryActionInterceptor) RECONCILER_CALLMETADATA(com.netflix.titus.api.jobmanager.service.JobManagerConstants.RECONCILER_CALLMETADATA) FeatureActivationConfiguration(com.netflix.titus.api.FeatureActivationConfiguration) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Map(java.util.Map) JobState(com.netflix.titus.api.jobmanager.model.job.JobState) BasicJobActions(com.netflix.titus.master.jobmanager.service.common.action.task.BasicJobActions) JobManagerConfiguration(com.netflix.titus.master.jobmanager.service.JobManagerConfiguration) Schedulers(rx.schedulers.Schedulers) JobStore(com.netflix.titus.api.jobmanager.store.JobStore) CallMetadata(com.netflix.titus.api.model.callmetadata.CallMetadata) JobManagerUtil(com.netflix.titus.master.jobmanager.service.JobManagerUtil) TaskRetryers(com.netflix.titus.master.jobmanager.service.common.action.TaskRetryers) Job(com.netflix.titus.api.jobmanager.model.job.Job) ImmutableMap(com.google.common.collect.ImmutableMap) TaskStatus(com.netflix.titus.api.jobmanager.model.job.TaskStatus) Set(java.util.Set) Scheduler(rx.Scheduler) DifferenceResolverUtils.getUnassignedIpAllocations(com.netflix.titus.master.jobmanager.service.common.DifferenceResolverUtils.getUnassignedIpAllocations) TaskState(com.netflix.titus.api.jobmanager.model.job.TaskState) List(java.util.List) VersionSupplier(com.netflix.titus.master.jobmanager.service.VersionSupplier) ReconciliationEngine(com.netflix.titus.common.framework.reconciler.ReconciliationEngine) Optional(java.util.Optional) JobManagerReconcilerEvent(com.netflix.titus.master.jobmanager.service.event.JobManagerReconcilerEvent) Clock(com.netflix.titus.common.util.time.Clock) KillInitiatedActions(com.netflix.titus.master.jobmanager.service.common.action.task.KillInitiatedActions) BatchJobTask(com.netflix.titus.api.jobmanager.model.job.BatchJobTask) ApplicationSlaManagementService(com.netflix.titus.master.service.management.ApplicationSlaManagementService) CreateOrReplaceBatchTaskActions.createOrReplaceTaskAction(com.netflix.titus.master.jobmanager.service.batch.action.CreateOrReplaceBatchTaskActions.createOrReplaceTaskAction) DifferenceResolverUtils(com.netflix.titus.master.jobmanager.service.common.DifferenceResolverUtils) Singleton(javax.inject.Singleton) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Inject(javax.inject.Inject) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) ChangeAction(com.netflix.titus.common.framework.reconciler.ChangeAction) ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA) DifferenceResolverUtils.getUnassignedEbsVolumes(com.netflix.titus.master.jobmanager.service.common.DifferenceResolverUtils.getUnassignedEbsVolumes) Named(javax.inject.Named) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) Logger(org.slf4j.Logger) DifferenceResolverUtils.getTaskContext(com.netflix.titus.master.jobmanager.service.common.DifferenceResolverUtils.getTaskContext) Retryers(com.netflix.titus.common.util.retry.Retryers) EntityHolder(com.netflix.titus.common.framework.reconciler.EntityHolder) TimeUnit(java.util.concurrent.TimeUnit) TaskAttributes(com.netflix.titus.api.jobmanager.TaskAttributes) BasicTaskActions(com.netflix.titus.master.jobmanager.service.common.action.task.BasicTaskActions) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) TokenBucket(com.netflix.titus.common.util.limiter.tokenbucket.TokenBucket) Collections(java.util.Collections) Task(com.netflix.titus.api.jobmanager.model.job.Task) BatchJobTask(com.netflix.titus.api.jobmanager.model.job.BatchJobTask) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA) TitusChangeAction(com.netflix.titus.master.jobmanager.service.common.action.TitusChangeAction)

Example 8 with ApplicationSLA

use of com.netflix.titus.api.model.ApplicationSLA in project titus-control-plane by Netflix.

the class ServiceDifferenceResolver method createNewTaskAction.

private Optional<TitusChangeAction> createNewTaskAction(ServiceJobView refJobView, Optional<EntityHolder> previousTask, List<String> unassignedIpAllocations, List<String> unassignedEbsVolumeIds) {
    // Safety check
    long numberOfNotFinishedTasks = getNumberOfNotFinishedTasks(refJobView);
    if (numberOfNotFinishedTasks >= refJobView.getRequiredSize()) {
        titusRuntime.getCodeInvariants().inconsistent("Service job reconciler attempts to create too many tasks: jobId=%s, requiredSize=%s, current=%s", refJobView.getJob().getId(), refJobView.getRequiredSize(), numberOfNotFinishedTasks);
        return Optional.empty();
    }
    Map<String, String> taskContext = getTaskContext(previousTask, unassignedIpAllocations, unassignedEbsVolumeIds);
    JobDescriptor jobDescriptor = refJobView.getJob().getJobDescriptor();
    ApplicationSLA capacityGroupDescriptor = JobManagerUtil.getCapacityGroupDescriptor(jobDescriptor, capacityGroupService);
    String resourcePool = capacityGroupDescriptor.getResourcePool();
    taskContext = CollectionsExt.copyAndAdd(taskContext, ImmutableMap.of(TaskAttributes.TASK_ATTRIBUTES_RESOURCE_POOL, resourcePool, TaskAttributes.TASK_ATTRIBUTES_TIER, capacityGroupDescriptor.getTier().name()));
    TitusChangeAction storeAction = storeWriteRetryInterceptor.apply(createOrReplaceTaskAction(runtime, jobStore, versionSupplier, refJobView.getJobHolder(), previousTask, clock, taskContext));
    return Optional.of(storeAction);
}
Also used : JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA) TitusChangeAction(com.netflix.titus.master.jobmanager.service.common.action.TitusChangeAction)

Example 9 with ApplicationSLA

use of com.netflix.titus.api.model.ApplicationSLA in project titus-control-plane by Netflix.

the class InMemoryApplicationSlaStore method create.

@Override
public Observable<Void> create(ApplicationSLA applicationSLA) {
    return Observable.create(subscriber -> {
        synchronized (lock) {
            ApplicationSLA previous = applicationSLAs.put(applicationSLA.getAppName(), applicationSLA);
            if (previous != null) {
                applicationSLAsBySchedulerName.remove(previous.getSchedulerName(), previous);
            }
            applicationSLAsBySchedulerName.put(applicationSLA.getSchedulerName(), applicationSLA);
            subscriber.onCompleted();
        }
    });
}
Also used : ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA)

Example 10 with ApplicationSLA

use of com.netflix.titus.api.model.ApplicationSLA in project titus-control-plane by Netflix.

the class DefaultApplicationSlaManagementServiceTest method testGetApplicationBySchedulerNameFailure.

@Test
public void testGetApplicationBySchedulerNameFailure() {
    ApplicationSLA myApp = ApplicationSlaSample.CriticalSmallKubeScheduler.build();
    Observable observable = Observable.error(new RuntimeException("generic failure"));
    when(storage.findBySchedulerName(myApp.getSchedulerName())).thenReturn(observable);
    assertThatThrownBy(() -> slaManagementService.getApplicationSLAsForScheduler(myApp.getSchedulerName())).isInstanceOf(RuntimeException.class).hasMessage("generic failure");
}
Also used : ApplicationSLA(com.netflix.titus.api.model.ApplicationSLA) Observable(rx.Observable) Test(org.junit.Test)

Aggregations

ApplicationSLA (com.netflix.titus.api.model.ApplicationSLA)28 Test (org.junit.Test)9 List (java.util.List)5 NotFoundException (com.netflix.titus.api.store.v2.exception.NotFoundException)4 Pair (com.netflix.titus.common.util.tuple.Pair)4 WebApplicationException (javax.ws.rs.WebApplicationException)4 Job (com.netflix.titus.api.jobmanager.model.job.Job)3 Task (com.netflix.titus.api.jobmanager.model.job.Task)3 CollectionsExt (com.netflix.titus.common.util.CollectionsExt)3 ArrayList (java.util.ArrayList)3 HashMap (java.util.HashMap)3 Map (java.util.Map)3 TaskAttributes (com.netflix.titus.api.jobmanager.TaskAttributes)2 JobDescriptor (com.netflix.titus.api.jobmanager.model.job.JobDescriptor)2 TaskState (com.netflix.titus.api.jobmanager.model.job.TaskState)2 BatchJobExt (com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt)2 ResourceDimension (com.netflix.titus.api.model.ResourceDimension)2 Tier (com.netflix.titus.api.model.Tier)2 ApplicationSlaStore (com.netflix.titus.api.store.v2.ApplicationSlaStore)2 TitusChangeAction (com.netflix.titus.master.jobmanager.service.common.action.TitusChangeAction)2