Search in sources :

Example 1 with ScheduleDescriptor

use of com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor in project titus-control-plane by Netflix.

the class KubeAndJobServiceSyncStatusWatcher method enterActiveMode.

@Activator
public Observable<Void> enterActiveMode() {
    try {
        kubeApiFacade.getPodInformer().addEventHandler(new ResourceEventHandler<V1Pod>() {

            @Override
            public void onAdd(V1Pod obj) {
                capturedState.put(obj.getMetadata().getName(), new TaskHolder(obj, false));
            }

            @Override
            public void onUpdate(V1Pod oldObj, V1Pod newObj) {
                capturedState.put(newObj.getMetadata().getName(), new TaskHolder(newObj, false));
            }

            @Override
            public void onDelete(V1Pod obj, boolean deletedFinalStateUnknown) {
                capturedState.put(obj.getMetadata().getName(), new TaskHolder(obj, true));
            }
        });
        ScheduleDescriptor scheduleDescriptor = ScheduleDescriptor.newBuilder().withName(KubeAndJobServiceSyncStatusWatcher.class.getSimpleName()).withDescription("Compare Kube pod state with Titus job service").withInitialDelay(Duration.ofSeconds(60)).withInterval(Duration.ofSeconds(10)).withTimeout(Duration.ofSeconds((60))).build();
        this.schedulerRef = titusRuntime.getLocalScheduler().schedule(scheduleDescriptor, this::process, ExecutorsExt.namedSingleThreadExecutor(KubeAndJobServiceSyncStatusWatcher.class.getSimpleName()));
    } catch (Exception e) {
        return Observable.error(e);
    }
    return Observable.empty();
}
Also used : ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor) V1Pod(io.kubernetes.client.openapi.models.V1Pod) Activator(com.netflix.titus.common.util.guice.annotation.Activator)

Example 2 with ScheduleDescriptor

use of com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor in project titus-control-plane by Netflix.

the class DefaultRelocationWorkflowExecutor method activate.

@Override
public void activate() {
    ScheduleDescriptor relocationScheduleDescriptor = ScheduleDescriptor.newBuilder().withName("relocationWorkflow").withDescription("Task relocation scheduler").withInitialDelay(Duration.ZERO).withInterval(Duration.ofMillis(configuration.getRelocationScheduleIntervalMs())).withTimeout(Duration.ofMillis(configuration.getRelocationTimeoutMs())).withRetryerSupplier(() -> Retryers.exponentialBackoff(1, 5, TimeUnit.MINUTES)).build();
    this.localSchedulerDisposable = titusRuntime.getLocalScheduler().schedule(relocationScheduleDescriptor, this::nextRelocationStep, true);
}
Also used : ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor)

Example 3 with ScheduleDescriptor

use of com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor in project titus-control-plane by Netflix.

the class DefaultNodeConditionController method activate.

@Override
public void activate() {
    logger.info("Activating DefaultNodeConditionController");
    ScheduleDescriptor nodeConditionControlLoopSchedulerDescriptor = ScheduleDescriptor.newBuilder().withName("nodeConditionCtrl").withDescription("Node Condition control loop").withInitialDelay(Duration.ZERO).withInterval(Duration.ofMillis(configuration.getNodeConditionControlLoopIntervalMs())).withTimeout(Duration.ofMillis(configuration.getNodeConditionControlLoopTimeoutMs())).withRetryerSupplier(() -> Retryers.exponentialBackoff(1, 5, TimeUnit.MINUTES)).build();
    this.scheduleReference = titusRuntime.getLocalScheduler().scheduleMono(nodeConditionControlLoopSchedulerDescriptor, this::handleNodesWithBadCondition, Schedulers.parallel());
}
Also used : ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor)

Example 4 with ScheduleDescriptor

use of com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor in project titus-control-plane by Netflix.

the class ArchivedTasksGc method enterActiveMode.

@Activator
public void enterActiveMode() {
    ScheduleDescriptor scheduleDescriptor = ScheduleDescriptor.newBuilder().withName("gcArchivedTasks").withDescription("GC oldest archived pasts once the criteria is met").withInitialDelay(Duration.ofMillis(configuration.getGcInitialDelayMs())).withInterval(Duration.ofMillis(configuration.getGcIntervalMs())).withTimeout(Duration.ofMillis(configuration.getGcTimeoutMs())).build();
    this.schedulerRef = titusRuntime.getLocalScheduler().schedule(scheduleDescriptor, e -> gc(), ExecutorsExt.namedSingleThreadExecutor(ArchivedTasksGc.class.getSimpleName()));
}
Also used : Completable(rx.Completable) Gauge(com.netflix.spectator.api.Gauge) Task(com.netflix.titus.api.jobmanager.model.job.Task) LoggerFactory(org.slf4j.LoggerFactory) Singleton(javax.inject.Singleton) ArrayList(java.util.ArrayList) ScheduleReference(com.netflix.titus.common.framework.scheduler.ScheduleReference) Observable(rx.Observable) Inject(javax.inject.Inject) PreDestroy(javax.annotation.PreDestroy) Pair(com.netflix.titus.common.util.tuple.Pair) Duration(java.time.Duration) JobStore(com.netflix.titus.api.jobmanager.store.JobStore) Deactivator(com.netflix.titus.common.util.guice.annotation.Deactivator) ExecutorsExt(com.netflix.titus.common.util.ExecutorsExt) Job(com.netflix.titus.api.jobmanager.model.job.Job) Logger(org.slf4j.Logger) JobFunctions(com.netflix.titus.api.jobmanager.model.job.JobFunctions) Collectors(java.util.stream.Collectors) TaskState(com.netflix.titus.api.jobmanager.model.job.TaskState) Activator(com.netflix.titus.common.util.guice.annotation.Activator) List(java.util.List) ExecutableStatus(com.netflix.titus.api.jobmanager.model.job.ExecutableStatus) V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor) Registry(com.netflix.spectator.api.Registry) METRIC_JOB_MANAGER(com.netflix.titus.master.MetricConstants.METRIC_JOB_MANAGER) VisibleForTesting(com.google.common.annotations.VisibleForTesting) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) Comparator(java.util.Comparator) Evaluators(com.netflix.titus.common.util.Evaluators) Collections(java.util.Collections) ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor) Activator(com.netflix.titus.common.util.guice.annotation.Activator)

Example 5 with ScheduleDescriptor

use of com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor in project titus-control-plane by Netflix.

the class DefaultKubeJobManagementReconciler method enterActiveMode.

@Activator
public void enterActiveMode() {
    ScheduleDescriptor scheduleDescriptor = ScheduleDescriptor.newBuilder().withName("reconcileNodesAndPods").withDescription("Reconcile nodes and pods").withInitialDelay(Duration.ofMillis(backendConfiguration.getReconcilerInitialDelayMs())).withInterval(Duration.ofMillis(backendConfiguration.getReconcilerIntervalMs())).withTimeout(Duration.ofMinutes(5)).build();
    this.schedulerRef = titusRuntime.getLocalScheduler().schedule(scheduleDescriptor, e -> reconcile(), ExecutorsExt.namedSingleThreadExecutor(DefaultKubeJobManagementReconciler.class.getSimpleName()));
}
Also used : Gauge(com.netflix.spectator.api.Gauge) Task(com.netflix.titus.api.jobmanager.model.job.Task) FluxSink(reactor.core.publisher.FluxSink) StdKubeApiFacade(com.netflix.titus.runtime.connector.kubernetes.std.StdKubeApiFacade) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) MetricConstants(com.netflix.titus.master.MetricConstants) StringExt(com.netflix.titus.common.util.StringExt) V1Node(io.kubernetes.client.openapi.models.V1Node) Singleton(javax.inject.Singleton) ReactorExt(com.netflix.titus.common.util.rx.ReactorExt) KubeUtil(com.netflix.titus.master.kubernetes.KubeUtil) Function(java.util.function.Function) ArrayList(java.util.ArrayList) ScheduleReference(com.netflix.titus.common.framework.scheduler.ScheduleReference) Inject(javax.inject.Inject) PreDestroy(javax.annotation.PreDestroy) Duration(java.time.Duration) Map(java.util.Map) DirectKubeConfiguration(com.netflix.titus.master.kubernetes.client.DirectKubeConfiguration) Named(javax.inject.Named) Deactivator(com.netflix.titus.common.util.guice.annotation.Deactivator) PodEvent(com.netflix.titus.master.kubernetes.client.model.PodEvent) ExecutorsExt(com.netflix.titus.common.util.ExecutorsExt) Logger(org.slf4j.Logger) DirectProcessor(reactor.core.publisher.DirectProcessor) TaskStatus(com.netflix.titus.api.jobmanager.model.job.TaskStatus) Set(java.util.Set) Collectors(java.util.stream.Collectors) TaskState(com.netflix.titus.api.jobmanager.model.job.TaskState) PodNotFoundEvent(com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent) Activator(com.netflix.titus.common.util.guice.annotation.Activator) FixedIntervalTokenBucketConfiguration(com.netflix.titus.common.util.limiter.tokenbucket.FixedIntervalTokenBucketConfiguration) Flux(reactor.core.publisher.Flux) List(java.util.List) V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) Stream(java.util.stream.Stream) TaskAttributes(com.netflix.titus.api.jobmanager.TaskAttributes) ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor) Registry(com.netflix.spectator.api.Registry) Optional(java.util.Optional) BackendConfiguration(com.netflix.titus.master.config.BackendConfiguration) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) Evaluators(com.netflix.titus.common.util.Evaluators) Clock(com.netflix.titus.common.util.time.Clock) V1Pod(io.kubernetes.client.openapi.models.V1Pod) ScheduleDescriptor(com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor) Activator(com.netflix.titus.common.util.guice.annotation.Activator)

Aggregations

ScheduleDescriptor (com.netflix.titus.common.framework.scheduler.model.ScheduleDescriptor)6 Activator (com.netflix.titus.common.util.guice.annotation.Activator)4 Gauge (com.netflix.spectator.api.Gauge)3 ScheduleReference (com.netflix.titus.common.framework.scheduler.ScheduleReference)3 TitusRuntime (com.netflix.titus.common.runtime.TitusRuntime)3 Evaluators (com.netflix.titus.common.util.Evaluators)3 ExecutorsExt (com.netflix.titus.common.util.ExecutorsExt)3 Deactivator (com.netflix.titus.common.util.guice.annotation.Deactivator)3 Duration (java.time.Duration)3 List (java.util.List)3 PreDestroy (javax.annotation.PreDestroy)3 Logger (org.slf4j.Logger)3 LoggerFactory (org.slf4j.LoggerFactory)3 Registry (com.netflix.spectator.api.Registry)2 Task (com.netflix.titus.api.jobmanager.model.job.Task)2 TaskState (com.netflix.titus.api.jobmanager.model.job.TaskState)2 V3JobOperations (com.netflix.titus.api.jobmanager.service.V3JobOperations)2 FixedIntervalTokenBucketConfiguration (com.netflix.titus.common.util.limiter.tokenbucket.FixedIntervalTokenBucketConfiguration)2 MetricConstants (com.netflix.titus.master.MetricConstants)2 ArrayList (java.util.ArrayList)2