Search in sources :

Example 1 with PodNotFoundEvent

use of com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent in project titus-control-plane by Netflix.

the class DefaultKubeJobManagementReconciler method publishPodEvent.

private void publishPodEvent(Task task, TaskStatus finalTaskStatus) {
    PodNotFoundEvent podEvent = PodEvent.onPodNotFound(task, finalTaskStatus);
    logger.debug("Publishing pod event: {}", podEvent);
    podEventSink.next(podEvent);
}
Also used : PodNotFoundEvent(com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent)

Example 2 with PodNotFoundEvent

use of com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent in project titus-control-plane by Netflix.

the class KubeNotificationProcessor method processEvent.

private Mono<Void> processEvent(PodEvent event) {
    Pair<Job<?>, Task> jobAndTask = v3JobOperations.findTaskById(event.getTaskId()).orElse(null);
    if (jobAndTask == null) {
        logger.warn("Got Kube notification about unknown task: {}", event.getTaskId());
        return Mono.empty();
    }
    Task task = jobAndTask.getRight();
    if (event instanceof PodNotFoundEvent) {
        return handlePodNotFoundEvent((PodNotFoundEvent) event);
    }
    // TODO Implement flatMapWithSequentialSubscription operator
    return handlePodUpdatedEvent(event, jobAndTask.getLeft(), task);
}
Also used : Task(com.netflix.titus.api.jobmanager.model.job.Task) PodNotFoundEvent(com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent) Job(com.netflix.titus.api.jobmanager.model.job.Job)

Example 3 with PodNotFoundEvent

use of com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent in project titus-control-plane by Netflix.

the class KubeNotificationProcessor method handlePodNotFoundEvent.

private Mono<Void> handlePodNotFoundEvent(PodNotFoundEvent event) {
    Task task = event.getTask();
    logger.info("Pod not found event: taskId={}, finalTaskStatus={}", task.getId(), event.getFinalTaskStatus());
    return ReactorExt.toMono(v3JobOperations.updateTask(task.getId(), currentTask -> {
        List<TaskStatus> newHistory = CollectionsExt.copyAndAdd(currentTask.getStatusHistory(), currentTask.getStatus());
        Task updatedTask = currentTask.toBuilder().withStatus(event.getFinalTaskStatus()).withStatusHistory(newHistory).build();
        metricsChangesApplied.increment();
        return Optional.of(updatedTask);
    }, V3JobOperations.Trigger.Kube, "Pod status updated from kubernetes node, it couldn't find the pod " + task.getId(), KUBE_CALL_METADATA));
}
Also used : Retry(reactor.util.retry.Retry) Task(com.netflix.titus.api.jobmanager.model.job.Task) CollectionsExt(com.netflix.titus.common.util.CollectionsExt) LoggerFactory(org.slf4j.LoggerFactory) V1PodStatus(io.kubernetes.client.openapi.models.V1PodStatus) ReactorExt(com.netflix.titus.common.util.rx.ReactorExt) KubeUtil(com.netflix.titus.master.kubernetes.KubeUtil) TITUS_NODE_DOMAIN(com.netflix.titus.runtime.kubernetes.KubeConstants.TITUS_NODE_DOMAIN) Duration(java.time.Duration) Map(java.util.Map) DirectKubeApiServerIntegrator(com.netflix.titus.master.kubernetes.client.DirectKubeApiServerIntegrator) Either(com.netflix.titus.common.util.tuple.Either) CallMetadata(com.netflix.titus.api.model.callmetadata.CallMetadata) PodEvent(com.netflix.titus.master.kubernetes.client.model.PodEvent) Job(com.netflix.titus.api.jobmanager.model.job.Job) TaskStatus(com.netflix.titus.api.jobmanager.model.job.TaskStatus) JobFunctions(com.netflix.titus.api.jobmanager.model.job.JobFunctions) TaskState(com.netflix.titus.api.jobmanager.model.job.TaskState) PodNotFoundEvent(com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent) Timer(com.netflix.spectator.api.Timer) List(java.util.List) Optional(java.util.Optional) PodWrapper(com.netflix.titus.master.kubernetes.client.model.PodWrapper) Gauge(com.netflix.spectator.api.Gauge) Disposable(reactor.core.Disposable) Stopwatch(com.google.common.base.Stopwatch) PodDeletedEvent(com.netflix.titus.master.kubernetes.client.model.PodDeletedEvent) Counter(com.netflix.spectator.api.Counter) HashMap(java.util.HashMap) MetricConstants(com.netflix.titus.master.MetricConstants) V1Node(io.kubernetes.client.openapi.models.V1Node) Singleton(javax.inject.Singleton) Scheduler(reactor.core.scheduler.Scheduler) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Pair(com.netflix.titus.common.util.tuple.Pair) ContainerResultCodeResolver(com.netflix.titus.master.kubernetes.ContainerResultCodeResolver) Schedulers(reactor.core.scheduler.Schedulers) Evaluators.acceptNotNull(com.netflix.titus.common.util.Evaluators.acceptNotNull) KubeJobManagementReconciler(com.netflix.titus.master.kubernetes.controller.KubeJobManagementReconciler) ExecutorService(java.util.concurrent.ExecutorService) ExecutorsExt(com.netflix.titus.common.util.ExecutorsExt) Logger(org.slf4j.Logger) PodUpdatedEvent(com.netflix.titus.master.kubernetes.client.model.PodUpdatedEvent) Mono(reactor.core.publisher.Mono) Activator(com.netflix.titus.common.util.guice.annotation.Activator) TimeUnit(java.util.concurrent.TimeUnit) AtomicLong(java.util.concurrent.atomic.AtomicLong) ExecutableStatus(com.netflix.titus.api.jobmanager.model.job.ExecutableStatus) V3JobOperations(com.netflix.titus.api.jobmanager.service.V3JobOperations) TaskAttributes(com.netflix.titus.api.jobmanager.TaskAttributes) PodToTaskMapper(com.netflix.titus.master.kubernetes.PodToTaskMapper) V1ContainerState(io.kubernetes.client.openapi.models.V1ContainerState) VisibleForTesting(com.google.common.annotations.VisibleForTesting) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) Comparator(java.util.Comparator) Evaluators(com.netflix.titus.common.util.Evaluators) Task(com.netflix.titus.api.jobmanager.model.job.Task) List(java.util.List) ArrayList(java.util.ArrayList)

Aggregations

PodNotFoundEvent (com.netflix.titus.master.kubernetes.client.model.PodNotFoundEvent)3 Job (com.netflix.titus.api.jobmanager.model.job.Job)2 Task (com.netflix.titus.api.jobmanager.model.job.Task)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 Stopwatch (com.google.common.base.Stopwatch)1 Counter (com.netflix.spectator.api.Counter)1 Gauge (com.netflix.spectator.api.Gauge)1 Timer (com.netflix.spectator.api.Timer)1 TaskAttributes (com.netflix.titus.api.jobmanager.TaskAttributes)1 ExecutableStatus (com.netflix.titus.api.jobmanager.model.job.ExecutableStatus)1 JobFunctions (com.netflix.titus.api.jobmanager.model.job.JobFunctions)1 TaskState (com.netflix.titus.api.jobmanager.model.job.TaskState)1 TaskStatus (com.netflix.titus.api.jobmanager.model.job.TaskStatus)1 V3JobOperations (com.netflix.titus.api.jobmanager.service.V3JobOperations)1 CallMetadata (com.netflix.titus.api.model.callmetadata.CallMetadata)1 TitusRuntime (com.netflix.titus.common.runtime.TitusRuntime)1 CollectionsExt (com.netflix.titus.common.util.CollectionsExt)1 Evaluators (com.netflix.titus.common.util.Evaluators)1 Evaluators.acceptNotNull (com.netflix.titus.common.util.Evaluators.acceptNotNull)1 ExecutorsExt (com.netflix.titus.common.util.ExecutorsExt)1