Search in sources :

Example 21 with Task

use of com.netflix.titus.grpc.protogen.Task in project titus-control-plane by Netflix.

the class CellWithFixedTasksService method findTasks.

@Override
public void findTasks(TaskQuery request, StreamObserver<TaskQueryResult> responseObserver) {
    Pair<List<Task>, Pagination> page = PaginationUtil.takePageWithCursor(toPage(request.getPage()), getTasksList(), JobManagerCursors.taskCursorOrderComparator(), JobManagerCursors::taskIndexOf, JobManagerCursors::newTaskCursorFrom);
    Set<String> fieldsFilter = new HashSet<>(request.getFieldsList());
    if (!fieldsFilter.isEmpty()) {
        fieldsFilter.add("id");
        page = page.mapLeft(tasks -> tasks.stream().map(task -> ProtobufExt.copy(task, fieldsFilter)).collect(Collectors.toList()));
    }
    TaskQueryResult result = TaskQueryResult.newBuilder().addAllItems(page.getLeft()).setPagination(toGrpcPagination(page.getRight())).build();
    responseObserver.onNext(result);
    responseObserver.onCompleted();
}
Also used : NOT_FOUND(io.grpc.Status.NOT_FOUND) TaskQueryResult(com.netflix.titus.grpc.protogen.TaskQueryResult) Task(com.netflix.titus.grpc.protogen.Task) TaskId(com.netflix.titus.grpc.protogen.TaskId) Set(java.util.Set) JobManagementServiceGrpc(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) ProtobufExt(com.netflix.titus.common.util.ProtobufExt) ArrayList(java.util.ArrayList) GrpcJobQueryModelConverters.toPage(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobQueryModelConverters.toPage) HashSet(java.util.HashSet) Empty(com.google.protobuf.Empty) List(java.util.List) GrpcJobQueryModelConverters.toGrpcPagination(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobQueryModelConverters.toGrpcPagination) StreamObserver(io.grpc.stub.StreamObserver) Pair(com.netflix.titus.common.util.tuple.Pair) JobManagerCursors(com.netflix.titus.runtime.jobmanager.JobManagerCursors) Map(java.util.Map) Pagination(com.netflix.titus.api.model.Pagination) Collections(java.util.Collections) PaginationUtil(com.netflix.titus.api.model.PaginationUtil) TaskQuery(com.netflix.titus.grpc.protogen.TaskQuery) TaskKillRequest(com.netflix.titus.grpc.protogen.TaskKillRequest) GrpcJobQueryModelConverters.toGrpcPagination(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobQueryModelConverters.toGrpcPagination) Pagination(com.netflix.titus.api.model.Pagination) ArrayList(java.util.ArrayList) List(java.util.List) JobManagerCursors(com.netflix.titus.runtime.jobmanager.JobManagerCursors) TaskQueryResult(com.netflix.titus.grpc.protogen.TaskQueryResult) HashSet(java.util.HashSet)

Example 22 with Task

use of com.netflix.titus.grpc.protogen.Task in project titus-control-plane by Netflix.

the class CellWithFixedTasksService method findTask.

@Override
public void findTask(TaskId request, StreamObserver<Task> responseObserver) {
    Task task = tasksIndex.get(request.getId());
    if (task == null) {
        responseObserver.onError(NOT_FOUND.asRuntimeException());
        return;
    }
    responseObserver.onNext(task);
    responseObserver.onCompleted();
}
Also used : Task(com.netflix.titus.grpc.protogen.Task)

Example 23 with Task

use of com.netflix.titus.grpc.protogen.Task in project titus-control-plane by Netflix.

the class LocalCacheQueryProcessor method findTasks.

public TaskQueryResult findTasks(TaskQuery taskQuery) {
    JobQueryCriteria<TaskStatus.TaskState, JobDescriptor.JobSpecCase> queryCriteria = GrpcJobQueryModelConverters.toJobQueryCriteria(taskQuery);
    Page page = toPage(taskQuery.getPage());
    List<com.netflix.titus.api.jobmanager.model.job.Task> matchingTasks = findMatchingTasks(queryCriteria);
    PageResult<com.netflix.titus.api.jobmanager.model.job.Task> pageResult = JobManagerCursors.newCoreTaskPaginationEvaluator().takePage(page, matchingTasks);
    Set<String> fields = newFieldsFilter(taskQuery.getFieldsList(), TASK_MINIMUM_FIELD_SET);
    List<Task> grpcTasks = pageResult.getItems().stream().map(task -> {
        Task grpcTask = GrpcJobManagementModelConverters.toGrpcTask(task, logStorageInfo);
        if (!fields.isEmpty()) {
            grpcTask = ProtobufExt.copy(grpcTask, fields);
        }
        return grpcTask;
    }).collect(Collectors.toList());
    return TaskQueryResult.newBuilder().setPagination(toGrpcPagination(pageResult.getPagination())).addAllItems(grpcTasks).build();
}
Also used : MetricConstants(com.netflix.titus.gateway.MetricConstants) ObserveJobsQuery(com.netflix.titus.grpc.protogen.ObserveJobsQuery) CollectionsExt(com.netflix.titus.common.util.CollectionsExt) LoggerFactory(org.slf4j.LoggerFactory) JobDataReplicator(com.netflix.titus.runtime.connector.jobmanager.JobDataReplicator) JOB_MINIMUM_FIELD_SET(com.netflix.titus.runtime.jobmanager.gateway.JobServiceGateway.JOB_MINIMUM_FIELD_SET) ReactorExt(com.netflix.titus.common.util.rx.ReactorExt) PreDestroy(javax.annotation.PreDestroy) JobQueryResult(com.netflix.titus.grpc.protogen.JobQueryResult) Matcher(java.util.regex.Matcher) JobSnapshot(com.netflix.titus.runtime.connector.jobmanager.snapshot.JobSnapshot) GrpcJobQueryModelConverters(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobQueryModelConverters) Map(java.util.Map) Status(io.grpc.Status) CallMetadata(com.netflix.titus.api.model.callmetadata.CallMetadata) JobDescriptor(com.netflix.titus.grpc.protogen.JobDescriptor) Job(com.netflix.titus.api.jobmanager.model.job.Job) TaskQueryResult(com.netflix.titus.grpc.protogen.TaskQueryResult) Task(com.netflix.titus.grpc.protogen.Task) Set(java.util.Set) JobManagerEvent(com.netflix.titus.api.jobmanager.model.job.event.JobManagerEvent) Collectors(java.util.stream.Collectors) ProtobufExt(com.netflix.titus.common.util.ProtobufExt) CommonRuntimeGrpcModelConverters.toGrpcPagination(com.netflix.titus.runtime.endpoint.common.grpc.CommonRuntimeGrpcModelConverters.toGrpcPagination) PageResult(com.netflix.titus.api.model.PageResult) List(java.util.List) TaskUpdateEvent(com.netflix.titus.api.jobmanager.model.job.event.TaskUpdateEvent) V3JobQueryCriteriaEvaluator(com.netflix.titus.runtime.endpoint.v3.grpc.query.V3JobQueryCriteriaEvaluator) Optional(java.util.Optional) Pattern(java.util.regex.Pattern) RegExpExt(com.netflix.titus.common.util.RegExpExt) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) JobStatus(com.netflix.titus.grpc.protogen.JobStatus) Counter(com.netflix.spectator.api.Counter) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Singleton(javax.inject.Singleton) Scheduler(reactor.core.scheduler.Scheduler) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Observable(rx.Observable) HashSet(java.util.HashSet) Inject(javax.inject.Inject) JobQuery(com.netflix.titus.grpc.protogen.JobQuery) SpectatorExt(com.netflix.titus.common.util.spectator.SpectatorExt) Pair(com.netflix.titus.common.util.tuple.Pair) TaskStatus(com.netflix.titus.grpc.protogen.TaskStatus) Schedulers(reactor.core.scheduler.Schedulers) TASK_MINIMUM_FIELD_SET(com.netflix.titus.runtime.jobmanager.gateway.JobServiceGateway.TASK_MINIMUM_FIELD_SET) TaskQuery(com.netflix.titus.grpc.protogen.TaskQuery) CommonRuntimeGrpcModelConverters.toPage(com.netflix.titus.runtime.endpoint.common.grpc.CommonRuntimeGrpcModelConverters.toPage) V3TaskQueryCriteriaEvaluator(com.netflix.titus.runtime.endpoint.v3.grpc.query.V3TaskQueryCriteriaEvaluator) Logger(org.slf4j.Logger) JobUpdateEvent(com.netflix.titus.api.jobmanager.model.job.event.JobUpdateEvent) LogStorageInfo(com.netflix.titus.api.jobmanager.model.job.LogStorageInfo) Mono(reactor.core.publisher.Mono) GrpcJobManagementModelConverters(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobManagementModelConverters) JobKeepAliveEvent(com.netflix.titus.api.jobmanager.model.job.event.JobKeepAliveEvent) StatusRuntimeException(io.grpc.StatusRuntimeException) Flux(reactor.core.publisher.Flux) ValueRangeCounter(com.netflix.titus.common.util.spectator.ValueRangeCounter) JobQueryCriteria(com.netflix.titus.runtime.endpoint.JobQueryCriteria) JobManagerCursors(com.netflix.titus.runtime.jobmanager.JobManagerCursors) Page(com.netflix.titus.api.model.Page) Registry(com.netflix.spectator.api.Registry) GrpcJobQueryModelConverters.toJobQueryCriteria(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobQueryModelConverters.toJobQueryCriteria) TitusRuntime(com.netflix.titus.common.runtime.TitusRuntime) Collections(java.util.Collections) MetricSelector(com.netflix.titus.common.util.spectator.MetricSelector) Task(com.netflix.titus.grpc.protogen.Task) CommonRuntimeGrpcModelConverters.toPage(com.netflix.titus.runtime.endpoint.common.grpc.CommonRuntimeGrpcModelConverters.toPage) Page(com.netflix.titus.api.model.Page)

Example 24 with Task

use of com.netflix.titus.grpc.protogen.Task in project titus-control-plane by Netflix.

the class LocalCacheQueryProcessor method findMatchingTasks.

private List<com.netflix.titus.api.jobmanager.model.job.Task> findMatchingTasks(JobQueryCriteria<TaskStatus.TaskState, JobDescriptor.JobSpecCase> queryCriteria) {
    JobSnapshot jobSnapshot = jobDataReplicator.getCurrent();
    Map<String, Job<?>> jobsById = jobSnapshot.getJobMap();
    V3TaskQueryCriteriaEvaluator queryFilter = new V3TaskQueryCriteriaEvaluator(queryCriteria, titusRuntime);
    List<com.netflix.titus.api.jobmanager.model.job.Task> matchingTasks = new ArrayList<>();
    jobsById.forEach((jobId, job) -> {
        Map<String, com.netflix.titus.api.jobmanager.model.job.Task> tasks = jobSnapshot.getTasks(jobId);
        if (!CollectionsExt.isNullOrEmpty(tasks)) {
            tasks.forEach((taskId, task) -> {
                Pair<Job<?>, com.netflix.titus.api.jobmanager.model.job.Task> jobTaskPair = Pair.of(job, task);
                if (queryFilter.test(jobTaskPair)) {
                    matchingTasks.add(task);
                }
            });
        }
    });
    return matchingTasks;
}
Also used : V3TaskQueryCriteriaEvaluator(com.netflix.titus.runtime.endpoint.v3.grpc.query.V3TaskQueryCriteriaEvaluator) Task(com.netflix.titus.grpc.protogen.Task) ArrayList(java.util.ArrayList) JobSnapshot(com.netflix.titus.runtime.connector.jobmanager.snapshot.JobSnapshot) Job(com.netflix.titus.api.jobmanager.model.job.Job)

Example 25 with Task

use of com.netflix.titus.grpc.protogen.Task in project titus-control-plane by Netflix.

the class TaskDataInjectorTest method testFindTaskWithoutRelocationDeadline.

@Test
public void testFindTaskWithoutRelocationDeadline() {
    when(relocationDataReplicator.getCurrent()).thenReturn(TaskRelocationSnapshot.empty());
    Task merged = Observable.just(TASK1).map(taskDataInjector::injectIntoTask).toBlocking().first();
    assertThat(merged.getMigrationDetails().getNeedsMigration()).isFalse();
}
Also used : Task(com.netflix.titus.grpc.protogen.Task) Test(org.junit.Test)

Aggregations

Task (com.netflix.titus.grpc.protogen.Task)30 Test (org.junit.Test)18 ArrayList (java.util.ArrayList)11 TaskQueryResult (com.netflix.titus.grpc.protogen.TaskQueryResult)10 Job (com.netflix.titus.grpc.protogen.Job)8 TaskQuery (com.netflix.titus.grpc.protogen.TaskQuery)6 BaseIntegrationTest (com.netflix.titus.master.integration.BaseIntegrationTest)6 IntegrationTest (com.netflix.titus.testkit.junit.category.IntegrationTest)6 HashSet (java.util.HashSet)6 List (java.util.List)6 Pair (com.netflix.titus.common.util.tuple.Pair)4 JobChangeNotification (com.netflix.titus.grpc.protogen.JobChangeNotification)4 Pagination (com.netflix.titus.api.model.Pagination)3 TitusServiceException (com.netflix.titus.api.service.TitusServiceException)3 TitusRuntime (com.netflix.titus.common.runtime.TitusRuntime)3 JobQuery (com.netflix.titus.grpc.protogen.JobQuery)3 JobQueryResult (com.netflix.titus.grpc.protogen.JobQueryResult)3 Page (com.netflix.titus.grpc.protogen.Page)3 TaskStatus (com.netflix.titus.grpc.protogen.TaskStatus)3 JobManagerCursors (com.netflix.titus.runtime.jobmanager.JobManagerCursors)3