Search in sources :

Example 1 with JobManagementServiceStub

use of com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub in project titus-control-plane by Netflix.

the class AggregatingJobServiceGatewayTest method createJobRouteToCorrectStack.

@Test
public void createJobRouteToCorrectStack() {
    // Build service handlers for each cell
    cellToServiceMap.forEach((cell, grpcServerRule) -> grpcServerRule.getServiceRegistry().addService(new CellWithCachedJobsService(cell.getName())));
    // Expected assignments based on routing rules in setUp()
    Map<String, String> expectedAssignmentMap = ImmutableMap.of("app1", "one", "app2", "one", "app3", "two", "app4", "one");
    expectedAssignmentMap.forEach((appName, expectedCellName) -> {
        // Create the job and let it get routed
        JobDescriptor jobDescriptor = JobDescriptor.newBuilder().setApplicationName(appName).setCapacityGroup(appName + "CapGroup").build();
        String jobId = service.createJob(jobDescriptor, JobManagerConstants.UNDEFINED_CALL_METADATA).toBlocking().first();
        // Get a client to the test gRPC service for the cell that we expect got it
        // TODO(Andrew L): This can use findJob() instead once AggregatingService implements it
        Cell expectedCell = getCellWithName(expectedCellName).orElseThrow(() -> TitusServiceException.cellNotFound(expectedCellName));
        JobManagementServiceStub expectedCellClient = JobManagementServiceGrpc.newStub(cellToServiceMap.get(expectedCell).getChannel());
        // Check that the cell has it with the correct attribute
        TestStreamObserver<Job> findJobResponse = new TestStreamObserver<>();
        expectedCellClient.findJob(JobId.newBuilder().setId(jobId).build(), findJobResponse);
        assertThatCode(() -> {
            Job job = findJobResponse.takeNext(1, TimeUnit.SECONDS);
            assertThat(job.getJobDescriptor().getAttributesOrThrow(JOB_ATTRIBUTES_CELL).equals(expectedCellName));
        }).doesNotThrowAnyException();
    });
}
Also used : TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) JobDescriptor(com.netflix.titus.grpc.protogen.JobDescriptor) Job(com.netflix.titus.grpc.protogen.Job) Cell(com.netflix.titus.api.federation.model.Cell) JobManagementServiceStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub) Test(org.junit.Test)

Example 2 with JobManagementServiceStub

use of com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub in project titus-control-plane by Netflix.

the class AggregatingJobServiceGateway method createJob.

@Override
public Observable<String> createJob(JobDescriptor jobDescriptor, CallMetadata callMetadata) {
    Cell cell = router.routeKey(jobDescriptor).orElse(null);
    if (cell == null) {
        // This should never happen in a correctly setup system.
        return Observable.error(new IllegalStateException("Internal system error. Routing rule not found"));
    }
    logger.debug("Routing JobDescriptor {} to Cell {}", jobDescriptor, cell);
    Optional<JobManagementServiceStub> optionalClient = CellConnectorUtil.toStub(cell, cellConnector, JobManagementServiceGrpc::newStub);
    if (!optionalClient.isPresent()) {
        return Observable.error(TitusServiceException.cellNotFound(cell.getName()));
    }
    JobManagementServiceStub client = optionalClient.get();
    JobDescriptor.Builder jobDescriptorBuilder = addStackName(jobDescriptor.toBuilder());
    jobDescriptorBuilder = removeFederationAttributes(jobDescriptorBuilder);
    jobDescriptorBuilder = addRoutingCell(jobDescriptorBuilder, cell);
    JobDescriptor jd = jobDescriptorBuilder.build();
    return createRequestObservable(emitter -> {
        StreamObserver<JobId> streamObserver = GrpcUtil.createClientResponseObserver(emitter, jobId -> emitter.onNext(jobId.getId()), emitter::onError, emitter::onCompleted);
        wrap(client, callMetadata).createJob(jd, streamObserver);
    }, grpcConfiguration.getRequestTimeoutMs());
}
Also used : JobManagementServiceGrpc(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc) JobDescriptor(com.netflix.titus.grpc.protogen.JobDescriptor) Cell(com.netflix.titus.api.federation.model.Cell) CellConnectorUtil.callToCell(com.netflix.titus.federation.service.CellConnectorUtil.callToCell) JobId(com.netflix.titus.grpc.protogen.JobId) JobManagementServiceStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub)

Example 3 with JobManagementServiceStub

use of com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub in project titus-control-plane by Netflix.

the class AggregatingJobServiceGateway method observeJobs.

@Override
public Observable<JobChangeNotification> observeJobs(ObserveJobsQuery query, CallMetadata callMetadata) {
    final Observable<JobChangeNotification> observable = createRequestObservable(delegate -> {
        Emitter<JobChangeNotification> emitter = new EmitterWithMultipleSubscriptions<>(delegate);
        Map<Cell, JobManagementServiceStub> clients = CellConnectorUtil.stubs(cellConnector, JobManagementServiceGrpc::newStub);
        final CountDownLatch markersEmitted = new CallbackCountDownLatch(clients.size(), () -> emitter.onNext(buildJobSnapshotEndMarker()));
        clients.forEach((cell, client) -> {
            StreamObserver<JobChangeNotification> streamObserver = new FilterOutFirstMarker(emitter, markersEmitted);
            wrapWithNoDeadline(client, callMetadata).observeJobs(query, streamObserver);
        });
    });
    return observable.map(this::addStackName);
}
Also used : EmitterWithMultipleSubscriptions(com.netflix.titus.common.util.rx.EmitterWithMultipleSubscriptions) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) JobManagementServiceGrpc(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc) CallbackCountDownLatch(com.netflix.titus.common.util.concurrency.CallbackCountDownLatch) CountDownLatch(java.util.concurrent.CountDownLatch) CallbackCountDownLatch(com.netflix.titus.common.util.concurrency.CallbackCountDownLatch) Cell(com.netflix.titus.api.federation.model.Cell) CellConnectorUtil.callToCell(com.netflix.titus.federation.service.CellConnectorUtil.callToCell) JobManagementServiceStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub)

Example 4 with JobManagementServiceStub

use of com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub in project titus-control-plane by Netflix.

the class TitusClientImplTest method setup.

@Before
public void setup() throws IOException {
    final MockJobManagerService mockJobManagerService = new MockJobManagerService();
    testServer = InProcessServerBuilder.forName("testServer").directExecutor().addService(mockJobManagerService).build().start();
    final ManagedChannel channel = InProcessChannelBuilder.forName("testServer").directExecutor().usePlaintext().build();
    final JobManagementServiceStub jobManagementServiceStub = JobManagementServiceGrpc.newStub(channel);
    final JobManagementServiceFutureStub jobManagementServiceFutureStub = JobManagementServiceGrpc.newFutureStub(channel);
    titusClient = new TitusClientImpl(jobManagementServiceStub, jobManagementServiceFutureStub, new DefaultRegistry());
}
Also used : JobManagementServiceFutureStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceFutureStub) ManagedChannel(io.grpc.ManagedChannel) DefaultRegistry(com.netflix.spectator.api.DefaultRegistry) JobManagementServiceStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub) Before(org.junit.Before)

Aggregations

JobManagementServiceStub (com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub)4 Cell (com.netflix.titus.api.federation.model.Cell)3 CellConnectorUtil.callToCell (com.netflix.titus.federation.service.CellConnectorUtil.callToCell)2 JobDescriptor (com.netflix.titus.grpc.protogen.JobDescriptor)2 JobManagementServiceGrpc (com.netflix.titus.grpc.protogen.JobManagementServiceGrpc)2 DefaultRegistry (com.netflix.spectator.api.DefaultRegistry)1 CallbackCountDownLatch (com.netflix.titus.common.util.concurrency.CallbackCountDownLatch)1 EmitterWithMultipleSubscriptions (com.netflix.titus.common.util.rx.EmitterWithMultipleSubscriptions)1 Job (com.netflix.titus.grpc.protogen.Job)1 JobChangeNotification (com.netflix.titus.grpc.protogen.JobChangeNotification)1 JobId (com.netflix.titus.grpc.protogen.JobId)1 JobManagementServiceFutureStub (com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceFutureStub)1 TestStreamObserver (com.netflix.titus.testkit.grpc.TestStreamObserver)1 ManagedChannel (io.grpc.ManagedChannel)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 Before (org.junit.Before)1 Test (org.junit.Test)1