Search in sources :

Example 31 with TestStreamObserver

use of com.netflix.titus.testkit.grpc.TestStreamObserver in project titus-control-plane by Netflix.

the class JobScenarioBuilder method updateJobStatus.

public JobScenarioBuilder updateJobStatus(boolean enabled) {
    logger.info("[{}] Changing job {} enable status to {}...", discoverActiveTest(), jobId, enabled);
    Stopwatch stopWatch = Stopwatch.createStarted();
    TestStreamObserver<Empty> responseObserver = new TestStreamObserver<>();
    client.updateJobStatus(JobStatusUpdate.newBuilder().setId(jobId).setEnableStatus(enabled).build(), responseObserver);
    rethrow(() -> responseObserver.awaitDone(TIMEOUT_MS, TimeUnit.MILLISECONDS));
    expectJobUpdateEvent(job -> {
        ServiceJobExt ext = (ServiceJobExt) job.getJobDescriptor().getExtensions();
        return ext.isEnabled() == enabled;
    }, "Job status update did not complete in time");
    logger.info("[{}] Changing job {} enable status to {} finished in {}ms", discoverActiveTest(), jobId, enabled, stopWatch.elapsed(TimeUnit.MILLISECONDS));
    return this;
}
Also used : Empty(com.google.protobuf.Empty) TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) ServiceJobExt(com.netflix.titus.api.jobmanager.model.job.ext.ServiceJobExt) Stopwatch(com.google.common.base.Stopwatch)

Example 32 with TestStreamObserver

use of com.netflix.titus.testkit.grpc.TestStreamObserver in project titus-control-plane by Netflix.

the class JobObserveTest method observeJobs.

@Test(timeout = LONG_TEST_TIMEOUT_MS)
public void observeJobs() throws InterruptedException {
    TestStreamObserver<JobChangeNotification> subscriber = subscribe(ObserveJobsQuery.getDefaultInstance());
    for (int i = 0; i < 2; i++) {
        String jobId = jobsScenarioBuilder.scheduleAndReturnJob(oneTaskBatchJobDescriptor(), jobScenarioBuilder -> jobScenarioBuilder.template(startTasksInNewJob()).template(ScenarioTemplates.killJob())).getId();
        JobChangeNotification event;
        while (true) {
            event = subscriber.takeNext(SHORT_TIMEOUT_MS, TimeUnit.MILLISECONDS);
            if (event.hasJobUpdate()) {
                Job job = event.getJobUpdate().getJob();
                assertThat(job.getId()).isEqualTo(jobId);
                CellAssertions.assertCellInfo(job, EmbeddedTitusMaster.CELL_NAME);
                if (job.getStatus().getState() == JobState.Finished) {
                    break;
                }
            }
        }
    }
    subscriber.cancel();
}
Also used : Arrays(java.util.Arrays) JobDescriptorGenerator.serviceJobDescriptors(com.netflix.titus.testkit.model.job.JobDescriptorGenerator.serviceJobDescriptors) JobModel(com.netflix.titus.api.jobmanager.model.job.JobModel) ObserveJobsQuery(com.netflix.titus.grpc.protogen.ObserveJobsQuery) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) Job(com.netflix.titus.grpc.protogen.Job) ScenarioTemplates(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates) ScenarioTemplates.startTasksInNewJob(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates.startTasksInNewJob) Image(com.netflix.titus.grpc.protogen.Image) ScenarioTemplates.killJob(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates.killJob) JobState(com.netflix.titus.grpc.protogen.JobStatus.JobState) TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) Owner(com.netflix.titus.api.jobmanager.model.job.Owner) SERVICE(com.netflix.titus.grpc.protogen.JobDescriptor.JobSpecCase.SERVICE) Assert.fail(org.junit.Assert.fail) Before(org.junit.Before) BaseIntegrationTest(com.netflix.titus.master.integration.BaseIntegrationTest) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) TaskState(com.netflix.titus.grpc.protogen.TaskStatus.TaskState) EmbeddedTitusCells(com.netflix.titus.testkit.embedded.cell.EmbeddedTitusCells) ImmutableMap(com.google.common.collect.ImmutableMap) Task(com.netflix.titus.grpc.protogen.Task) JobDescriptorGenerator.oneTaskBatchJobDescriptor(com.netflix.titus.testkit.model.job.JobDescriptorGenerator.oneTaskBatchJobDescriptor) CellAssertions(com.netflix.titus.master.integration.v3.job.CellAssertions) IntegrationTest(com.netflix.titus.testkit.junit.category.IntegrationTest) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) RuleChain(org.junit.rules.RuleChain) TitusStackResource(com.netflix.titus.testkit.junit.master.TitusStackResource) Rule(org.junit.Rule) JobDescriptorGenerator.batchJobDescriptors(com.netflix.titus.testkit.model.job.JobDescriptorGenerator.batchJobDescriptors) EmbeddedTitusMaster(com.netflix.titus.testkit.embedded.cell.master.EmbeddedTitusMaster) JobsScenarioBuilder(com.netflix.titus.master.integration.v3.scenario.JobsScenarioBuilder) JobManagementServiceStub(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc.JobManagementServiceStub) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) Job(com.netflix.titus.grpc.protogen.Job) ScenarioTemplates.startTasksInNewJob(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates.startTasksInNewJob) ScenarioTemplates.killJob(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates.killJob) BaseIntegrationTest(com.netflix.titus.master.integration.BaseIntegrationTest) IntegrationTest(com.netflix.titus.testkit.junit.category.IntegrationTest) Test(org.junit.Test)

Example 33 with TestStreamObserver

use of com.netflix.titus.testkit.grpc.TestStreamObserver in project titus-control-plane by Netflix.

the class JobsScenarioBuilder method loadJobs.

private List<JobScenarioBuilder> loadJobs() {
    JobQuery query = JobQuery.newBuilder().setPage(Page.newBuilder().setPageSize(1000)).build();
    Throwable lastFailure = null;
    // since we do not have retry interceptor installed, we make a few attempts directly.
    for (int i = 0; i < 3; i++) {
        try {
            TestStreamObserver<JobQueryResult> responseObserver = new TestStreamObserver<>();
            client.findJobs(query, responseObserver);
            JobQueryResult queryResult = rethrow(() -> responseObserver.takeNext(TIMEOUT_MS, TimeUnit.MILLISECONDS));
            List<JobScenarioBuilder> result = new ArrayList<>();
            queryResult.getItemsList().forEach(job -> {
                result.add(new JobScenarioBuilder(titusOperations, this, job.getId()));
            });
            return result;
        } catch (Exception e) {
            lastFailure = e;
            logger.info("Cannot load jobs from TitusMaster (might be not ready yet). Waiting 1sec before next try...");
            try {
                Thread.sleep(1_000);
            } catch (InterruptedException ignore) {
            }
        }
    }
    throw new IllegalStateException("Cannot load jobs: " + lastFailure, lastFailure);
}
Also used : TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) ArrayList(java.util.ArrayList) JobQuery(com.netflix.titus.grpc.protogen.JobQuery) JobQueryResult(com.netflix.titus.grpc.protogen.JobQueryResult)

Example 34 with TestStreamObserver

use of com.netflix.titus.testkit.grpc.TestStreamObserver in project titus-control-plane by Netflix.

the class JobsScenarioBuilder method schedule.

public JobsScenarioBuilder schedule(JobDescriptor jobDescriptor, Function<JobScenarioBuilder, JobScenarioBuilder> jobScenario) {
    TestStreamObserver<JobId> responseObserver = new TestStreamObserver<>();
    client.withDeadlineAfter(TIMEOUT_MS, TimeUnit.MILLISECONDS).createJob(GrpcJobManagementModelConverters.toGrpcJobDescriptor(jobDescriptor), responseObserver);
    JobId jobId;
    try {
        jobId = responseObserver.takeNext(TIMEOUT_MS, TimeUnit.MILLISECONDS);
    } catch (Exception e) {
        throw new IllegalStateException(e);
    }
    Preconditions.checkNotNull(jobId, "Job create operation not completed in time");
    TestStreamObserver<JobChangeNotification> eventStream = new TestStreamObserver<>();
    client.observeJob(jobId, eventStream);
    JobScenarioBuilder jobScenarioBuilder = new JobScenarioBuilder(titusOperations, this, jobId.getId());
    jobScenarioBuilders.add(jobScenarioBuilder);
    jobScenario.apply(jobScenarioBuilder);
    return this;
}
Also used : TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) JobId(com.netflix.titus.grpc.protogen.JobId)

Example 35 with TestStreamObserver

use of com.netflix.titus.testkit.grpc.TestStreamObserver in project titus-control-plane by Netflix.

the class TaskScenarioBuilder method deleteTaskAttributes.

public TaskScenarioBuilder deleteTaskAttributes(List<String> keys) {
    String taskId = getTask().getId();
    logger.info("[{}] Deleting attributes of task {} of job {}...", discoverActiveTest(), taskId, jobScenarioBuilder.getJobId());
    Stopwatch stopWatch = Stopwatch.createStarted();
    TestStreamObserver<Empty> responseObserver = new TestStreamObserver<>();
    jobClient.deleteTaskAttributes(TaskAttributesDeleteRequest.newBuilder().setTaskId(taskId).addAllKeys(keys).build(), responseObserver);
    rethrow(() -> responseObserver.awaitDone(TIMEOUT_MS, TimeUnit.MILLISECONDS));
    logger.info("[{}] Task {} updated in {}[ms]", discoverActiveTest(), taskId, stopWatch.elapsed(TimeUnit.MILLISECONDS));
    return this;
}
Also used : Empty(com.google.protobuf.Empty) TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) Stopwatch(com.google.common.base.Stopwatch)

Aggregations

TestStreamObserver (com.netflix.titus.testkit.grpc.TestStreamObserver)42 Empty (com.google.protobuf.Empty)24 Test (org.junit.Test)19 GetPolicyResult (com.netflix.titus.grpc.protogen.GetPolicyResult)14 Stopwatch (com.google.common.base.Stopwatch)13 ScalingPolicyID (com.netflix.titus.grpc.protogen.ScalingPolicyID)12 IntegrationTest (com.netflix.titus.testkit.junit.category.IntegrationTest)11 BaseIntegrationTest (com.netflix.titus.master.integration.BaseIntegrationTest)9 ServiceJobExt (com.netflix.titus.api.jobmanager.model.job.ext.ServiceJobExt)6 PutPolicyRequest (com.netflix.titus.grpc.protogen.PutPolicyRequest)6 JobId (com.netflix.titus.grpc.protogen.JobId)5 HashSet (java.util.HashSet)4 Capacity (com.netflix.titus.api.jobmanager.model.job.Capacity)3 GrpcJobManagementModelConverters.toGrpcCapacity (com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobManagementModelConverters.toGrpcCapacity)3 DoubleValue (com.google.protobuf.DoubleValue)2 Job (com.netflix.titus.api.jobmanager.model.job.Job)2 JobDescriptor (com.netflix.titus.api.jobmanager.model.job.JobDescriptor)2 Task (com.netflix.titus.api.jobmanager.model.job.Task)2 BatchJobExt (com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt)2 DeletePolicyRequest (com.netflix.titus.grpc.protogen.DeletePolicyRequest)2