Search in sources :

Example 6 with JobScenarioBuilder

use of com.netflix.titus.master.integration.v3.scenario.JobScenarioBuilder in project titus-control-plane by Netflix.

the class JobSubmitAndControlBasicTest method testSubmitBatchJobWithEfsMount.

/**
 * Verify batch job submit with the expected state transitions. Verify agent receives proper EFS mount data.
 */
@Test(timeout = 30_000)
public void testSubmitBatchJobWithEfsMount() {
    EfsMount efsMount1 = ContainersGenerator.efsMounts().getValue().toBuilder().withMountPoint("/data/logs").build();
    EfsMount efsMount2 = ContainersGenerator.efsMounts().skip(1).getValue().toBuilder().withMountPoint("/data").build();
    List<EfsMount> efsMounts = asList(efsMount1, efsMount2);
    List<EfsMount> expectedOrder = asList(efsMount2, efsMount1);
    JobDescriptor<BatchJobExt> jobWithEfs = ONE_TASK_BATCH_JOB.but(jd -> jd.getContainer().but(c -> c.getContainerResources().toBuilder().withEfsMounts(efsMounts)));
    jobsScenarioBuilder.schedule(jobWithEfs, jobScenarioBuilder -> jobScenarioBuilder.template(ScenarioTemplates.startTasksInNewJob()).assertEachPod(podWithEfsMounts(expectedOrder), "Container not assigned the expected EFS mount").allTasks(ScenarioTemplates.completeTask()).template(ScenarioTemplates.jobFinished()).expectJobEventStreamCompletes());
}
Also used : TaskScenarioBuilder(com.netflix.titus.master.integration.v3.scenario.TaskScenarioBuilder) AwsInstanceType(com.netflix.titus.common.aws.AwsInstanceType) JobModel(com.netflix.titus.api.jobmanager.model.job.JobModel) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) ScenarioTemplates(com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates) JobManagerConfiguration(com.netflix.titus.runtime.jobmanager.JobManagerConfiguration) V3_ENGINE_APP_PREFIX(com.netflix.titus.testkit.junit.master.TitusStackResource.V3_ENGINE_APP_PREFIX) EfsMount(com.netflix.titus.api.model.EfsMount) JobAsserts.podWithResources(com.netflix.titus.master.integration.v3.scenario.JobAsserts.podWithResources) Arrays.asList(java.util.Arrays.asList) After(org.junit.After) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) ContainersGenerator(com.netflix.titus.testkit.model.job.ContainersGenerator) JobDescriptorGenerator.oneTaskServiceJobDescriptor(com.netflix.titus.testkit.model.job.JobDescriptorGenerator.oneTaskServiceJobDescriptor) Before(org.junit.Before) BaseIntegrationTest(com.netflix.titus.master.integration.BaseIntegrationTest) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) TaskState(com.netflix.titus.grpc.protogen.TaskStatus.TaskState) JobAsserts.podWithEfsMounts(com.netflix.titus.master.integration.v3.scenario.JobAsserts.podWithEfsMounts) ServiceJobExt(com.netflix.titus.api.jobmanager.model.job.ext.ServiceJobExt) JobDescriptorGenerator.oneTaskBatchJobDescriptor(com.netflix.titus.testkit.model.job.JobDescriptorGenerator.oneTaskBatchJobDescriptor) IntegrationTest(com.netflix.titus.testkit.junit.category.IntegrationTest) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) RuleChain(org.junit.rules.RuleChain) TitusStackResource(com.netflix.titus.testkit.junit.master.TitusStackResource) List(java.util.List) Rule(org.junit.Rule) EmbeddedTitusCells.basicKubeCell(com.netflix.titus.testkit.embedded.cell.EmbeddedTitusCells.basicKubeCell) JobsScenarioBuilder(com.netflix.titus.master.integration.v3.scenario.JobsScenarioBuilder) JobAttributes(com.netflix.titus.api.jobmanager.JobAttributes) BatchJobExt(com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt) EfsMount(com.netflix.titus.api.model.EfsMount) BaseIntegrationTest(com.netflix.titus.master.integration.BaseIntegrationTest) IntegrationTest(com.netflix.titus.testkit.junit.category.IntegrationTest) Test(org.junit.Test)

Example 7 with JobScenarioBuilder

use of com.netflix.titus.master.integration.v3.scenario.JobScenarioBuilder in project titus-control-plane by Netflix.

the class JobsScenarioBuilder method schedule.

public JobsScenarioBuilder schedule(JobDescriptor<?> jobDescriptor, int times, Function<JobScenarioBuilder, JobScenarioBuilder> jobScenario) throws Exception {
    String sequence = jobDescriptor.getJobGroupInfo().getSequence();
    for (int i = 0; i < times; i++) {
        int finalI = i;
        JobDescriptor effective = sequence.isEmpty() ? jobDescriptor : jobDescriptor.but(jd -> jd.getJobGroupInfo().toBuilder().withSequence(sequence + '_' + finalI));
        schedule(effective, jobScenario);
    }
    return this;
}
Also used : TitusMasterResource(com.netflix.titus.testkit.junit.master.TitusMasterResource) LoggerFactory(org.slf4j.LoggerFactory) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) ArrayList(java.util.ArrayList) TestStreamObserver(com.netflix.titus.testkit.grpc.TestStreamObserver) JobQuery(com.netflix.titus.grpc.protogen.JobQuery) JobQueryResult(com.netflix.titus.grpc.protogen.JobQueryResult) EmbeddedTitusOperations(com.netflix.titus.testkit.embedded.EmbeddedTitusOperations) TIMEOUT_MS(com.netflix.titus.master.integration.v3.scenario.ScenarioBuilderUtil.TIMEOUT_MS) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor) Job(com.netflix.titus.api.jobmanager.model.job.Job) Logger(org.slf4j.Logger) Predicate(java.util.function.Predicate) JobManagementServiceGrpc(com.netflix.titus.grpc.protogen.JobManagementServiceGrpc) GrpcJobManagementModelConverters(com.netflix.titus.runtime.endpoint.v3.grpc.GrpcJobManagementModelConverters) Collectors(java.util.stream.Collectors) Page(com.netflix.titus.grpc.protogen.Page) TimeUnit(java.util.concurrent.TimeUnit) TitusStackResource(com.netflix.titus.testkit.junit.master.TitusStackResource) List(java.util.List) ExternalResource(org.junit.rules.ExternalResource) Preconditions(com.google.common.base.Preconditions) ExceptionExt.rethrow(com.netflix.titus.common.util.ExceptionExt.rethrow) JobChangeNotification(com.netflix.titus.grpc.protogen.JobChangeNotification) JobId(com.netflix.titus.grpc.protogen.JobId) JobDescriptor(com.netflix.titus.api.jobmanager.model.job.JobDescriptor)

Aggregations

JobDescriptor (com.netflix.titus.api.jobmanager.model.job.JobDescriptor)7 TitusStackResource (com.netflix.titus.testkit.junit.master.TitusStackResource)7 BaseIntegrationTest (com.netflix.titus.master.integration.BaseIntegrationTest)6 JobsScenarioBuilder (com.netflix.titus.master.integration.v3.scenario.JobsScenarioBuilder)6 ScenarioTemplates (com.netflix.titus.master.integration.v3.scenario.ScenarioTemplates)6 IntegrationTest (com.netflix.titus.testkit.junit.category.IntegrationTest)6 Test (org.junit.Test)6 Category (org.junit.experimental.categories.Category)6 RuleChain (org.junit.rules.RuleChain)6 BatchJobExt (com.netflix.titus.api.jobmanager.model.job.ext.BatchJobExt)5 JobDescriptorGenerator.oneTaskBatchJobDescriptor (com.netflix.titus.testkit.model.job.JobDescriptorGenerator.oneTaskBatchJobDescriptor)5 List (java.util.List)5 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)5 ServiceJobExt (com.netflix.titus.api.jobmanager.model.job.ext.ServiceJobExt)4 JobManagementServiceGrpc (com.netflix.titus.grpc.protogen.JobManagementServiceGrpc)4 Page (com.netflix.titus.grpc.protogen.Page)4 ImmutableMap (com.google.common.collect.ImmutableMap)3 JobFunctions (com.netflix.titus.api.jobmanager.model.job.JobFunctions)3 JobState (com.netflix.titus.api.jobmanager.model.job.JobState)3 Owner (com.netflix.titus.api.jobmanager.model.job.Owner)3