Search in sources :

Example 46 with ApplicationPackage

use of com.yahoo.vespa.hosted.controller.application.ApplicationPackage in project vespa by vespa-engine.

the class FailureRedeployerTest method testRetryingFailedJobsDuringDeployment.

@Test
public void testRetryingFailedJobsDuringDeployment() {
    DeploymentTester tester = new DeploymentTester();
    ApplicationPackage applicationPackage = new ApplicationPackageBuilder().upgradePolicy("canary").environment(Environment.prod).region("us-east-3").build();
    Version version = Version.fromString("5.0");
    tester.updateVersionStatus(version);
    Application app = tester.createApplication("app1", "tenant1", 1, 11L);
    tester.jobCompletion(component).application(app).uploadArtifact(applicationPackage).submit();
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.systemTest);
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.stagingTest);
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.productionUsEast3);
    // New version is released
    version = Version.fromString("5.1");
    tester.updateVersionStatus(version);
    assertEquals(version, tester.controller().versionStatus().systemVersion().get().versionNumber());
    tester.upgrader().maintain();
    tester.readyJobTrigger().maintain();
    // Test environments pass
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.systemTest);
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.stagingTest);
    // Production job fails and is retried
    // Advance time so that we can detect jobs in progress
    tester.clock().advance(Duration.ofSeconds(1));
    tester.deployAndNotify(app, applicationPackage, false, DeploymentJobs.JobType.productionUsEast3);
    assertEquals("Production job is retried", 1, tester.deploymentQueue().jobs().size());
    assertEquals("Application has pending upgrade to " + version, version, tester.application(app.id()).change().platform().get());
    // Another version is released, which cancels any pending upgrades to lower versions
    version = Version.fromString("5.2");
    tester.updateVersionStatus(version);
    tester.upgrader().maintain();
    tester.readyJobTrigger().maintain();
    assertEquals("Application starts upgrading to new version", 1, tester.deploymentQueue().jobs().size());
    assertEquals("Application has pending upgrade to " + version, version, tester.application(app.id()).change().platform().get());
    // Failure re-deployer does not retry failing job for prod.us-east-3, since it no longer has an available change
    tester.clock().advance(Duration.ofMinutes(1));
    tester.jobCompletion(DeploymentJobs.JobType.productionUsEast3).application(app).unsuccessful().submit();
    assertFalse("Job is not retried", tester.deploymentQueue().jobs().stream().anyMatch(j -> j.jobName().equals(DeploymentJobs.JobType.productionUsEast3.jobName())));
    // Test environments pass
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.systemTest);
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.stagingTest);
    // Production job fails again, and is retried
    tester.deployAndNotify(app, applicationPackage, false, DeploymentJobs.JobType.productionUsEast3);
    tester.readyJobTrigger().maintain();
    assertEquals("Job is retried", Collections.singletonList(new BuildService.BuildJob(app.deploymentJobs().projectId().get(), productionUsEast3.jobName())), tester.deploymentQueue().jobs());
    // Production job finally succeeds
    tester.deployAndNotify(app, applicationPackage, true, DeploymentJobs.JobType.productionUsEast3);
    assertTrue("All jobs consumed", tester.deploymentQueue().jobs().isEmpty());
    assertFalse("No failures", tester.application(app.id()).deploymentJobs().hasFailures());
}
Also used : Collection(com.yahoo.log.event.Collection) Version(com.yahoo.component.Version) Files(java.nio.file.Files) Environment(com.yahoo.config.provision.Environment) Slime(com.yahoo.slime.Slime) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) DeploymentJobs(com.yahoo.vespa.hosted.controller.application.DeploymentJobs) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) JobType.systemTest(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.systemTest) ZoneId(com.yahoo.vespa.hosted.controller.api.integration.zone.ZoneId) SlimeUtils(com.yahoo.vespa.config.SlimeUtils) Paths(java.nio.file.Paths) Assert.assertFalse(org.junit.Assert.assertFalse) Duration(java.time.Duration) SystemName(com.yahoo.config.provision.SystemName) BuildService(com.yahoo.vespa.hosted.controller.api.integration.BuildService) JobType.productionUsEast3(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.productionUsEast3) Collections(java.util.Collections) JobType.component(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.component) Application(com.yahoo.vespa.hosted.controller.Application) Assert.assertEquals(org.junit.Assert.assertEquals) DeploymentTester(com.yahoo.vespa.hosted.controller.deployment.DeploymentTester) Version(com.yahoo.component.Version) DeploymentTester(com.yahoo.vespa.hosted.controller.deployment.DeploymentTester) ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) Application(com.yahoo.vespa.hosted.controller.Application) Test(org.junit.Test) JobType.systemTest(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.systemTest)

Example 47 with ApplicationPackage

use of com.yahoo.vespa.hosted.controller.application.ApplicationPackage in project vespa by vespa-engine.

the class RotationTest method application_with_only_one_non_corp_region.

@Test
public void application_with_only_one_non_corp_region() {
    ApplicationPackage applicationPackage = new ApplicationPackageBuilder().globalServiceId("foo").region("us-east-3").region("corp-us-east-1").build();
    Application application = tester.createApplication("app2", "tenant2", 22L, 2L);
    thrown.expect(RuntimeException.class);
    thrown.expectMessage("less than 2 prod zones are defined");
    tester.deployCompletely(application, applicationPackage);
}
Also used : ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) Application(com.yahoo.vespa.hosted.controller.Application) Test(org.junit.Test)

Example 48 with ApplicationPackage

use of com.yahoo.vespa.hosted.controller.application.ApplicationPackage in project vespa by vespa-engine.

the class RotationTest method application_with_corp_region_and_two_non_corp_region.

@Test
public void application_with_corp_region_and_two_non_corp_region() {
    ApplicationPackage applicationPackage = new ApplicationPackageBuilder().globalServiceId("foo").region("us-east-3").region("corp-us-east-1").region("us-west-1").build();
    Application application = tester.createApplication("app2", "tenant2", 22L, 2L);
    tester.deployCompletely(application, applicationPackage);
    assertEquals(new RotationId("foo-1"), tester.applications().require(application.id()).rotation().get().id());
}
Also used : ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) Application(com.yahoo.vespa.hosted.controller.Application) Test(org.junit.Test)

Example 49 with ApplicationPackage

use of com.yahoo.vespa.hosted.controller.application.ApplicationPackage in project vespa by vespa-engine.

the class RotationTest method too_few_zones.

@Test
public void too_few_zones() {
    ApplicationPackage applicationPackage = new ApplicationPackageBuilder().globalServiceId("foo").region("us-east-3").build();
    Application application = tester.createApplication("app2", "tenant2", 22L, 2L);
    thrown.expect(RuntimeException.class);
    thrown.expectMessage("less than 2 prod zones are defined");
    tester.deployCompletely(application, applicationPackage);
}
Also used : ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) Application(com.yahoo.vespa.hosted.controller.Application) Test(org.junit.Test)

Example 50 with ApplicationPackage

use of com.yahoo.vespa.hosted.controller.application.ApplicationPackage in project vespa by vespa-engine.

the class VersionStatusTest method testVersionStatusAfterApplicationUpdates.

@Test
public void testVersionStatusAfterApplicationUpdates() {
    DeploymentTester tester = new DeploymentTester();
    ApplicationPackage applicationPackage = new ApplicationPackageBuilder().upgradePolicy("default").environment(Environment.prod).region("us-west-1").region("us-east-3").build();
    Version version1 = new Version("5.1");
    Version version2 = new Version("5.2");
    tester.upgradeSystem(version1);
    // Setup applications
    Application app1 = tester.createAndDeploy("app1", 11, applicationPackage);
    Application app2 = tester.createAndDeploy("app2", 22, applicationPackage);
    Application app3 = tester.createAndDeploy("app3", 33, applicationPackage);
    // version2 is released
    tester.upgradeSystem(version2);
    // - app1 is in production on version1, but then fails in system test on version2
    tester.completeUpgradeWithError(app1, version2, applicationPackage, systemTest);
    // - app2 is partially in production on version1 and partially on version2
    tester.completeUpgradeWithError(app2, version2, applicationPackage, productionUsEast3);
    // - app3 is in production on version1, but then fails in staging test on version2
    tester.completeUpgradeWithError(app3, version2, applicationPackage, stagingTest);
    tester.updateVersionStatus();
    List<VespaVersion> versions = tester.controller().versionStatus().versions();
    assertEquals("The two versions above exist", 2, versions.size());
    System.err.println(tester.controller().applications().deploymentTrigger().jobTimeoutLimit());
    VespaVersion v1 = versions.get(0);
    assertEquals(version1, v1.versionNumber());
    assertEquals("No applications are failing on version1.", ImmutableSet.of(), v1.statistics().failing());
    assertEquals("All applications have at least one active production deployment on version 1.", ImmutableSet.of(app1.id(), app2.id(), app3.id()), v1.statistics().production());
    assertEquals("No applications have active deployment jobs on version1.", ImmutableSet.of(), v1.statistics().deploying());
    VespaVersion v2 = versions.get(1);
    assertEquals(version2, v2.versionNumber());
    assertEquals("All applications have failed on version2 in at least one zone.", ImmutableSet.of(app1.id(), app2.id(), app3.id()), v2.statistics().failing());
    assertEquals("Only app2 has successfully deployed to production on version2.", ImmutableSet.of(app2.id()), v2.statistics().production());
// Should test the below, but can't easily be done with current test framework. This test passes in DeploymentApiTest.
// assertEquals("All applications are being retried on version2.", ImmutableSet.of(app1.id(), app2.id(), app3.id()), v2.statistics().deploying());
}
Also used : Version(com.yahoo.component.Version) DeploymentTester(com.yahoo.vespa.hosted.controller.deployment.DeploymentTester) ApplicationPackageBuilder(com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder) ApplicationPackage(com.yahoo.vespa.hosted.controller.application.ApplicationPackage) Application(com.yahoo.vespa.hosted.controller.Application) Test(org.junit.Test) JobType.systemTest(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.systemTest) JobType.stagingTest(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.stagingTest)

Aggregations

ApplicationPackage (com.yahoo.vespa.hosted.controller.application.ApplicationPackage)50 Test (org.junit.Test)44 Application (com.yahoo.vespa.hosted.controller.Application)37 ApplicationPackageBuilder (com.yahoo.vespa.hosted.controller.deployment.ApplicationPackageBuilder)33 JobType.systemTest (com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.systemTest)31 JobType.stagingTest (com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobType.stagingTest)26 Version (com.yahoo.component.Version)23 DeploymentTester (com.yahoo.vespa.hosted.controller.deployment.DeploymentTester)22 VespaVersion (com.yahoo.vespa.hosted.controller.versions.VespaVersion)11 LockedApplication (com.yahoo.vespa.hosted.controller.LockedApplication)10 ApplicationVersion (com.yahoo.vespa.hosted.controller.application.ApplicationVersion)10 TenantId (com.yahoo.vespa.hosted.controller.api.identifiers.TenantId)8 ApplicationId (com.yahoo.config.provision.ApplicationId)5 ManualClock (com.yahoo.test.ManualClock)5 ControllerTester (com.yahoo.vespa.hosted.controller.ControllerTester)5 ZoneId (com.yahoo.vespa.hosted.controller.api.integration.zone.ZoneId)5 Environment (com.yahoo.config.provision.Environment)4 DeployOptions (com.yahoo.vespa.hosted.controller.api.application.v4.model.DeployOptions)4 BuildService (com.yahoo.vespa.hosted.controller.api.integration.BuildService)4 Record (com.yahoo.vespa.hosted.controller.api.integration.dns.Record)4