Search in sources :

Example 1 with ApplicationList

use of com.yahoo.vespa.hosted.controller.application.ApplicationList in project vespa by vespa-engine.

the class VersionStatus method computeDeploymentStatistics.

private static Collection<DeploymentStatistics> computeDeploymentStatistics(Set<Version> infrastructureVersions, List<Application> applications) {
    Map<Version, DeploymentStatistics> versionMap = new HashMap<>();
    for (Version infrastructureVersion : infrastructureVersions) {
        versionMap.put(infrastructureVersion, DeploymentStatistics.empty(infrastructureVersion));
    }
    ApplicationList applicationList = ApplicationList.from(applications).notPullRequest().hasProductionDeployment();
    for (Application application : applicationList.asList()) {
        // (ignore non-production versions)
        for (Deployment deployment : application.productionDeployments().values()) {
            versionMap.computeIfAbsent(deployment.version(), DeploymentStatistics::empty);
        }
        // List versions which have failing jobs, versions which are in production, and versions for which there are running deployment jobs
        // Failing versions
        JobList.from(application).failing().not().failingApplicationChange().not().failingBecause(outOfCapacity).mapToList(job -> job.lastCompleted().get().version()).forEach(version -> versionMap.put(version, versionMap.getOrDefault(version, DeploymentStatistics.empty(version)).withFailing(application.id())));
        // Succeeding versions
        JobList.from(application).lastSuccess().present().production().mapToList(job -> job.lastSuccess().get().version()).forEach(version -> versionMap.put(version, versionMap.getOrDefault(version, DeploymentStatistics.empty(version)).withProduction(application.id())));
        // Deploying versions
        JobList.from(application).upgrading().mapToList(job -> job.lastTriggered().get().version()).forEach(version -> versionMap.put(version, versionMap.getOrDefault(version, DeploymentStatistics.empty(version)).withDeploying(application.id())));
    }
    return versionMap.values();
}
Also used : JobError.outOfCapacity(com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobError.outOfCapacity) Version(com.yahoo.component.Version) Vtag(com.yahoo.component.Vtag) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Level(java.util.logging.Level) ApplicationList(com.yahoo.vespa.hosted.controller.application.ApplicationList) HashSet(java.util.HashSet) ZoneId(com.yahoo.vespa.hosted.controller.api.integration.zone.ZoneId) ImmutableList(com.google.common.collect.ImmutableList) JobList(com.yahoo.vespa.hosted.controller.application.JobList) Map(java.util.Map) URI(java.net.URI) Application(com.yahoo.vespa.hosted.controller.Application) Collection(java.util.Collection) Set(java.util.Set) Instant(java.time.Instant) Logger(java.util.logging.Logger) Collectors(java.util.stream.Collectors) GitSha(com.yahoo.vespa.hosted.controller.api.integration.github.GitSha) List(java.util.List) Optional(java.util.Optional) Deployment(com.yahoo.vespa.hosted.controller.application.Deployment) Collections(java.util.Collections) ListMap(com.yahoo.collections.ListMap) Controller(com.yahoo.vespa.hosted.controller.Controller) Version(com.yahoo.component.Version) HashMap(java.util.HashMap) ApplicationList(com.yahoo.vespa.hosted.controller.application.ApplicationList) Deployment(com.yahoo.vespa.hosted.controller.application.Deployment) Application(com.yahoo.vespa.hosted.controller.Application)

Example 2 with ApplicationList

use of com.yahoo.vespa.hosted.controller.application.ApplicationList in project vespa by vespa-engine.

the class VespaVersion method confidenceFrom.

public static Confidence confidenceFrom(DeploymentStatistics statistics, Controller controller) {
    // 'production on this': All deployment jobs upgrading to this version have completed without failure
    ApplicationList productionOnThis = ApplicationList.from(statistics.production(), controller.applications()).notUpgradingTo(statistics.version()).notFailing();
    ApplicationList failingOnThis = ApplicationList.from(statistics.failing(), controller.applications());
    ApplicationList all = ApplicationList.from(controller.applications().asList()).hasDeployment().notPullRequest();
    // 'broken' if any Canary fails
    if (!failingOnThis.with(UpgradePolicy.canary).isEmpty())
        return Confidence.broken;
    // 'broken' if 4 non-canary was broken by this, and that is at least 10% of all
    if (nonCanaryApplicationsBroken(statistics.version(), failingOnThis, productionOnThis))
        return Confidence.broken;
    // 'low' unless all canary applications are upgraded
    if (productionOnThis.with(UpgradePolicy.canary).size() < all.with(UpgradePolicy.canary).size())
        return Confidence.low;
    // 'high' if 90% of all default upgrade applications upgraded
    if (productionOnThis.with(UpgradePolicy.defaultPolicy).size() >= all.with(UpgradePolicy.defaultPolicy).size() * 0.9)
        return Confidence.high;
    return Confidence.normal;
}
Also used : ApplicationList(com.yahoo.vespa.hosted.controller.application.ApplicationList)

Example 3 with ApplicationList

use of com.yahoo.vespa.hosted.controller.application.ApplicationList in project vespa by vespa-engine.

the class VespaVersion method nonCanaryApplicationsBroken.

private static boolean nonCanaryApplicationsBroken(Version version, ApplicationList failingOnThis, ApplicationList productionOnThis) {
    ApplicationList failingNonCanaries = failingOnThis.without(UpgradePolicy.canary).startedFailingOn(version);
    ApplicationList productionNonCanaries = productionOnThis.without(UpgradePolicy.canary);
    if (productionNonCanaries.size() + failingNonCanaries.size() == 0)
        return false;
    // 'broken' if 4 non-canary was broken by this, and that is at least 10% of all
    int brokenByThisVersion = failingNonCanaries.size();
    return brokenByThisVersion >= 4 && brokenByThisVersion >= productionOnThis.size() * 0.1;
}
Also used : ApplicationList(com.yahoo.vespa.hosted.controller.application.ApplicationList)

Example 4 with ApplicationList

use of com.yahoo.vespa.hosted.controller.application.ApplicationList in project vespa by vespa-engine.

the class DeploymentTrigger method triggerReadyJobs.

/**
 * Find jobs that can and should run but are currently not.
 */
public void triggerReadyJobs() {
    ApplicationList applications = ApplicationList.from(applications().asList());
    applications = applications.notPullRequest();
    for (Application application : applications.asList()) applications().lockIfPresent(application.id(), this::triggerReadyJobs);
}
Also used : ApplicationList(com.yahoo.vespa.hosted.controller.application.ApplicationList) Application(com.yahoo.vespa.hosted.controller.Application) LockedApplication(com.yahoo.vespa.hosted.controller.LockedApplication)

Aggregations

ApplicationList (com.yahoo.vespa.hosted.controller.application.ApplicationList)4 Application (com.yahoo.vespa.hosted.controller.Application)2 ImmutableList (com.google.common.collect.ImmutableList)1 ListMap (com.yahoo.collections.ListMap)1 Version (com.yahoo.component.Version)1 Vtag (com.yahoo.component.Vtag)1 Controller (com.yahoo.vespa.hosted.controller.Controller)1 LockedApplication (com.yahoo.vespa.hosted.controller.LockedApplication)1 GitSha (com.yahoo.vespa.hosted.controller.api.integration.github.GitSha)1 ZoneId (com.yahoo.vespa.hosted.controller.api.integration.zone.ZoneId)1 Deployment (com.yahoo.vespa.hosted.controller.application.Deployment)1 JobError.outOfCapacity (com.yahoo.vespa.hosted.controller.application.DeploymentJobs.JobError.outOfCapacity)1 JobList (com.yahoo.vespa.hosted.controller.application.JobList)1 URI (java.net.URI)1 Instant (java.time.Instant)1 ArrayList (java.util.ArrayList)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1