Search in sources :

Example 16 with ApplicationInstance

use of com.yahoo.vespa.applicationmodel.ApplicationInstance in project vespa by vespa-engine.

the class OrchestratorImpl method suspend.

@Override
public void suspend(HostName hostName) throws HostStateChangeDeniedException, HostNameNotFoundException {
    ApplicationInstance appInstance = getApplicationInstance(hostName);
    NodeGroup nodeGroup = new NodeGroup(appInstance, hostName);
    suspendGroup(nodeGroup);
}
Also used : ApplicationInstance(com.yahoo.vespa.applicationmodel.ApplicationInstance) NodeGroup(com.yahoo.vespa.orchestrator.model.NodeGroup)

Example 17 with ApplicationInstance

use of com.yahoo.vespa.applicationmodel.ApplicationInstance in project vespa by vespa-engine.

the class OrchestratorImpl method setApplicationStatus.

private void setApplicationStatus(ApplicationId appId, ApplicationInstanceStatus status) throws ApplicationStateChangeDeniedException, ApplicationIdNotFoundException {
    ApplicationInstanceReference appRef = OrchestratorUtil.toApplicationInstanceReference(appId, instanceLookupService);
    try (MutableStatusRegistry statusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(appRef)) {
        // Short-circuit if already in wanted state
        if (status == statusRegistry.getApplicationInstanceStatus())
            return;
        // Set content clusters for this application in maintenance on suspend
        if (status == ApplicationInstanceStatus.ALLOWED_TO_BE_DOWN) {
            ApplicationInstance application = getApplicationInstance(appRef);
            // Mark it allowed to be down before we manipulate the clustercontroller
            OrchestratorUtil.getHostsUsedByApplicationInstance(application).forEach(h -> statusRegistry.setHostState(h, HostStatus.ALLOWED_TO_BE_DOWN));
            // If the clustercontroller throws an error the nodes will be marked as allowed to be down
            // and be set back up on next resume invocation.
            setClusterStateInController(application, ClusterControllerNodeState.MAINTENANCE);
        }
        statusRegistry.setApplicationInstanceStatus(status);
    }
}
Also used : ApplicationInstance(com.yahoo.vespa.applicationmodel.ApplicationInstance) MutableStatusRegistry(com.yahoo.vespa.orchestrator.status.MutableStatusRegistry) ApplicationInstanceReference(com.yahoo.vespa.applicationmodel.ApplicationInstanceReference)

Example 18 with ApplicationInstance

use of com.yahoo.vespa.applicationmodel.ApplicationInstance in project vespa by vespa-engine.

the class OrchestratorImpl method acquirePermissionToRemove.

@Override
public void acquirePermissionToRemove(HostName hostName) throws OrchestrationException {
    ApplicationInstance appInstance = getApplicationInstance(hostName);
    NodeGroup nodeGroup = new NodeGroup(appInstance, hostName);
    try (MutableStatusRegistry statusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(appInstance.reference())) {
        ApplicationApi applicationApi = new ApplicationApiImpl(nodeGroup, statusRegistry, clusterControllerClientFactory);
        policy.acquirePermissionToRemove(applicationApi);
    }
}
Also used : ApplicationInstance(com.yahoo.vespa.applicationmodel.ApplicationInstance) ApplicationApi(com.yahoo.vespa.orchestrator.model.ApplicationApi) ApplicationApiImpl(com.yahoo.vespa.orchestrator.model.ApplicationApiImpl) MutableStatusRegistry(com.yahoo.vespa.orchestrator.status.MutableStatusRegistry) NodeGroup(com.yahoo.vespa.orchestrator.model.NodeGroup)

Example 19 with ApplicationInstance

use of com.yahoo.vespa.applicationmodel.ApplicationInstance in project vespa by vespa-engine.

the class OrchestratorImpl method nodeGroupsOrderedForSuspend.

/**
 * PROBLEM
 * Take the example of 2 Docker hosts:
 *  - Docker host 1 has two nodes A1 and B1, belonging to the application with
 *    a globally unique ID A and B, respectively.
 *  - Similarly, Docker host 2 has two nodes running content nodes A2 and B2,
 *    and we assume both A1 and A2 (and B1 and B2) have services within the same service cluster.
 *
 * Suppose both Docker hosts wanting to reboot, and
 *  - Docker host 1 asks to suspend A1 and B1, while
 *  - Docker host 2 asks to suspend B2 and A2.
 *
 * The Orchestrator may allow suspend of A1 and B2, before requesting the suspension of B1 and A2.
 * None of these can be suspended (assuming max 1 suspended content node per content cluster),
 * and so both requests for suspension will fail.
 *
 * Note that it's not a deadlock - both client will fail immediately and resume both A1 and B2 before
 * responding to the client, and if host 1 asks later w/o host 2 asking at the same time,
 * it will be given permission to suspend. However if both hosts were to request in lock-step,
 * there would be starvation. And in general, it would fail requests for suspension more
 * than necessary.
 *
 * SOLUTION
 * The solution we're using is to order the hostnames by the globally unique application instance ID,
 * e.g. hosted-vespa:routing:dev:some-region:default. In the example above, it would guarantee
 * Docker host 2 would ensure ask to suspend B2 before A2. We take care of that ordering here.
 *
 * NodeGroups complicate the above picture a little:  Each A1, A2, B1, and B2 is a NodeGroup that may
 * contain several nodes (on the same Docker host).  But the argument still applies.
 */
private List<NodeGroup> nodeGroupsOrderedForSuspend(List<HostName> hostNames) throws HostNameNotFoundException {
    Map<ApplicationInstanceReference, NodeGroup> nodeGroupMap = new HashMap<>(hostNames.size());
    for (HostName hostName : hostNames) {
        ApplicationInstance application = getApplicationInstance(hostName);
        NodeGroup nodeGroup = nodeGroupMap.get(application.reference());
        if (nodeGroup == null) {
            nodeGroup = new NodeGroup(application);
            nodeGroupMap.put(application.reference(), nodeGroup);
        }
        nodeGroup.addNode(hostName);
    }
    return nodeGroupMap.values().stream().sorted(OrchestratorImpl::compareNodeGroupsForSuspend).collect(Collectors.toList());
}
Also used : ApplicationInstance(com.yahoo.vespa.applicationmodel.ApplicationInstance) HashMap(java.util.HashMap) ApplicationInstanceReference(com.yahoo.vespa.applicationmodel.ApplicationInstanceReference) HostName(com.yahoo.vespa.applicationmodel.HostName) NodeGroup(com.yahoo.vespa.orchestrator.model.NodeGroup)

Example 20 with ApplicationInstance

use of com.yahoo.vespa.applicationmodel.ApplicationInstance in project vespa by vespa-engine.

the class NodeGroupTest method testBasics.

@Test
public void testBasics() {
    ApplicationInstance applicationInstance = new ApplicationInstance(new TenantId("tenant"), new ApplicationInstanceId("application-instance"), new HashSet<>());
    HostName hostName1 = new HostName("host1");
    HostName hostName2 = new HostName("host2");
    HostName hostName3 = new HostName("host3");
    NodeGroup nodeGroup = new NodeGroup(applicationInstance, hostName1, hostName3);
    nodeGroup.addNode(hostName2);
    // hostnames are sorted (for no good reason other than testability due to stability, readability)
    assertEquals(Arrays.asList(hostName1, hostName2, hostName3), nodeGroup.getHostNames());
    assertEquals("host1,host2,host3", nodeGroup.toCommaSeparatedString());
}
Also used : ApplicationInstanceId(com.yahoo.vespa.applicationmodel.ApplicationInstanceId) TenantId(com.yahoo.vespa.applicationmodel.TenantId) ApplicationInstance(com.yahoo.vespa.applicationmodel.ApplicationInstance) HostName(com.yahoo.vespa.applicationmodel.HostName) Test(org.junit.Test)

Aggregations

ApplicationInstance (com.yahoo.vespa.applicationmodel.ApplicationInstance)24 HostName (com.yahoo.vespa.applicationmodel.HostName)14 ApplicationInstanceReference (com.yahoo.vespa.applicationmodel.ApplicationInstanceReference)11 Test (org.junit.Test)10 ServiceType (com.yahoo.vespa.applicationmodel.ServiceType)9 ClusterId (com.yahoo.vespa.applicationmodel.ClusterId)6 ServiceCluster (com.yahoo.vespa.applicationmodel.ServiceCluster)6 ServiceInstance (com.yahoo.vespa.applicationmodel.ServiceInstance)6 ApplicationInstanceId (com.yahoo.vespa.applicationmodel.ApplicationInstanceId)5 TenantId (com.yahoo.vespa.applicationmodel.TenantId)5 HashMap (java.util.HashMap)5 ConfigId (com.yahoo.vespa.applicationmodel.ConfigId)4 NodeGroup (com.yahoo.vespa.orchestrator.model.NodeGroup)4 HostStatus (com.yahoo.vespa.orchestrator.status.HostStatus)4 MutableStatusRegistry (com.yahoo.vespa.orchestrator.status.MutableStatusRegistry)4 ServiceModel (com.yahoo.vespa.service.monitor.ServiceModel)4 List (java.util.List)4 Collectors (java.util.stream.Collectors)4 SuperModel (com.yahoo.config.model.api.SuperModel)3 ApplicationId (com.yahoo.config.provision.ApplicationId)3