use of com.yahoo.vespa.orchestrator.status.MutableStatusRegistry in project vespa by vespa-engine.
the class OrchestratorImpl method resume.
@Override
public void resume(HostName hostName) throws HostStateChangeDeniedException, HostNameNotFoundException {
/*
* When making a state transition to this state, we have to consider that if the host has been in
* ALLOWED_TO_BE_DOWN state, services on the host may recently have been stopped (and, presumably, started).
* Service monitoring may not have had enough time to detect that services were stopped,
* and may therefore mistakenly report services as up, even if they still haven't initialized and
* are not yet ready for serving. Erroneously reporting both host and services as up causes a race
* where services on other hosts may be stopped prematurely. A delay here ensures that service
* monitoring will have had time to catch up. Since we don't want do the delay with the lock held,
* and the host status service's locking functionality does not support something like condition
* variables or Object.wait(), we break out here, releasing the lock before delaying.
*/
sleep(serviceMonitorConvergenceLatencySeconds, TimeUnit.SECONDS);
ApplicationInstance appInstance = getApplicationInstance(hostName);
try (MutableStatusRegistry statusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(appInstance.reference())) {
final HostStatus currentHostState = statusRegistry.getHostStatus(hostName);
if (HostStatus.NO_REMARKS == currentHostState) {
return;
}
ApplicationInstanceStatus appStatus = statusService.forApplicationInstance(appInstance.reference()).getApplicationInstanceStatus();
if (appStatus == ApplicationInstanceStatus.NO_REMARKS) {
policy.releaseSuspensionGrant(appInstance, hostName, statusRegistry);
}
}
}
use of com.yahoo.vespa.orchestrator.status.MutableStatusRegistry in project vespa by vespa-engine.
the class OrchestratorImpl method suspendGroup.
// Public for testing purposes
@Override
public void suspendGroup(NodeGroup nodeGroup) throws HostStateChangeDeniedException, HostNameNotFoundException {
ApplicationInstanceReference applicationReference = nodeGroup.getApplicationReference();
try (MutableStatusRegistry hostStatusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(applicationReference)) {
ApplicationInstanceStatus appStatus = statusService.forApplicationInstance(applicationReference).getApplicationInstanceStatus();
if (appStatus == ApplicationInstanceStatus.ALLOWED_TO_BE_DOWN) {
return;
}
ApplicationApi applicationApi = new ApplicationApiImpl(nodeGroup, hostStatusRegistry, clusterControllerClientFactory);
policy.grantSuspensionRequest(applicationApi);
}
}
use of com.yahoo.vespa.orchestrator.status.MutableStatusRegistry in project vespa by vespa-engine.
the class OrchestratorImpl method setApplicationStatus.
private void setApplicationStatus(ApplicationId appId, ApplicationInstanceStatus status) throws ApplicationStateChangeDeniedException, ApplicationIdNotFoundException {
ApplicationInstanceReference appRef = OrchestratorUtil.toApplicationInstanceReference(appId, instanceLookupService);
try (MutableStatusRegistry statusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(appRef)) {
// Short-circuit if already in wanted state
if (status == statusRegistry.getApplicationInstanceStatus())
return;
// Set content clusters for this application in maintenance on suspend
if (status == ApplicationInstanceStatus.ALLOWED_TO_BE_DOWN) {
ApplicationInstance application = getApplicationInstance(appRef);
// Mark it allowed to be down before we manipulate the clustercontroller
OrchestratorUtil.getHostsUsedByApplicationInstance(application).forEach(h -> statusRegistry.setHostState(h, HostStatus.ALLOWED_TO_BE_DOWN));
// If the clustercontroller throws an error the nodes will be marked as allowed to be down
// and be set back up on next resume invocation.
setClusterStateInController(application, ClusterControllerNodeState.MAINTENANCE);
}
statusRegistry.setApplicationInstanceStatus(status);
}
}
use of com.yahoo.vespa.orchestrator.status.MutableStatusRegistry in project vespa by vespa-engine.
the class OrchestratorImpl method acquirePermissionToRemove.
@Override
public void acquirePermissionToRemove(HostName hostName) throws OrchestrationException {
ApplicationInstance appInstance = getApplicationInstance(hostName);
NodeGroup nodeGroup = new NodeGroup(appInstance, hostName);
try (MutableStatusRegistry statusRegistry = statusService.lockApplicationInstance_forCurrentThreadOnly(appInstance.reference())) {
ApplicationApi applicationApi = new ApplicationApiImpl(nodeGroup, statusRegistry, clusterControllerClientFactory);
policy.acquirePermissionToRemove(applicationApi);
}
}
Aggregations