Search in sources :

Example 16 with DeploymentGroup

use of com.spotify.helios.common.descriptors.DeploymentGroup in project helios by spotify.

the class ZooKeeperMasterModel method rollingUpdate.

@Override
public void rollingUpdate(final DeploymentGroup deploymentGroup, final JobId jobId, final RolloutOptions options) throws DeploymentGroupDoesNotExistException, JobDoesNotExistException {
    checkNotNull(deploymentGroup, "deploymentGroup");
    log.info("preparing to initiate rolling-update on deployment-group: name={}, jobId={}", deploymentGroup.getName(), jobId);
    final DeploymentGroup updated = deploymentGroup.toBuilder().setJobId(jobId).setRolloutOptions(options).setRollingUpdateReason(MANUAL).build();
    if (getJob(jobId) == null) {
        throw new JobDoesNotExistException(jobId);
    }
    final List<ZooKeeperOperation> operations = Lists.newArrayList();
    final ZooKeeperClient client = provider.get("rollingUpdate");
    operations.add(set(Paths.configDeploymentGroup(updated.getName()), updated));
    try {
        final RollingUpdateOp op = getInitRollingUpdateOps(updated, client);
        operations.addAll(op.operations());
        log.info("starting zookeeper transaction for rolling-update on " + "deployment-group name={} jobId={}. List of operations: {}", deploymentGroup.getName(), jobId, operations);
        client.transaction(operations);
        emitEvents(deploymentGroupEventTopic, op.events());
        log.info("initiated rolling-update on deployment-group: name={}, jobId={}", deploymentGroup.getName(), jobId);
    } catch (final NoNodeException e) {
        throw new DeploymentGroupDoesNotExistException(deploymentGroup.getName());
    } catch (final KeeperException e) {
        throw new HeliosRuntimeException("rolling-update on deployment-group " + deploymentGroup.getName() + " failed", e);
    }
}
Also used : RollingUpdateOp(com.spotify.helios.rollingupdate.RollingUpdateOp) NoNodeException(org.apache.zookeeper.KeeperException.NoNodeException) ZooKeeperOperation(com.spotify.helios.servicescommon.coordination.ZooKeeperOperation) ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) DeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup) KeeperException(org.apache.zookeeper.KeeperException)

Example 17 with DeploymentGroup

use of com.spotify.helios.common.descriptors.DeploymentGroup in project helios by spotify.

the class DeploymentGroupCreateCommandTest method testCreateDeploymentGroup.

@Test
public void testCreateDeploymentGroup() throws Exception {
    mockCreateResponse(CreateDeploymentGroupResponse.Status.CREATED);
    assertEquals(0, command.run(options, client, out, false, null));
    final ArgumentCaptor<DeploymentGroup> captor = ArgumentCaptor.forClass(DeploymentGroup.class);
    verify(client).createDeploymentGroup(captor.capture());
    assertEquals(GROUP_NAME, captor.getValue().getName());
    assertEquals(HOST_SELECTORS, captor.getValue().getHostSelectors());
    final String output = baos.toString();
    assertThat(output, containsString("\"name\":\"foo-group\""));
    assertThat(output, containsString("\"hostSelectors\":[" + "{\"label\":\"foo\",\"operand\":\"bar\",\"operator\":\"EQUALS\"}," + "{\"label\":\"baz\",\"operand\":\"qux\",\"operator\":\"EQUALS\"}" + "]"));
    assertThat(output, containsString("CREATED"));
}
Also used : CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) DeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup) Test(org.junit.Test)

Example 18 with DeploymentGroup

use of com.spotify.helios.common.descriptors.DeploymentGroup in project helios by spotify.

the class DeploymentGroupInspectCommandTest method setUp.

@Before
public void setUp() {
    // use a real, dummy Subparser impl to avoid having to mock out every single call
    final ArgumentParser parser = ArgumentParsers.newArgumentParser("test");
    final Subparser subparser = parser.addSubparsers().addParser("inspect");
    command = new DeploymentGroupInspectCommand(subparser);
    when(client.deploymentGroup(NAME)).thenReturn(Futures.immediateFuture(DEPLOYMENT_GROUP));
    final ListenableFuture<DeploymentGroup> nullFuture = Futures.immediateFuture(null);
    when(client.deploymentGroup(NON_EXISTENT_NAME)).thenReturn(nullFuture);
}
Also used : Subparser(net.sourceforge.argparse4j.inf.Subparser) ArgumentParser(net.sourceforge.argparse4j.inf.ArgumentParser) DeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup) Before(org.junit.Before)

Example 19 with DeploymentGroup

use of com.spotify.helios.common.descriptors.DeploymentGroup in project helios by spotify.

the class ZooKeeperMasterModel method getInitRollingUpdateOps.

private RollingUpdateOp getInitRollingUpdateOps(final DeploymentGroup deploymentGroup, final List<String> updateHosts, final List<String> undeployHosts, final ZooKeeperClient zooKeeperClient) throws KeeperException {
    final List<RolloutTask> rolloutTasks = new ArrayList<>();
    // give precedence to the updateHosts list so we don't end up in a state where we updated a host
    // and then removed the job from it (because of buggy logic in the calling method)
    final List<String> updateHostsCopy = new ArrayList<>(updateHosts);
    final List<String> undeployHostsCopy = new ArrayList<>(undeployHosts);
    undeployHostsCopy.removeAll(updateHostsCopy);
    // we only care about hosts that are UP
    final List<String> upHostsToUndeploy = undeployHostsCopy.stream().filter(host -> checkHostUp(zooKeeperClient, host)).collect(Collectors.toList());
    final List<String> upHostsToDeploy = updateHostsCopy.stream().filter(host -> checkHostUp(zooKeeperClient, host)).collect(Collectors.toList());
    rolloutTasks.addAll(RollingUndeployPlanner.of(deploymentGroup).plan(upHostsToUndeploy));
    rolloutTasks.addAll(RollingUpdatePlanner.of(deploymentGroup).plan(upHostsToDeploy));
    log.info("generated rolloutTasks for deployment-group name={} " + "updateHosts={} undeployHosts={}: {}", deploymentGroup.getName(), updateHosts, undeployHosts, rolloutTasks);
    final DeploymentGroupTasks tasks = DeploymentGroupTasks.newBuilder().setRolloutTasks(rolloutTasks).setTaskIndex(0).setDeploymentGroup(deploymentGroup).build();
    return new RollingUpdateOpFactory(tasks, DEPLOYMENT_GROUP_EVENT_FACTORY).start(deploymentGroup, zooKeeperClient);
}
Also used : Descriptor.parse(com.spotify.helios.common.descriptors.Descriptor.parse) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) LoggerFactory(org.slf4j.LoggerFactory) Stat(org.apache.zookeeper.data.Stat) ThrottleState(com.spotify.helios.common.descriptors.ThrottleState) FAILED(com.spotify.helios.common.descriptors.DeploymentGroupStatus.State.FAILED) ZooKeeperOperations.create(com.spotify.helios.servicescommon.coordination.ZooKeeperOperations.create) Collections.singletonList(java.util.Collections.singletonList) Optional.fromNullable(com.google.common.base.Optional.fromNullable) Json(com.spotify.helios.common.Json) RolloutOptions(com.spotify.helios.common.descriptors.RolloutOptions) RollingUndeployPlanner(com.spotify.helios.rollingupdate.RollingUndeployPlanner) ZooKeeperOperations.set(com.spotify.helios.servicescommon.coordination.ZooKeeperOperations.set) VersionedValue(com.spotify.helios.servicescommon.VersionedValue) Map(java.util.Map) Deployment(com.spotify.helios.common.descriptors.Deployment) TypeReference(com.fasterxml.jackson.core.type.TypeReference) JsonParseException(com.fasterxml.jackson.core.JsonParseException) HostInfo(com.spotify.helios.common.descriptors.HostInfo) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Collections.emptyList(java.util.Collections.emptyList) DeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup) EventSender(com.spotify.helios.servicescommon.EventSender) Set(java.util.Set) PortMapping(com.spotify.helios.common.descriptors.PortMapping) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) RollingUpdateOp(com.spotify.helios.rollingupdate.RollingUpdateOp) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) Objects(java.util.Objects) Nullable(org.jetbrains.annotations.Nullable) HOSTS_CHANGED(com.spotify.helios.common.descriptors.DeploymentGroup.RollingUpdateReason.HOSTS_CHANGED) TaskStatusEvent(com.spotify.helios.common.descriptors.TaskStatusEvent) List(java.util.List) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) NotEmptyException(org.apache.zookeeper.KeeperException.NotEmptyException) MANUAL(com.spotify.helios.common.descriptors.DeploymentGroup.RollingUpdateReason.MANUAL) UP(com.spotify.helios.common.descriptors.HostStatus.Status.UP) JsonMappingException(com.fasterxml.jackson.databind.JsonMappingException) RollingUpdateOpFactory(com.spotify.helios.rollingupdate.RollingUpdateOpFactory) Lists.reverse(com.google.common.collect.Lists.reverse) RollingUpdatePlanner(com.spotify.helios.rollingupdate.RollingUpdatePlanner) ZooKeeperOperations.delete(com.spotify.helios.servicescommon.coordination.ZooKeeperOperations.delete) Joiner(com.google.common.base.Joiner) JobId(com.spotify.helios.common.descriptors.JobId) RollingUpdateError(com.spotify.helios.rollingupdate.RollingUpdateError) Goal(com.spotify.helios.common.descriptors.Goal) Strings.isNullOrEmpty(com.google.common.base.Strings.isNullOrEmpty) Paths(com.spotify.helios.servicescommon.coordination.Paths) ZooKeeperRegistrarUtil(com.spotify.helios.servicescommon.ZooKeeperRegistrarUtil) ArrayList(java.util.ArrayList) ROLLING_OUT(com.spotify.helios.common.descriptors.DeploymentGroupStatus.State.ROLLING_OUT) Strings(com.google.common.base.Strings) Lists(com.google.common.collect.Lists) ImmutableList(com.google.common.collect.ImmutableList) DeploymentGroupTasks(com.spotify.helios.common.descriptors.DeploymentGroupTasks) HostStatus(com.spotify.helios.common.descriptors.HostStatus) DOWN(com.spotify.helios.common.descriptors.HostStatus.Status.DOWN) OpResult(org.apache.zookeeper.OpResult) Task(com.spotify.helios.common.descriptors.Task) TaskStatus(com.spotify.helios.common.descriptors.TaskStatus) BadVersionException(org.apache.zookeeper.KeeperException.BadVersionException) Collections.emptyMap(java.util.Collections.emptyMap) Logger(org.slf4j.Logger) Job(com.spotify.helios.common.descriptors.Job) KeeperException(org.apache.zookeeper.KeeperException) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) MoreObjects(com.google.common.base.MoreObjects) Throwables(com.google.common.base.Throwables) IOException(java.io.IOException) ZooKeeperOperation(com.spotify.helios.servicescommon.coordination.ZooKeeperOperation) Maps(com.google.common.collect.Maps) DeploymentGroupEventFactory(com.spotify.helios.rollingupdate.DeploymentGroupEventFactory) RolloutTask(com.spotify.helios.common.descriptors.RolloutTask) Node(com.spotify.helios.servicescommon.coordination.Node) Ordering(com.google.common.collect.Ordering) NoNodeException(org.apache.zookeeper.KeeperException.NoNodeException) JobStatus(com.spotify.helios.common.descriptors.JobStatus) ZooKeeperClientProvider(com.spotify.helios.servicescommon.coordination.ZooKeeperClientProvider) DeploymentGroupStatus(com.spotify.helios.common.descriptors.DeploymentGroupStatus) ZooKeeperOperations.check(com.spotify.helios.servicescommon.coordination.ZooKeeperOperations.check) Preconditions(com.google.common.base.Preconditions) AgentInfo(com.spotify.helios.common.descriptors.AgentInfo) Comparator(java.util.Comparator) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) Collections(java.util.Collections) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) ArrayList(java.util.ArrayList) DeploymentGroupTasks(com.spotify.helios.common.descriptors.DeploymentGroupTasks) RolloutTask(com.spotify.helios.common.descriptors.RolloutTask) RollingUpdateOpFactory(com.spotify.helios.rollingupdate.RollingUpdateOpFactory)

Example 20 with DeploymentGroup

use of com.spotify.helios.common.descriptors.DeploymentGroup in project helios by spotify.

the class ZooKeeperMasterModel method getDeploymentGroupHosts.

@Override
public List<String> getDeploymentGroupHosts(final String name) throws DeploymentGroupDoesNotExistException {
    log.debug("getting deployment group hosts: {}", name);
    final ZooKeeperClient client = provider.get("getDeploymentGroupHosts");
    final DeploymentGroup deploymentGroup = getDeploymentGroup(client, name);
    if (deploymentGroup == null) {
        throw new DeploymentGroupDoesNotExistException(name);
    }
    return getHosts(client, Paths.statusDeploymentGroupHosts(name));
}
Also used : ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) DeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup)

Aggregations

DeploymentGroup (com.spotify.helios.common.descriptors.DeploymentGroup)30 Test (org.junit.Test)18 RolloutTask (com.spotify.helios.common.descriptors.RolloutTask)10 ZooKeeperClient (com.spotify.helios.servicescommon.coordination.ZooKeeperClient)10 HeliosRuntimeException (com.spotify.helios.common.HeliosRuntimeException)6 ZooKeeperOperation (com.spotify.helios.servicescommon.coordination.ZooKeeperOperation)6 Job (com.spotify.helios.common.descriptors.Job)4 CreateDeploymentGroupResponse (com.spotify.helios.common.protocol.CreateDeploymentGroupResponse)4 RollingUpdateResponse (com.spotify.helios.common.protocol.RollingUpdateResponse)4 KeeperException (org.apache.zookeeper.KeeperException)4 NoNodeException (org.apache.zookeeper.KeeperException.NoNodeException)4 DeploymentGroupStatus (com.spotify.helios.common.descriptors.DeploymentGroupStatus)3 HostStatus (com.spotify.helios.common.descriptors.HostStatus)3 JobId (com.spotify.helios.common.descriptors.JobId)3 RemoveDeploymentGroupResponse (com.spotify.helios.common.protocol.RemoveDeploymentGroupResponse)3 DefaultZooKeeperClient (com.spotify.helios.servicescommon.coordination.DefaultZooKeeperClient)3 IOException (java.io.IOException)3 Response (javax.ws.rs.core.Response)3 ExceptionMetered (com.codahale.metrics.annotation.ExceptionMetered)2 Timed (com.codahale.metrics.annotation.Timed)2