Search in sources :

Example 1 with PluginSettings

use of com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings in project gocd-ecs-elastic-agent by gocd.

the class ContainerInstanceHelper method terminateMostIdleStoppedInstance.

private void terminateMostIdleStoppedInstance(PluginSettings pluginSettings, List<Instance> stoppedInstances) {
    stoppedInstances.sort(new MostIdleInstanceComparator(Clock.DEFAULT.now()));
    final String instanceId = stoppedInstances.get(0).getInstanceId();
    LOG.info(format("Terminating stopped instance as max cluster limit is reached {0}.", instanceId));
    final Optional<ContainerInstance> containerInstance = getContainerInstances(pluginSettings).stream().filter(ci -> ci.getEc2InstanceId().equals(instanceId)).findFirst();
    containerInstance.ifPresent(self -> new TerminateOperation().execute(pluginSettings, self));
}
Also used : ListUtils.union(org.apache.commons.collections4.ListUtils.union) com.amazonaws.services.ecs.model(com.amazonaws.services.ecs.model) LimitExceededException(com.thoughtworks.gocd.elasticagent.ecs.exceptions.LimitExceededException) java.util(java.util) ConsoleLogAppender(com.thoughtworks.gocd.elasticagent.ecs.domain.ConsoleLogAppender) ECSElasticPlugin(com.thoughtworks.gocd.elasticagent.ecs.ECSElasticPlugin) com.amazonaws.services.ec2.model(com.amazonaws.services.ec2.model) ElasticAgentProfileProperties(com.thoughtworks.gocd.elasticagent.ecs.domain.ElasticAgentProfileProperties) PeriodFormatter(org.joda.time.format.PeriodFormatter) Poller(com.thoughtworks.gocd.elasticagent.ecs.aws.wait.Poller) Function(java.util.function.Function) Supplier(java.util.function.Supplier) Constants(com.thoughtworks.gocd.elasticagent.ecs.Constants) LABEL_SERVER_ID(com.thoughtworks.gocd.elasticagent.ecs.Constants.LABEL_SERVER_ID) EC2InstanceState(com.thoughtworks.gocd.elasticagent.ecs.domain.EC2InstanceState) MessageFormat.format(java.text.MessageFormat.format) Result(com.thoughtworks.gocd.elasticagent.ecs.aws.wait.Result) MostIdleInstanceComparator(com.thoughtworks.gocd.elasticagent.ecs.aws.comparator.MostIdleInstanceComparator) LOG(com.thoughtworks.gocd.elasticagent.ecs.ECSElasticPlugin.LOG) Platform(com.thoughtworks.gocd.elasticagent.ecs.domain.Platform) InstanceMatcher(com.thoughtworks.gocd.elasticagent.ecs.aws.matcher.InstanceMatcher) SPOT_INSTANCE_NAME_FORMAT(com.thoughtworks.gocd.elasticagent.ecs.aws.SpotInstanceHelper.SPOT_INSTANCE_NAME_FORMAT) Clock(com.thoughtworks.gocd.elasticagent.ecs.Clock) Predicate(java.util.function.Predicate) Collections.emptyList(java.util.Collections.emptyList) PluginSettings(com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings) ContainerInstanceFailedToRegisterException(com.thoughtworks.gocd.elasticagent.ecs.exceptions.ContainerInstanceFailedToRegisterException) Collectors(java.util.stream.Collectors) LAST_SEEN_IDLE(com.thoughtworks.gocd.elasticagent.ecs.Constants.LAST_SEEN_IDLE) ECSElasticPlugin.getServerId(com.thoughtworks.gocd.elasticagent.ecs.ECSElasticPlugin.getServerId) TerminateOperation(com.thoughtworks.gocd.elasticagent.ecs.aws.strategy.TerminateOperation) Stream(java.util.stream.Stream) String.valueOf(java.lang.String.valueOf) StringUtils.isNotBlank(org.apache.commons.lang3.StringUtils.isNotBlank) StringUtils.isBlank(org.apache.commons.lang3.StringUtils.isBlank) PeriodFormatterBuilder(org.joda.time.format.PeriodFormatterBuilder) Tag(com.amazonaws.services.ec2.model.Tag) MostIdleInstanceComparator(com.thoughtworks.gocd.elasticagent.ecs.aws.comparator.MostIdleInstanceComparator) TerminateOperation(com.thoughtworks.gocd.elasticagent.ecs.aws.strategy.TerminateOperation)

Example 2 with PluginSettings

use of com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings in project gocd-ecs-elastic-agent by gocd.

the class JobCompletionRequestExecutor method execute.

@Override
public GoPluginApiResponse execute() throws Exception {
    PluginSettings clusterProfileProperties = jobCompletionRequest.clusterProfileProperties();
    String elasticAgentId = jobCompletionRequest.getElasticAgentId();
    Agents agents = pluginRequest.listAgents();
    if (!agents.agentIds().contains(elasticAgentId)) {
        LOG.debug("[Job Completion] Skipping request to delete agent with id '{}' as the agent does not exist on the server.", elasticAgentId);
        return DefaultGoPluginApiResponse.success("");
    }
    Agent agent = new Agent(elasticAgentId);
    LOG.debug("[Job Completion] Disabling elastic agent with id {} on job completion {}.", agent.elasticAgentId(), jobCompletionRequest.jobIdentifier());
    pluginRequest.disableAgents(Collections.singletonList(agent));
    LOG.debug("[Job Completion] Terminating elastic agent with id {} on job completion {}.", agent.elasticAgentId(), jobCompletionRequest.jobIdentifier());
    agentInstances.terminate(agent.elasticAgentId(), clusterProfileProperties);
    LOG.debug("[Job Completion] Deleting elastic agent with id {} on job completion {}.", agent.elasticAgentId(), jobCompletionRequest.jobIdentifier());
    pluginRequest.deleteAgents(Collections.singletonList(agent));
    return DefaultGoPluginApiResponse.success("");
}
Also used : Agent(com.thoughtworks.gocd.elasticagent.ecs.domain.Agent) PluginSettings(com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings) Agents(com.thoughtworks.gocd.elasticagent.ecs.Agents)

Example 3 with PluginSettings

use of com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings in project gocd-ecs-elastic-agent by gocd.

the class MigrateConfigurationRequestExecutor method execute.

@Override
public GoPluginApiResponse execute() throws Exception {
    LOG.info("[Migrate Config] Request for Config Migration Started...");
    PluginSettings pluginSettings = migrateConfigurationRequest.getPluginSettings();
    List<ClusterProfile> existingClusterProfiles = migrateConfigurationRequest.getClusterProfiles();
    List<ElasticAgentProfile> existingElasticAgentProfiles = migrateConfigurationRequest.getElasticAgentProfiles();
    if (!arePluginSettingsConfigured(pluginSettings)) {
        LOG.info("[Migrate Config] No Plugin Settings are configured. Skipping Config Migration...");
        return new DefaultGoPluginApiResponse(200, migrateConfigurationRequest.toJSON());
    }
    if (existingClusterProfiles.size() == 0) {
        LOG.info("[Migrate Config] Did not find any Cluster Profile. Possibly, user just have configured plugin settings and haven't define any elastic agent profiles.");
        String newClusterId = UUID.randomUUID().toString();
        LOG.info("[Migrate Config] Migrating existing plugin settings to new cluster profile '{}'", newClusterId);
        ClusterProfile clusterProfile = new ClusterProfile(newClusterId, Constants.PLUGIN_ID, pluginSettings);
        return getGoPluginApiResponse(pluginSettings, Collections.singletonList(clusterProfile), existingElasticAgentProfiles);
    }
    LOG.info("[Migrate Config] Checking to perform migrations on Cluster Profiles '{}'.", existingClusterProfiles.stream().map(ClusterProfile::getId).collect(Collectors.toList()));
    for (ClusterProfile clusterProfile : existingClusterProfiles) {
        List<ElasticAgentProfile> associatedElasticAgentProfiles = findAssociatedElasticAgentProfiles(clusterProfile, existingElasticAgentProfiles);
        if (associatedElasticAgentProfiles.size() == 0) {
            LOG.info("[Migrate Config] Skipping migration for the cluster '{}' as no Elastic Agent Profiles are associated with it.", clusterProfile.getId());
            continue;
        }
        if (!arePluginSettingsConfigured(clusterProfile.getClusterProfileProperties())) {
            List<String> associatedProfileIds = associatedElasticAgentProfiles.stream().map(ElasticAgentProfile::getId).collect(Collectors.toList());
            LOG.info("[Migrate Config] Found an empty cluster profile '{}' associated with '{}' elastic agent profiles.", clusterProfile.getId(), associatedProfileIds);
            migrateConfigForCluster(pluginSettings, associatedElasticAgentProfiles, clusterProfile);
        } else {
            LOG.info("[Migrate Config] Skipping migration for the cluster '{}' as cluster has already been configured.", clusterProfile.getId());
        }
    }
    return new DefaultGoPluginApiResponse(200, migrateConfigurationRequest.toJSON());
}
Also used : ElasticAgentProfile(com.thoughtworks.gocd.elasticagent.ecs.domain.ElasticAgentProfile) DefaultGoPluginApiResponse(com.thoughtworks.go.plugin.api.response.DefaultGoPluginApiResponse) PluginSettings(com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings) ClusterProfile(com.thoughtworks.gocd.elasticagent.ecs.domain.ClusterProfile)

Example 4 with PluginSettings

use of com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings in project gocd-ecs-elastic-agent by gocd.

the class InstanceSelectionStrategy method instanceForScheduling.

public Optional<ContainerInstance> instanceForScheduling(PluginSettings pluginSettings, ElasticAgentProfileProperties elasticAgentProfileProperties, ContainerDefinition containerDefinition) {
    List<ContainerInstance> containerInstanceList = containerInstanceHelper.getContainerInstances(pluginSettings);
    final EC2Config ec2Config = new EC2Config.Builder().withProfile(elasticAgentProfileProperties).withSettings(pluginSettings).build();
    if (containerInstanceList.isEmpty()) {
        return Optional.empty();
    }
    final List<Instance> ec2Instances = containerInstanceHelper.ec2InstancesFromContainerInstances(pluginSettings, containerInstanceList).stream().filter(instance -> ACCEPTABLE_STATES.contains(instance.getState().getName().toLowerCase())).collect(toList());
    final Map<String, ContainerInstance> instanceMap = toMap(containerInstanceList, ContainerInstance::getEc2InstanceId, containerInstance -> containerInstance);
    sortInstancesForScheduling(ec2Instances);
    for (Instance instance : ec2Instances) {
        final ContainerInstance containerInstance = instanceMap.get(instance.getInstanceId());
        if (instanceMatcher.matches(ec2Config, instance) && containerInstanceMatcher.matches(containerInstance, containerDefinition)) {
            if (isSpotInstance(instance)) {
                containerInstanceHelper.removeLastSeenIdleTag(pluginSettings, asList(instance.getInstanceId()));
            }
            return Optional.of(containerInstance);
        } else {
            LOG.info(format("Skipped container creation on container instance {0}: required resources are not available.", instance.getInstanceId()));
        }
    }
    return Optional.empty();
}
Also used : ContainerInstance(com.amazonaws.services.ecs.model.ContainerInstance) ContainerInstanceHelper(com.thoughtworks.gocd.elasticagent.ecs.aws.ContainerInstanceHelper) PENDING(com.thoughtworks.gocd.elasticagent.ecs.domain.EC2InstanceState.PENDING) WINDOWS(com.thoughtworks.gocd.elasticagent.ecs.domain.Platform.WINDOWS) ElasticAgentProfileProperties(com.thoughtworks.gocd.elasticagent.ecs.domain.ElasticAgentProfileProperties) StringUtils.isNotEmpty(org.apache.commons.lang3.StringUtils.isNotEmpty) ContainerInstance(com.amazonaws.services.ecs.model.ContainerInstance) MessageFormat.format(java.text.MessageFormat.format) Util(com.thoughtworks.gocd.elasticagent.ecs.utils.Util) Arrays.asList(java.util.Arrays.asList) Map(java.util.Map) RUNNING(com.thoughtworks.gocd.elasticagent.ecs.domain.EC2InstanceState.RUNNING) LOG(com.thoughtworks.gocd.elasticagent.ecs.ECSElasticPlugin.LOG) Platform(com.thoughtworks.gocd.elasticagent.ecs.domain.Platform) Instance(com.amazonaws.services.ec2.model.Instance) ContainerInstanceMatcher(com.thoughtworks.gocd.elasticagent.ecs.aws.matcher.ContainerInstanceMatcher) InstanceMatcher(com.thoughtworks.gocd.elasticagent.ecs.aws.matcher.InstanceMatcher) PluginSettings(com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings) Set(java.util.Set) Collectors(java.util.stream.Collectors) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) Stream(java.util.stream.Stream) EC2Config(com.thoughtworks.gocd.elasticagent.ecs.aws.EC2Config) Util.toMap(com.thoughtworks.gocd.elasticagent.ecs.utils.Util.toMap) ContainerDefinition(com.amazonaws.services.ecs.model.ContainerDefinition) Optional(java.util.Optional) ContainerInstance(com.amazonaws.services.ecs.model.ContainerInstance) Instance(com.amazonaws.services.ec2.model.Instance) EC2Config(com.thoughtworks.gocd.elasticagent.ecs.aws.EC2Config)

Example 5 with PluginSettings

use of com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings in project gocd-ecs-elastic-agent by gocd.

the class StopIdleInstanceSelectionStrategy method findInstancesToStop.

@Override
protected List<ContainerInstance> findInstancesToStop(PluginSettings pluginSettings, Platform platform, Map<String, ContainerInstance> instanceIdToContainerInstance, List<Instance> idleInstances) {
    final Period timeInstanceCanStayIdle = platform == Platform.LINUX ? pluginSettings.stopLinuxInstanceAfter() : pluginSettings.stopWindowsInstanceAfter();
    idleInstances.sort(new MostIdleInstanceComparator(clock.now()));
    return idleInstances.stream().filter(isIdlePeriodIsMoreThan(timeInstanceCanStayIdle)).map(instance -> instanceIdToContainerInstance.get(instance.getInstanceId())).collect(Collectors.toList());
}
Also used : ContainerInstanceHelper(com.thoughtworks.gocd.elasticagent.ecs.aws.ContainerInstanceHelper) Period(org.joda.time.Period) Clock(com.thoughtworks.gocd.elasticagent.ecs.Clock) Predicate(java.util.function.Predicate) DateTime(org.joda.time.DateTime) PluginSettings(com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings) Collectors(java.util.stream.Collectors) LAST_SEEN_IDLE(com.thoughtworks.gocd.elasticagent.ecs.Constants.LAST_SEEN_IDLE) ContainerInstance(com.amazonaws.services.ecs.model.ContainerInstance) List(java.util.List) MostIdleInstanceComparator(com.thoughtworks.gocd.elasticagent.ecs.aws.comparator.MostIdleInstanceComparator) Tag(com.amazonaws.services.ec2.model.Tag) Map(java.util.Map) Optional(java.util.Optional) Platform(com.thoughtworks.gocd.elasticagent.ecs.domain.Platform) Instance(com.amazonaws.services.ec2.model.Instance) MostIdleInstanceComparator(com.thoughtworks.gocd.elasticagent.ecs.aws.comparator.MostIdleInstanceComparator) Period(org.joda.time.Period)

Aggregations

PluginSettings (com.thoughtworks.gocd.elasticagent.ecs.domain.PluginSettings)5 Platform (com.thoughtworks.gocd.elasticagent.ecs.domain.Platform)3 Collectors (java.util.stream.Collectors)3 Instance (com.amazonaws.services.ec2.model.Instance)2 Tag (com.amazonaws.services.ec2.model.Tag)2 ContainerInstance (com.amazonaws.services.ecs.model.ContainerInstance)2 Clock (com.thoughtworks.gocd.elasticagent.ecs.Clock)2 LAST_SEEN_IDLE (com.thoughtworks.gocd.elasticagent.ecs.Constants.LAST_SEEN_IDLE)2 LOG (com.thoughtworks.gocd.elasticagent.ecs.ECSElasticPlugin.LOG)2 ContainerInstanceHelper (com.thoughtworks.gocd.elasticagent.ecs.aws.ContainerInstanceHelper)2 MostIdleInstanceComparator (com.thoughtworks.gocd.elasticagent.ecs.aws.comparator.MostIdleInstanceComparator)2 InstanceMatcher (com.thoughtworks.gocd.elasticagent.ecs.aws.matcher.InstanceMatcher)2 ElasticAgentProfileProperties (com.thoughtworks.gocd.elasticagent.ecs.domain.ElasticAgentProfileProperties)2 MessageFormat.format (java.text.MessageFormat.format)2 List (java.util.List)2 Map (java.util.Map)2 Optional (java.util.Optional)2 Predicate (java.util.function.Predicate)2 Stream (java.util.stream.Stream)2 com.amazonaws.services.ec2.model (com.amazonaws.services.ec2.model)1