Search in sources :

Example 1 with InstanceGroup

use of com.netflix.titus.api.connector.cloud.InstanceGroup in project titus-control-plane by Netflix.

the class AwsInstanceCloudConnector method scaleUp.

@Override
public Completable scaleUp(String instanceGroupId, int scaleUpCount) {
    return getInstanceGroups(singletonList(instanceGroupId)).map(list -> list.get(0)).flatMap(instanceGroup -> {
        int newDesired = instanceGroup.getDesired() + scaleUpCount;
        if (newDesired > instanceGroup.getMax()) {
            return Observable.error(CloudConnectorException.invalidArgument("Instance group requested desired size %s > max size %s", newDesired, instanceGroup.getMax()));
        }
        logger.info("Scaling up instance group {}, by {} instances (desired changed from {} to {})", instanceGroup, scaleUpCount, instanceGroup.getDesired(), newDesired);
        UpdateAutoScalingGroupRequest request = new UpdateAutoScalingGroupRequest().withAutoScalingGroupName(instanceGroupId).withDesiredCapacity(newDesired);
        Observable<UpdateAutoScalingGroupResult> observable = toObservable(request, autoScalingClient::updateAutoScalingGroupAsync);
        return observable;
    }).toCompletable().timeout(configuration.getAwsRequestTimeoutMs(), TimeUnit.MILLISECONDS);
}
Also used : InstanceLaunchConfiguration(com.netflix.titus.api.connector.cloud.InstanceLaunchConfiguration) BiFunction(java.util.function.BiFunction) CollectionsExt(com.netflix.titus.common.util.CollectionsExt) TerminateInstancesResult(com.amazonaws.services.ec2.model.TerminateInstancesResult) LoggerFactory(org.slf4j.LoggerFactory) DescribeLaunchConfigurationsResult(com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsResult) Collections.singletonList(java.util.Collections.singletonList) PreDestroy(javax.annotation.PreDestroy) Future(java.util.concurrent.Future) CreateTagsRequest(com.amazonaws.services.ec2.model.CreateTagsRequest) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) CloudConnectorException(com.netflix.titus.api.connector.cloud.CloudConnectorException) Map(java.util.Map) Schedulers(rx.schedulers.Schedulers) Either(com.netflix.titus.common.util.tuple.Either) ResourceDimension(com.netflix.titus.api.model.ResourceDimension) Reservation(com.amazonaws.services.ec2.model.Reservation) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) Scheduler(rx.Scheduler) Collectors(java.util.stream.Collectors) List(java.util.List) AutoScalingInstanceDetails(com.amazonaws.services.autoscaling.model.AutoScalingInstanceDetails) UpdateAutoScalingGroupResult(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupResult) Tag(com.amazonaws.services.ec2.model.Tag) ProxyConfiguration(com.netflix.titus.common.util.guice.annotation.ProxyConfiguration) Optional(java.util.Optional) ObservableExt(com.netflix.titus.common.util.rx.ObservableExt) Completable(rx.Completable) AwsInstanceType(com.netflix.titus.common.aws.AwsInstanceType) ProxyType(com.netflix.titus.common.util.guice.ProxyType) Singleton(javax.inject.Singleton) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Observable(rx.Observable) Inject(javax.inject.Inject) Strings(com.google.common.base.Strings) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest) Pair(com.netflix.titus.common.util.tuple.Pair) InstanceCloudConnector(com.netflix.titus.api.connector.cloud.InstanceCloudConnector) AwsInstanceDescriptor(com.netflix.titus.common.aws.AwsInstanceDescriptor) DescribeAutoScalingInstancesResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesResult) DescribeAutoScalingGroupsResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult) Filter(com.amazonaws.services.ec2.model.Filter) Named(javax.inject.Named) TagDescription(com.amazonaws.services.autoscaling.model.TagDescription) Instance(com.netflix.titus.api.connector.cloud.Instance) AmazonWebServiceRequest(com.amazonaws.AmazonWebServiceRequest) Logger(org.slf4j.Logger) DescribeLaunchConfigurationsRequest(com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsRequest) DetachInstancesRequest(com.amazonaws.services.autoscaling.model.DetachInstancesRequest) InstanceState(com.netflix.titus.api.connector.cloud.Instance.InstanceState) SuspendedProcess(com.amazonaws.services.autoscaling.model.SuspendedProcess) CreateTagsResult(com.amazonaws.services.ec2.model.CreateTagsResult) AmazonEC2Async(com.amazonaws.services.ec2.AmazonEC2Async) Mono(reactor.core.publisher.Mono) TerminateInstancesRequest(com.amazonaws.services.ec2.model.TerminateInstancesRequest) Emitter(rx.Emitter) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) Activator(com.netflix.titus.common.util.guice.annotation.Activator) TimeUnit(java.util.concurrent.TimeUnit) DescribeAutoScalingInstancesRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesRequest) AsyncHandler(com.amazonaws.handlers.AsyncHandler) AmazonEC2Exception(com.amazonaws.services.ec2.model.AmazonEC2Exception) InstanceGroup(com.netflix.titus.api.connector.cloud.InstanceGroup) AmazonAutoScalingAsync(com.amazonaws.services.autoscaling.AmazonAutoScalingAsync) Collections(java.util.Collections) DetachInstancesResult(com.amazonaws.services.autoscaling.model.DetachInstancesResult) LaunchConfiguration(com.amazonaws.services.autoscaling.model.LaunchConfiguration) UpdateAutoScalingGroupResult(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupResult) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest)

Example 2 with InstanceGroup

use of com.netflix.titus.api.connector.cloud.InstanceGroup in project titus-control-plane by Netflix.

the class Main method scaleUp.

private void scaleUp(List<String> params) {
    String instanceGroupId = params.get(0);
    int scaleUpCount = Integer.parseInt(params.get(1));
    InstanceGroup current = connector.getInstanceGroups(Collections.singletonList(instanceGroupId)).toBlocking().first().get(0);
    System.out.println("Desired before scale-up: " + current.getDesired());
    Throwable error = connector.scaleUp(instanceGroupId, scaleUpCount).get();
    if (error != null) {
        System.err.println("Scale-up error: " + error.getMessage());
        error.printStackTrace();
    }
    InstanceGroup after = connector.getInstanceGroups(Collections.singletonList(instanceGroupId)).toBlocking().first().get(0);
    System.out.println("Desired after scale-up: " + after.getDesired());
}
Also used : InstanceGroup(com.netflix.titus.api.connector.cloud.InstanceGroup)

Example 3 with InstanceGroup

use of com.netflix.titus.api.connector.cloud.InstanceGroup in project titus-control-plane by Netflix.

the class Main method scaleDown.

private void scaleDown(List<String> params) {
    String instanceGroupId = params.get(0);
    int scaleDownCount = Integer.parseInt(params.get(1));
    InstanceGroup current = connector.getInstanceGroups(Collections.singletonList(instanceGroupId)).toBlocking().first().get(0);
    System.out.println("Desired before scale-down: " + current.getDesired());
    Throwable error = connector.updateCapacity(instanceGroupId, Optional.empty(), Optional.of(current.getDesired() - scaleDownCount)).get();
    if (error != null) {
        System.err.println("Scale-down error: " + error.getMessage());
        error.printStackTrace();
    }
    InstanceGroup after = connector.getInstanceGroups(Collections.singletonList(instanceGroupId)).toBlocking().first().get(0);
    System.out.println("Desired after scale-down: " + after.getDesired());
}
Also used : InstanceGroup(com.netflix.titus.api.connector.cloud.InstanceGroup)

Example 4 with InstanceGroup

use of com.netflix.titus.api.connector.cloud.InstanceGroup in project titus-control-plane by Netflix.

the class AwsInstanceCloudConnector method getInstanceGroups.

@Override
public Observable<List<InstanceGroup>> getInstanceGroups(List<String> instanceGroupIds) {
    if (instanceGroupIds.isEmpty()) {
        return Observable.just(Collections.emptyList());
    }
    DescribeAutoScalingGroupsRequest request = new DescribeAutoScalingGroupsRequest().withAutoScalingGroupNames(instanceGroupIds);
    Observable<DescribeAutoScalingGroupsResult> observable = toObservable(request, autoScalingClient::describeAutoScalingGroupsAsync);
    return observable.map(response -> toInstanceGroups(response.getAutoScalingGroups())).timeout(configuration.getAwsRequestTimeoutMs(), TimeUnit.MILLISECONDS);
}
Also used : InstanceLaunchConfiguration(com.netflix.titus.api.connector.cloud.InstanceLaunchConfiguration) BiFunction(java.util.function.BiFunction) CollectionsExt(com.netflix.titus.common.util.CollectionsExt) TerminateInstancesResult(com.amazonaws.services.ec2.model.TerminateInstancesResult) LoggerFactory(org.slf4j.LoggerFactory) DescribeLaunchConfigurationsResult(com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsResult) Collections.singletonList(java.util.Collections.singletonList) PreDestroy(javax.annotation.PreDestroy) Future(java.util.concurrent.Future) CreateTagsRequest(com.amazonaws.services.ec2.model.CreateTagsRequest) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) CloudConnectorException(com.netflix.titus.api.connector.cloud.CloudConnectorException) Map(java.util.Map) Schedulers(rx.schedulers.Schedulers) Either(com.netflix.titus.common.util.tuple.Either) ResourceDimension(com.netflix.titus.api.model.ResourceDimension) Reservation(com.amazonaws.services.ec2.model.Reservation) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) Scheduler(rx.Scheduler) Collectors(java.util.stream.Collectors) List(java.util.List) AutoScalingInstanceDetails(com.amazonaws.services.autoscaling.model.AutoScalingInstanceDetails) UpdateAutoScalingGroupResult(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupResult) Tag(com.amazonaws.services.ec2.model.Tag) ProxyConfiguration(com.netflix.titus.common.util.guice.annotation.ProxyConfiguration) Optional(java.util.Optional) ObservableExt(com.netflix.titus.common.util.rx.ObservableExt) Completable(rx.Completable) AwsInstanceType(com.netflix.titus.common.aws.AwsInstanceType) ProxyType(com.netflix.titus.common.util.guice.ProxyType) Singleton(javax.inject.Singleton) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Observable(rx.Observable) Inject(javax.inject.Inject) Strings(com.google.common.base.Strings) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest) Pair(com.netflix.titus.common.util.tuple.Pair) InstanceCloudConnector(com.netflix.titus.api.connector.cloud.InstanceCloudConnector) AwsInstanceDescriptor(com.netflix.titus.common.aws.AwsInstanceDescriptor) DescribeAutoScalingInstancesResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesResult) DescribeAutoScalingGroupsResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult) Filter(com.amazonaws.services.ec2.model.Filter) Named(javax.inject.Named) TagDescription(com.amazonaws.services.autoscaling.model.TagDescription) Instance(com.netflix.titus.api.connector.cloud.Instance) AmazonWebServiceRequest(com.amazonaws.AmazonWebServiceRequest) Logger(org.slf4j.Logger) DescribeLaunchConfigurationsRequest(com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsRequest) DetachInstancesRequest(com.amazonaws.services.autoscaling.model.DetachInstancesRequest) InstanceState(com.netflix.titus.api.connector.cloud.Instance.InstanceState) SuspendedProcess(com.amazonaws.services.autoscaling.model.SuspendedProcess) CreateTagsResult(com.amazonaws.services.ec2.model.CreateTagsResult) AmazonEC2Async(com.amazonaws.services.ec2.AmazonEC2Async) Mono(reactor.core.publisher.Mono) TerminateInstancesRequest(com.amazonaws.services.ec2.model.TerminateInstancesRequest) Emitter(rx.Emitter) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) Activator(com.netflix.titus.common.util.guice.annotation.Activator) TimeUnit(java.util.concurrent.TimeUnit) DescribeAutoScalingInstancesRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesRequest) AsyncHandler(com.amazonaws.handlers.AsyncHandler) AmazonEC2Exception(com.amazonaws.services.ec2.model.AmazonEC2Exception) InstanceGroup(com.netflix.titus.api.connector.cloud.InstanceGroup) AmazonAutoScalingAsync(com.amazonaws.services.autoscaling.AmazonAutoScalingAsync) Collections(java.util.Collections) DetachInstancesResult(com.amazonaws.services.autoscaling.model.DetachInstancesResult) LaunchConfiguration(com.amazonaws.services.autoscaling.model.LaunchConfiguration) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) DescribeAutoScalingGroupsResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult)

Example 5 with InstanceGroup

use of com.netflix.titus.api.connector.cloud.InstanceGroup in project titus-control-plane by Netflix.

the class AwsInstanceCloudConnector method toInstanceGroup.

private InstanceGroup toInstanceGroup(AutoScalingGroup awsScalingGroup) {
    boolean isLaunchSuspended = false;
    boolean isTerminateSuspended = false;
    for (SuspendedProcess suspendedProcess : awsScalingGroup.getSuspendedProcesses()) {
        if (suspendedProcess.getProcessName().equals("Launch")) {
            isLaunchSuspended = true;
        } else if (suspendedProcess.getProcessName().equals("Terminate")) {
            isTerminateSuspended = true;
        }
    }
    Map<String, String> attributes = CollectionsExt.isNullOrEmpty(awsScalingGroup.getTags()) ? Collections.emptyMap() : awsScalingGroup.getTags().stream().collect(Collectors.toMap(TagDescription::getKey, TagDescription::getValue));
    return new InstanceGroup(awsScalingGroup.getAutoScalingGroupName(), awsScalingGroup.getLaunchConfigurationName(), DEFAULT_INSTANCE_TYPE, awsScalingGroup.getMinSize(), awsScalingGroup.getDesiredCapacity(), awsScalingGroup.getMaxSize(), isLaunchSuspended, isTerminateSuspended, attributes, awsScalingGroup.getInstances().stream().map(com.amazonaws.services.autoscaling.model.Instance::getInstanceId).collect(Collectors.toList()));
}
Also used : Instance(com.netflix.titus.api.connector.cloud.Instance) TagDescription(com.amazonaws.services.autoscaling.model.TagDescription) SuspendedProcess(com.amazonaws.services.autoscaling.model.SuspendedProcess) InstanceGroup(com.netflix.titus.api.connector.cloud.InstanceGroup)

Aggregations

InstanceGroup (com.netflix.titus.api.connector.cloud.InstanceGroup)6 SuspendedProcess (com.amazonaws.services.autoscaling.model.SuspendedProcess)4 TagDescription (com.amazonaws.services.autoscaling.model.TagDescription)4 Instance (com.netflix.titus.api.connector.cloud.Instance)4 AmazonWebServiceRequest (com.amazonaws.AmazonWebServiceRequest)3 AsyncHandler (com.amazonaws.handlers.AsyncHandler)3 AmazonAutoScalingAsync (com.amazonaws.services.autoscaling.AmazonAutoScalingAsync)3 AutoScalingGroup (com.amazonaws.services.autoscaling.model.AutoScalingGroup)3 AutoScalingInstanceDetails (com.amazonaws.services.autoscaling.model.AutoScalingInstanceDetails)3 DescribeAutoScalingGroupsRequest (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest)3 DescribeAutoScalingGroupsResult (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult)3 DescribeAutoScalingInstancesRequest (com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesRequest)3 DescribeAutoScalingInstancesResult (com.amazonaws.services.autoscaling.model.DescribeAutoScalingInstancesResult)3 DescribeLaunchConfigurationsRequest (com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsRequest)3 DescribeLaunchConfigurationsResult (com.amazonaws.services.autoscaling.model.DescribeLaunchConfigurationsResult)3 DetachInstancesRequest (com.amazonaws.services.autoscaling.model.DetachInstancesRequest)3 DetachInstancesResult (com.amazonaws.services.autoscaling.model.DetachInstancesResult)3 LaunchConfiguration (com.amazonaws.services.autoscaling.model.LaunchConfiguration)3 UpdateAutoScalingGroupRequest (com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest)3 UpdateAutoScalingGroupResult (com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupResult)3