Search in sources :

Example 11 with AutoScalingData

use of org.apache.druid.indexing.overlord.autoscaling.AutoScalingData in project druid by druid-io.

the class EC2AutoScaler method terminateWithIds.

@Override
public AutoScalingData terminateWithIds(List<String> ids) {
    if (ids.isEmpty()) {
        return new AutoScalingData(new ArrayList<>());
    }
    try {
        log.info("Terminating instances[%s]", ids);
        amazonEC2Client.terminateInstances(new TerminateInstancesRequest(ids));
        return new AutoScalingData(ids);
    } catch (Exception e) {
        log.error(e, "Unable to terminate any instances.");
    }
    return null;
}
Also used : AutoScalingData(org.apache.druid.indexing.overlord.autoscaling.AutoScalingData) TerminateInstancesRequest(com.amazonaws.services.ec2.model.TerminateInstancesRequest)

Example 12 with AutoScalingData

use of org.apache.druid.indexing.overlord.autoscaling.AutoScalingData in project druid by druid-io.

the class EC2AutoScaler method provision.

@Override
public AutoScalingData provision() {
    try {
        final EC2NodeData workerConfig = envConfig.getNodeData();
        final String userDataBase64;
        if (envConfig.getUserData() == null) {
            userDataBase64 = null;
        } else {
            if (config.getWorkerVersion() == null) {
                userDataBase64 = envConfig.getUserData().getUserDataBase64();
            } else {
                userDataBase64 = envConfig.getUserData().withVersion(config.getWorkerVersion()).getUserDataBase64();
            }
        }
        RunInstancesRequest request = new RunInstancesRequest(workerConfig.getAmiId(), workerConfig.getMinInstances(), workerConfig.getMaxInstances()).withInstanceType(workerConfig.getInstanceType()).withPlacement(new Placement(envConfig.getAvailabilityZone())).withKeyName(workerConfig.getKeyName()).withIamInstanceProfile(workerConfig.getIamProfile() == null ? null : workerConfig.getIamProfile().toIamInstanceProfileSpecification()).withUserData(userDataBase64);
        // leaving it null uses the EC2 default.
        if (workerConfig.getAssociatePublicIpAddress() != null) {
            request.withNetworkInterfaces(new InstanceNetworkInterfaceSpecification().withAssociatePublicIpAddress(workerConfig.getAssociatePublicIpAddress()).withSubnetId(workerConfig.getSubnetId()).withGroups(workerConfig.getSecurityGroupIds()).withDeviceIndex(0));
        } else {
            request.withSecurityGroupIds(workerConfig.getSecurityGroupIds()).withSubnetId(workerConfig.getSubnetId());
        }
        final RunInstancesResult result = amazonEC2Client.runInstances(request);
        final List<String> instanceIds = Lists.transform(result.getReservation().getInstances(), new Function<Instance, String>() {

            @Override
            public String apply(Instance input) {
                return input.getInstanceId();
            }
        });
        log.info("Created instances: %s", instanceIds);
        return new AutoScalingData(Lists.transform(result.getReservation().getInstances(), new Function<Instance, String>() {

            @Override
            public String apply(Instance input) {
                return input.getInstanceId();
            }
        }));
    } catch (Exception e) {
        log.error(e, "Unable to provision any EC2 instances.");
    }
    return null;
}
Also used : AutoScalingData(org.apache.druid.indexing.overlord.autoscaling.AutoScalingData) Instance(com.amazonaws.services.ec2.model.Instance) InstanceNetworkInterfaceSpecification(com.amazonaws.services.ec2.model.InstanceNetworkInterfaceSpecification) Function(com.google.common.base.Function) Placement(com.amazonaws.services.ec2.model.Placement) RunInstancesResult(com.amazonaws.services.ec2.model.RunInstancesResult) RunInstancesRequest(com.amazonaws.services.ec2.model.RunInstancesRequest)

Aggregations

AutoScalingData (org.apache.druid.indexing.overlord.autoscaling.AutoScalingData)12 Operation (com.google.api.services.compute.model.Operation)7 Test (org.junit.Test)7 InstanceGroupManagersListManagedInstancesResponse (com.google.api.services.compute.model.InstanceGroupManagersListManagedInstancesResponse)6 InstanceGroupManagersDeleteInstancesRequest (com.google.api.services.compute.model.InstanceGroupManagersDeleteInstancesRequest)4 DescribeInstancesRequest (com.amazonaws.services.ec2.model.DescribeInstancesRequest)2 Instance (com.amazonaws.services.ec2.model.Instance)2 RunInstancesRequest (com.amazonaws.services.ec2.model.RunInstancesRequest)2 RunInstancesResult (com.amazonaws.services.ec2.model.RunInstancesResult)2 TerminateInstancesRequest (com.amazonaws.services.ec2.model.TerminateInstancesRequest)2 Compute (com.google.api.services.compute.Compute)2 Function (com.google.common.base.Function)2 IOException (java.io.IOException)2 GeneralSecurityException (java.security.GeneralSecurityException)2 DescribeInstancesResult (com.amazonaws.services.ec2.model.DescribeInstancesResult)1 Filter (com.amazonaws.services.ec2.model.Filter)1 InstanceNetworkInterfaceSpecification (com.amazonaws.services.ec2.model.InstanceNetworkInterfaceSpecification)1 Placement (com.amazonaws.services.ec2.model.Placement)1 Reservation (com.amazonaws.services.ec2.model.Reservation)1 Instance (com.google.api.services.compute.model.Instance)1