Search in sources :

Example 1 with SpotPrice

use of com.amazonaws.services.ec2.model.SpotPrice in project herd by FINRAOS.

the class Ec2DaoImpl method getLatestSpotPrices.

/**
 * This implementation uses DescribeSpotPriceHistory API which returns the latest spot price history for the specified AZ and instance types. This method
 * then filters the returned list to only contain the latest spot price for each instance type.
 */
@Override
public List<SpotPrice> getLatestSpotPrices(String availabilityZone, Collection<String> instanceTypes, Collection<String> productDescriptions, AwsParamsDto awsParamsDto) {
    AmazonEC2Client ec2Client = getEc2Client(awsParamsDto);
    DescribeSpotPriceHistoryRequest describeSpotPriceHistoryRequest = new DescribeSpotPriceHistoryRequest();
    describeSpotPriceHistoryRequest.setAvailabilityZone(availabilityZone);
    describeSpotPriceHistoryRequest.setInstanceTypes(instanceTypes);
    describeSpotPriceHistoryRequest.setProductDescriptions(productDescriptions);
    DescribeSpotPriceHistoryResult describeSpotPriceHistoryResult = ec2Operations.describeSpotPriceHistory(ec2Client, describeSpotPriceHistoryRequest);
    List<SpotPrice> spotPrices = new ArrayList<>();
    Set<String> instanceTypesFound = new HashSet<>();
    for (SpotPrice spotPriceHistoryEntry : describeSpotPriceHistoryResult.getSpotPriceHistory()) {
        if (instanceTypesFound.add(spotPriceHistoryEntry.getInstanceType())) {
            spotPrices.add(spotPriceHistoryEntry);
        }
    }
    return spotPrices;
}
Also used : AmazonEC2Client(com.amazonaws.services.ec2.AmazonEC2Client) DescribeSpotPriceHistoryResult(com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryResult) SpotPrice(com.amazonaws.services.ec2.model.SpotPrice) DescribeSpotPriceHistoryRequest(com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryRequest) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet)

Example 2 with SpotPrice

use of com.amazonaws.services.ec2.model.SpotPrice in project herd by FINRAOS.

the class MockEc2OperationsImpl method describeSpotPriceHistory.

@Override
public DescribeSpotPriceHistoryResult describeSpotPriceHistory(AmazonEC2Client ec2Client, DescribeSpotPriceHistoryRequest describeSpotPriceHistoryRequest) {
    List<SpotPrice> spotPriceHistories = new ArrayList<>();
    String requestedAvailabilityZone = describeSpotPriceHistoryRequest.getAvailabilityZone();
    // Get availability zones to search for
    Set<MockAvailabilityZone> requestedAvailabilityZones = new HashSet<>();
    // If requested zone is specified, get and add
    if (requestedAvailabilityZone != null) {
        requestedAvailabilityZones.add(mockAvailabilityZones.get(requestedAvailabilityZone));
    } else // If requested zone is not specified, add all
    {
        requestedAvailabilityZones.addAll(mockAvailabilityZones.values());
    }
    // Get instance types to search for
    List<String> requestedInstanceTypes = describeSpotPriceHistoryRequest.getInstanceTypes();
    // If not specified, add all instance types
    if (requestedInstanceTypes.isEmpty()) {
        requestedInstanceTypes.addAll(mockInstanceTypes);
    }
    // search for price for all AZ and instance types requested
    for (MockAvailabilityZone mockAvailabilityZone : requestedAvailabilityZones) {
        for (String requestedInstanceType : requestedInstanceTypes) {
            MockSpotPrice mockSpotPrice = mockAvailabilityZone.getSpotPrices().get(requestedInstanceType);
            if (mockSpotPrice != null) {
                spotPriceHistories.add(mockSpotPrice.toAwsObject());
            }
        }
    }
    DescribeSpotPriceHistoryResult describeSpotPriceHistoryResult = new DescribeSpotPriceHistoryResult();
    describeSpotPriceHistoryResult.setSpotPriceHistory(spotPriceHistories);
    return describeSpotPriceHistoryResult;
}
Also used : DescribeSpotPriceHistoryResult(com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryResult) SpotPrice(com.amazonaws.services.ec2.model.SpotPrice) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet)

Example 3 with SpotPrice

use of com.amazonaws.services.ec2.model.SpotPrice in project herd by FINRAOS.

the class MockSpotPrice method toAwsObject.

/**
 * Converts this object into an AWS equivalent object.
 *
 * @return A new equivalent AWS object
 */
public SpotPrice toAwsObject() {
    SpotPrice spotPrice = new SpotPrice();
    spotPrice.setAvailabilityZone(availabilityZone);
    spotPrice.setInstanceType(type);
    spotPrice.setSpotPrice(this.spotPrice);
    return spotPrice;
}
Also used : SpotPrice(com.amazonaws.services.ec2.model.SpotPrice)

Example 4 with SpotPrice

use of com.amazonaws.services.ec2.model.SpotPrice in project herd by FINRAOS.

the class Ec2DaoTest method testGetLatestSpotPricesSpotPriceHistoryContainDuplicateTypeInstances.

@Test
public void testGetLatestSpotPricesSpotPriceHistoryContainDuplicateTypeInstances() {
    // Initialize inputs.
    Collection<String> instanceTypes = Arrays.asList(MockEc2OperationsImpl.INSTANCE_TYPE_1, MockEc2OperationsImpl.INSTANCE_TYPE_2);
    Collection<String> productDescriptions = Arrays.asList("b", "c");
    // Set up mock.
    mock(RetryPolicyFactory.class);
    Ec2Operations ec2Operations = mock(Ec2Operations.class);
    ((Ec2DaoImpl) ec2Dao).setEc2Operations(ec2Operations);
    DescribeSpotPriceHistoryResult describeSpotPriceHistoryResult = new DescribeSpotPriceHistoryResult();
    List<SpotPrice> spotPrices = new ArrayList<>();
    spotPrices.add(new MockSpotPrice(MockEc2OperationsImpl.INSTANCE_TYPE_1, MockEc2OperationsImpl.AVAILABILITY_ZONE_1, MockEc2OperationsImpl.SPOT_PRICE_HIGH).toAwsObject());
    spotPrices.add(new MockSpotPrice(MockEc2OperationsImpl.INSTANCE_TYPE_1, MockEc2OperationsImpl.AVAILABILITY_ZONE_2, MockEc2OperationsImpl.SPOT_PRICE_HIGH).toAwsObject());
    describeSpotPriceHistoryResult.setSpotPriceHistory(spotPrices);
    when(ec2Operations.describeSpotPriceHistory(any(), any())).thenReturn(describeSpotPriceHistoryResult);
    // Execute MUT.
    List<SpotPrice> result = ec2Dao.getLatestSpotPrices(MockEc2OperationsImpl.AVAILABILITY_ZONE_1, instanceTypes, productDescriptions, new AwsParamsDto());
    // Verify that the dependency was called with the correct parameters.
    verify(ec2Operations).describeSpotPriceHistory(any(), equalsDescribeSpotPriceHistoryRequest(MockEc2OperationsImpl.AVAILABILITY_ZONE_1, instanceTypes, productDescriptions));
    // Verify that result contains only one spot price entry.
    assertEquals(1, CollectionUtils.size(result));
}
Also used : DescribeSpotPriceHistoryResult(com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryResult) MockSpotPrice(org.finra.herd.dao.impl.MockSpotPrice) SpotPrice(com.amazonaws.services.ec2.model.SpotPrice) AwsParamsDto(org.finra.herd.model.dto.AwsParamsDto) Ec2DaoImpl(org.finra.herd.dao.impl.Ec2DaoImpl) MockSpotPrice(org.finra.herd.dao.impl.MockSpotPrice) ArrayList(java.util.ArrayList) Test(org.junit.Test)

Example 5 with SpotPrice

use of com.amazonaws.services.ec2.model.SpotPrice in project herd by FINRAOS.

the class EmrPricingHelper method getInstanceTypeSpotPrices.

/**
 * Returns a mapping of instance types to spot prices for the given AZ and instance types. The spot prices are retrieved from EC2 API.
 * <p/>
 * This method also validates that the given instance types are real instance types supported by AWS.
 *
 * @param availabilityZone the AZ of the spot instances
 * @param instanceTypes the size of the spot instances
 * @param awsParamsDto the AWS related parameters for access/secret keys and proxy details
 *
 * @return the mapping of instance type to spot prices
 * @throws ObjectNotFoundException when any of the instance type does not exist in AWS
 */
private Map<String, BigDecimal> getInstanceTypeSpotPrices(AvailabilityZone availabilityZone, Set<String> instanceTypes, AwsParamsDto awsParamsDto) {
    List<String> productDescriptions = herdStringHelper.getDelimitedConfigurationValue(ConfigurationValue.EMR_SPOT_PRICE_HISTORY_PRODUCT_DESCRIPTIONS);
    List<SpotPrice> spotPrices = ec2Dao.getLatestSpotPrices(availabilityZone.getZoneName(), instanceTypes, productDescriptions, awsParamsDto);
    Map<String, BigDecimal> instanceTypeSpotPrices = new HashMap<>();
    for (SpotPrice spotPrice : spotPrices) {
        instanceTypeSpotPrices.put(spotPrice.getInstanceType(), new BigDecimal(spotPrice.getSpotPrice()));
    }
    return instanceTypeSpotPrices;
}
Also used : SpotPrice(com.amazonaws.services.ec2.model.SpotPrice) HashMap(java.util.HashMap) BigDecimal(java.math.BigDecimal)

Aggregations

SpotPrice (com.amazonaws.services.ec2.model.SpotPrice)5 DescribeSpotPriceHistoryResult (com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryResult)3 ArrayList (java.util.ArrayList)3 HashSet (java.util.HashSet)2 AmazonEC2Client (com.amazonaws.services.ec2.AmazonEC2Client)1 DescribeSpotPriceHistoryRequest (com.amazonaws.services.ec2.model.DescribeSpotPriceHistoryRequest)1 BigDecimal (java.math.BigDecimal)1 HashMap (java.util.HashMap)1 Ec2DaoImpl (org.finra.herd.dao.impl.Ec2DaoImpl)1 MockSpotPrice (org.finra.herd.dao.impl.MockSpotPrice)1 AwsParamsDto (org.finra.herd.model.dto.AwsParamsDto)1 Test (org.junit.Test)1