Search in sources :

Example 1 with Instance

use of com.amazonaws.services.elasticmapreduce.model.Instance in project herd by FINRAOS.

the class MockEmrOperationsImpl method listClusterInstancesRequest.

@Override
public ListInstancesResult listClusterInstancesRequest(AmazonElasticMapReduceClient emrClient, ListInstancesRequest listInstancesRequest) {
    MockEmrJobFlow cluster = getClusterByName(buildEmrClusterName(AbstractDaoTest.NAMESPACE, AbstractDaoTest.EMR_CLUSTER_DEFINITION_NAME, MOCK_CLUSTER_NOT_PROVISIONED_NAME));
    if (cluster != null && listInstancesRequest.getClusterId().equals(cluster.getJobFlowId())) {
        return new ListInstancesResult();
    }
    Instance instance = new Instance().withEc2InstanceId("EC2_EMR_MASTER_INSTANCE").withPrivateIpAddress("INSTANCE_IP_ADDRESS");
    return new ListInstancesResult().withInstances(instance);
}
Also used : ListInstancesResult(com.amazonaws.services.elasticmapreduce.model.ListInstancesResult) Instance(com.amazonaws.services.elasticmapreduce.model.Instance)

Example 2 with Instance

use of com.amazonaws.services.elasticmapreduce.model.Instance in project herd by FINRAOS.

the class EmrDaoImpl method getEmrMasterInstance.

@Override
public Instance getEmrMasterInstance(String clusterId, AwsParamsDto awsParams) throws Exception {
    // Get the master EC2 instance
    ListInstancesRequest listInstancesRequest = new ListInstancesRequest().withClusterId(clusterId).withInstanceGroupTypes(InstanceGroupType.MASTER);
    List<Instance> instances = emrOperations.listClusterInstancesRequest(getEmrClient(awsParams), listInstancesRequest).getInstances();
    // Throw error in case there are no master instances found yet
    if (instances.size() == 0) {
        throw new IllegalArgumentException("No master instances found for the cluster \"" + clusterId + "\".");
    }
    // EMR has only one master node.
    return instances.get(0);
}
Also used : Instance(com.amazonaws.services.elasticmapreduce.model.Instance) ListInstancesRequest(com.amazonaws.services.elasticmapreduce.model.ListInstancesRequest)

Example 3 with Instance

use of com.amazonaws.services.elasticmapreduce.model.Instance in project herd by FINRAOS.

the class EmrDaoImpl method addEmrMasterSecurityGroups.

@Override
public List<String> addEmrMasterSecurityGroups(String clusterId, List<String> securityGroups, AwsParamsDto awsParams) throws Exception {
    // Get the master EC2 instance
    ListInstancesRequest listInstancesRequest = new ListInstancesRequest().withClusterId(clusterId).withInstanceGroupTypes(InstanceGroupType.MASTER);
    List<Instance> instances = emrOperations.listClusterInstancesRequest(getEmrClient(awsParams), listInstancesRequest).getInstances();
    // Throw error in case there are no master instances found yet
    if (instances.size() == 0) {
        throw new IllegalArgumentException("No master instances found for the cluster \"" + clusterId + "\".");
    }
    for (Instance instance : instances) {
        ec2Dao.addSecurityGroupsToEc2Instance(instance.getEc2InstanceId(), securityGroups, awsParams);
    }
    return securityGroups;
}
Also used : Instance(com.amazonaws.services.elasticmapreduce.model.Instance) ListInstancesRequest(com.amazonaws.services.elasticmapreduce.model.ListInstancesRequest)

Example 4 with Instance

use of com.amazonaws.services.elasticmapreduce.model.Instance in project herd by FINRAOS.

the class EmrDaoTest method getEmrMasterInstanceReturnsInstance.

@Test
public void getEmrMasterInstanceReturnsInstance() throws Exception {
    String clusterId = "clusterId";
    Instance expectedInstance = new Instance();
    /*
         * Stub EmrOperations.listClusterInstancesRequest() and assert correct parameters are being passed into the method when the call is made.
         */
    when(mockEmrOperations.listClusterInstancesRequest(any(), any())).thenAnswer(new Answer<ListInstancesResult>() {

        @Override
        public ListInstancesResult answer(InvocationOnMock invocation) throws Throwable {
            /*
                 * Assert correct parameters are used when calling this method.
                 */
            ListInstancesRequest listInstancesRequest = invocation.getArgument(1);
            assertEquals(clusterId, listInstancesRequest.getClusterId());
            List<String> instanceGroupTypes = listInstancesRequest.getInstanceGroupTypes();
            assertEquals(1, instanceGroupTypes.size());
            assertEquals("MASTER", instanceGroupTypes.get(0));
            ListInstancesResult listInstancesResult = new ListInstancesResult();
            listInstancesResult.setInstances(Arrays.asList(expectedInstance));
            return listInstancesResult;
        }
    });
    Instance actualInstance = emrDao.getEmrMasterInstance(clusterId, new AwsParamsDto());
    assertEquals(expectedInstance, actualInstance);
}
Also used : AwsParamsDto(org.finra.herd.model.dto.AwsParamsDto) ListInstancesResult(com.amazonaws.services.elasticmapreduce.model.ListInstancesResult) Instance(com.amazonaws.services.elasticmapreduce.model.Instance) InvocationOnMock(org.mockito.invocation.InvocationOnMock) ListInstancesRequest(com.amazonaws.services.elasticmapreduce.model.ListInstancesRequest) List(java.util.List) ArrayList(java.util.ArrayList) Test(org.junit.Test)

Example 5 with Instance

use of com.amazonaws.services.elasticmapreduce.model.Instance in project herd by FINRAOS.

the class EmrDaoTest method addEmrMasterSecurityGroupsCallsEc2AddSecurityGroup.

@Test
public void addEmrMasterSecurityGroupsCallsEc2AddSecurityGroup() throws Exception {
    String clusterName = "clusterName";
    List<String> securityGroups = Arrays.asList("securityGroup");
    AwsParamsDto awsParams = new AwsParamsDto();
    String ec2InstanceId = "ec2InstanceId";
    ListClustersResult listClustersResult = new ListClustersResult();
    listClustersResult.setClusters(new ArrayList<>());
    ClusterSummary clusterSummary = new ClusterSummary();
    clusterSummary.setId("clusterId");
    clusterSummary.setName(clusterName);
    listClustersResult.getClusters().add(clusterSummary);
    when(mockEmrOperations.listEmrClusters(any(), any())).thenReturn(listClustersResult);
    ListInstancesResult listInstancesResult = new ListInstancesResult();
    listInstancesResult.setInstances(new ArrayList<>());
    Instance instance = new Instance();
    instance.setEc2InstanceId(ec2InstanceId);
    listInstancesResult.getInstances().add(instance);
    when(mockEmrOperations.listClusterInstancesRequest(any(), any())).thenReturn(listInstancesResult);
    emrDao.addEmrMasterSecurityGroups(clusterName, securityGroups, awsParams);
    verify(mockEc2Dao).addSecurityGroupsToEc2Instance(eq(ec2InstanceId), eq(securityGroups), any());
    verifyNoMoreInteractions(mockEc2Dao);
}
Also used : AwsParamsDto(org.finra.herd.model.dto.AwsParamsDto) ListInstancesResult(com.amazonaws.services.elasticmapreduce.model.ListInstancesResult) Instance(com.amazonaws.services.elasticmapreduce.model.Instance) ClusterSummary(com.amazonaws.services.elasticmapreduce.model.ClusterSummary) ListClustersResult(com.amazonaws.services.elasticmapreduce.model.ListClustersResult) Test(org.junit.Test)

Aggregations

Instance (com.amazonaws.services.elasticmapreduce.model.Instance)5 ListInstancesRequest (com.amazonaws.services.elasticmapreduce.model.ListInstancesRequest)3 ListInstancesResult (com.amazonaws.services.elasticmapreduce.model.ListInstancesResult)3 AwsParamsDto (org.finra.herd.model.dto.AwsParamsDto)2 Test (org.junit.Test)2 ClusterSummary (com.amazonaws.services.elasticmapreduce.model.ClusterSummary)1 ListClustersResult (com.amazonaws.services.elasticmapreduce.model.ListClustersResult)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 InvocationOnMock (org.mockito.invocation.InvocationOnMock)1