Search in sources :

Example 1 with Container

use of com.sequenceiq.cloudbreak.cloud.yarn.client.model.core.Container in project cloudbreak by hortonworks.

the class YarnMetadataCollector method groupContainersByInstanceGroup.

private ListMultimap<String, Container> groupContainersByInstanceGroup(Iterable<Container> containers) {
    ListMultimap<String, Container> groupByInstanceGroup = ArrayListMultimap.create();
    for (Container container : containers) {
        String groupName = container.getComponentName();
        groupByInstanceGroup.put(groupName, container);
    }
    return groupByInstanceGroup;
}
Also used : Container(com.sequenceiq.cloudbreak.cloud.yarn.client.model.core.Container)

Example 2 with Container

use of com.sequenceiq.cloudbreak.cloud.yarn.client.model.core.Container in project cloudbreak by hortonworks.

the class YarnMetadataCollector method collect.

@Override
public List<CloudVmMetaDataStatus> collect(AuthenticatedContext authenticatedContext, List<CloudResource> resources, List<CloudInstance> vms) {
    try {
        YarnClient yarnClient = yarnClientUtil.createYarnClient(authenticatedContext);
        CloudResource yarnApplication = getYarnApplcationResource(resources);
        ApplicationDetailRequest applicationDetailRequest = new ApplicationDetailRequest();
        applicationDetailRequest.setName(yarnApplication.getName());
        ResponseContext responseContext = yarnClient.getApplicationDetail(applicationDetailRequest);
        if (responseContext.getStatusCode() == YarnResourceConstants.HTTP_SUCCESS) {
            ApplicationDetailResponse applicationDetailResponse = (ApplicationDetailResponse) responseContext.getResponseObject();
            ListMultimap<String, CloudInstance> groupInstancesByInstanceGroup = groupInstancesByInstanceGroup(vms);
            ListMultimap<String, Container> groupContainersByInstanceGroup = groupContainersByInstanceGroup(applicationDetailResponse.getContainers());
            List<CloudVmMetaDataStatus> cloudVmMetaDataStatuses = Lists.newArrayList();
            for (String groupName : groupContainersByInstanceGroup.keySet()) {
                List<CloudInstance> groupInstances = groupInstancesByInstanceGroup.get(groupName);
                List<Container> groupContainers = groupContainersByInstanceGroup.get(groupName);
                Map<String, CloudInstance> mapByInstanceId = mapByInstanceId(groupInstances);
                Queue<CloudInstance> untrackedInstances = untrackedInstances(groupInstances);
                for (Container container : groupContainers) {
                    String containerId = container.getId();
                    CloudInstance cloudInstance = mapByInstanceId.get(containerId);
                    if (cloudInstance == null) {
                        if (!untrackedInstances.isEmpty()) {
                            cloudInstance = untrackedInstances.remove();
                            cloudInstance = new CloudInstance(containerId, cloudInstance.getTemplate(), cloudInstance.getAuthentication());
                        }
                    }
                    if (cloudInstance != null) {
                        String ipAddress = container.getIp();
                        CloudInstanceMetaData md = new CloudInstanceMetaData(ipAddress, ipAddress);
                        CloudVmInstanceStatus cloudVmInstanceStatus = new CloudVmInstanceStatus(cloudInstance, InstanceStatus.CREATED);
                        CloudVmMetaDataStatus cloudVmMetaDataStatus = new CloudVmMetaDataStatus(cloudVmInstanceStatus, md);
                        cloudVmMetaDataStatuses.add(cloudVmMetaDataStatus);
                    }
                }
            }
            return cloudVmMetaDataStatuses;
        } else {
            ApplicationErrorResponse errorResponse = responseContext.getResponseError();
            throw new CloudConnectorException(String.format("Failed to get yarn application details: HTTP Return: %d Error: %s", responseContext.getStatusCode(), errorResponse == null ? "unknown" : errorResponse.getDiagnostics()));
        }
    } catch (MalformedURLException ex) {
        throw new CloudConnectorException("Failed to get yarn application details", ex);
    }
}
Also used : MalformedURLException(java.net.MalformedURLException) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) ApplicationErrorResponse(com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ApplicationErrorResponse) YarnClient(com.sequenceiq.cloudbreak.cloud.yarn.client.YarnClient) ApplicationDetailResponse(com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ApplicationDetailResponse) Container(com.sequenceiq.cloudbreak.cloud.yarn.client.model.core.Container) CloudInstanceMetaData(com.sequenceiq.cloudbreak.cloud.model.CloudInstanceMetaData) ApplicationDetailRequest(com.sequenceiq.cloudbreak.cloud.yarn.client.model.request.ApplicationDetailRequest) CloudVmMetaDataStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus) ResponseContext(com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ResponseContext) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource)

Aggregations

Container (com.sequenceiq.cloudbreak.cloud.yarn.client.model.core.Container)2 CloudConnectorException (com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException)1 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)1 CloudInstanceMetaData (com.sequenceiq.cloudbreak.cloud.model.CloudInstanceMetaData)1 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)1 CloudVmInstanceStatus (com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus)1 CloudVmMetaDataStatus (com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus)1 YarnClient (com.sequenceiq.cloudbreak.cloud.yarn.client.YarnClient)1 ApplicationDetailRequest (com.sequenceiq.cloudbreak.cloud.yarn.client.model.request.ApplicationDetailRequest)1 ApplicationDetailResponse (com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ApplicationDetailResponse)1 ApplicationErrorResponse (com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ApplicationErrorResponse)1 ResponseContext (com.sequenceiq.cloudbreak.cloud.yarn.client.model.response.ResponseContext)1 MalformedURLException (java.net.MalformedURLException)1