Search in sources :

Example 1 with CloudIdentity

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity in project cloudbreak by hortonworks.

the class CloudStorageConverter method requestToFileSystem.

public FileSystem requestToFileSystem(CloudStorageBase cloudStorageRequest) {
    FileSystem fileSystem = new FileSystem();
    fileSystem.setName(nameGenerator.generateName(FILESYSTEM));
    FileSystemType fileSystemType = fileSystemResolver.determineFileSystemType(cloudStorageRequest);
    fileSystem.setType(fileSystemType);
    CloudStorage cloudStorage = new CloudStorage();
    String s3GuardDynamoTableName = getS3GuardDynamoTableName(cloudStorageRequest);
    cloudStorage.setS3GuardDynamoTableName(s3GuardDynamoTableName);
    List<StorageLocation> storageLocations = cloudStorageRequest.getLocations().stream().map(this::storageLocationRequestToStorageLocation).collect(Collectors.toList());
    cloudStorage.setLocations(storageLocations);
    if (cloudStorageRequest.getIdentities() != null) {
        List<CloudIdentity> cloudIdentities = cloudStorageRequest.getIdentities().stream().map(this::identityRequestToCloudIdentity).collect(Collectors.toList());
        cloudStorage.setCloudIdentities(cloudIdentities);
    }
    cloudStorage.setAccountMapping(accountMappingRequestToAccountMapping(cloudStorageRequest.getAccountMapping()));
    fileSystem.setCloudStorage(cloudStorage);
    return fileSystem;
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) SpiFileSystem(com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) FileSystemType(com.sequenceiq.common.model.FileSystemType) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation)

Example 2 with CloudIdentity

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity in project cloudbreak by hortonworks.

the class CloudFileSystemViewProviderTest method getCloudIdentities.

private List<CloudIdentity> getCloudIdentities() {
    List<CloudIdentity> cloudIdentities = new ArrayList<>();
    CloudIdentity idBroker = new CloudIdentity();
    idBroker.setIdentityType(CloudIdentityType.ID_BROKER);
    S3Identity idBrokerS3Identity = new S3Identity();
    idBrokerS3Identity.setInstanceProfile(ID_BROKER_INSTANCE_PROFILE);
    idBroker.setS3Identity(idBrokerS3Identity);
    cloudIdentities.add(idBroker);
    CloudIdentity log = new CloudIdentity();
    log.setIdentityType(CloudIdentityType.LOG);
    S3Identity logS3Identity = new S3Identity();
    logS3Identity.setInstanceProfile(LOG_INSTANCE_PROFILE);
    log.setS3Identity(logS3Identity);
    cloudIdentities.add(log);
    return cloudIdentities;
}
Also used : CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) ArrayList(java.util.ArrayList) S3Identity(com.sequenceiq.cloudbreak.domain.cloudstorage.S3Identity)

Example 3 with CloudIdentity

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity in project cloudbreak by hortonworks.

the class ClouderaManagerStorageErrorMapperTest method setUp.

@BeforeEach
void setUp() {
    underTest = new ClouderaManagerStorageErrorMapper();
    exception = new CloudStorageConfigurationFailedException(EXCEPTION_MESSAGE);
    cluster = new Cluster();
    fileSystem = new FileSystem();
    cloudStorage = new CloudStorage();
    cloudIdentity = new CloudIdentity();
    cloudIdentity.setIdentityType(CloudIdentityType.ID_BROKER);
    cloudStorage.setCloudIdentities(List.of(cloudIdentity));
    accountMapping = new AccountMapping();
    accountMapping.setUserMappings(Map.ofEntries(entry("hive", "myDataAccessRole"), entry("solr", "myRangerAuditRole")));
    cloudStorage.setAccountMapping(accountMapping);
    StorageLocation locationRangerAudit = new StorageLocation();
    locationRangerAudit.setType(CloudStorageCdpService.RANGER_AUDIT);
    locationRangerAudit.setValue("myRangerAuditLocation");
    cloudStorage.setLocations(List.of(locationRangerAudit));
    fileSystem.setCloudStorage(cloudStorage);
    cluster.setFileSystem(fileSystem);
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) Cluster(com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster) CloudStorageConfigurationFailedException(com.sequenceiq.cloudbreak.cm.exception.CloudStorageConfigurationFailedException) AccountMapping(com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 4 with CloudIdentity

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity in project cloudbreak by hortonworks.

the class CloudStorageConverter method identityRequestToCloudIdentity.

private CloudIdentity identityRequestToCloudIdentity(StorageIdentityBase storageIdentityRequest) {
    CloudIdentity cloudIdentity = new CloudIdentity();
    cloudIdentity.setIdentityType(storageIdentityRequest.getType());
    if (storageIdentityRequest.getS3() != null) {
        S3Identity s3Identity = identityRequestToS3(storageIdentityRequest);
        cloudIdentity.setS3Identity(s3Identity);
    }
    if (storageIdentityRequest.getEfs() != null) {
        EfsIdentity efsIdentity = identityRequestToEfs(storageIdentityRequest);
        cloudIdentity.setEfsIdentity(efsIdentity);
    }
    if (storageIdentityRequest.getWasb() != null) {
        WasbIdentity wasbIdentity = identityRequestToWasb(storageIdentityRequest);
        cloudIdentity.setWasbIdentity(wasbIdentity);
    }
    if (storageIdentityRequest.getAdlsGen2() != null) {
        AdlsGen2Identity identity = identityRequestToAdlsGen2(storageIdentityRequest);
        cloudIdentity.setAdlsGen2Identity(identity);
    }
    if (storageIdentityRequest.getGcs() != null) {
        GcsIdentity identity = identityRequestToGcs(storageIdentityRequest);
        cloudIdentity.setGcsIdentity(identity);
    }
    if (storageIdentityRequest.getAdls() != null) {
        throw new BadRequestException("ADLS cloud storage is not (yet) supported.");
    }
    return cloudIdentity;
}
Also used : AdlsGen2Identity(com.sequenceiq.cloudbreak.domain.cloudstorage.AdlsGen2Identity) GcsIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.GcsIdentity) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) EfsIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.EfsIdentity) BadRequestException(com.sequenceiq.cloudbreak.common.exception.BadRequestException) S3Identity(com.sequenceiq.cloudbreak.domain.cloudstorage.S3Identity) WasbIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.WasbIdentity)

Example 5 with CloudIdentity

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity in project cloudbreak by hortonworks.

the class CloudStorageConverter method requestToAdditionalFileSystem.

public FileSystem requestToAdditionalFileSystem(CloudStorageBase cloudStorageRequest) {
    FileSystem fileSystem = new FileSystem();
    AwsEfsParameters efsParameters = cloudStorageRequest.getAws() == null ? null : cloudStorageRequest.getAws().getEfsParameters();
    if (efsParameters == null || StringUtils.isEmpty(efsParameters.getName())) {
        return null;
    }
    fileSystem.setName(efsParameters.getName());
    FileSystemType fileSystemType = FileSystemType.EFS;
    fileSystem.setType(fileSystemType);
    Map<String, Object> configurations = new HashMap<>();
    configurations.put(CloudEfsConfiguration.KEY_BACKUP_POLICY_STATUS, efsParameters.getBackupPolicyStatus());
    configurations.put(CloudEfsConfiguration.KEY_ENCRYPTED, efsParameters.getEncrypted());
    configurations.put(CloudEfsConfiguration.KEY_FILESYSTEM_POLICY, efsParameters.getFileSystemPolicy());
    configurations.put(CloudEfsConfiguration.KEY_FILESYSTEM_TAGS, efsParameters.getFileSystemTags());
    configurations.put(CloudEfsConfiguration.KEY_KMSKEYID, efsParameters.getKmsKeyId());
    configurations.put(CloudEfsConfiguration.KEY_LIFECYCLE_POLICIES, efsParameters.getLifeCyclePolicies());
    configurations.put(CloudEfsConfiguration.KEY_PERFORMANCE_MODE, efsParameters.getPerformanceMode());
    configurations.put(CloudEfsConfiguration.KEY_PROVISIONED_THROUGHPUT_INMIBPS, efsParameters.getProvisionedThroughputInMibps());
    configurations.put(CloudEfsConfiguration.KEY_THROUGHPUT_MODE, efsParameters.getThroughputMode());
    String configString;
    try {
        configString = JsonUtil.writeValueAsString(configurations);
    } catch (JsonProcessingException ignored) {
        configString = configurations.toString();
    }
    fileSystem.setConfigurations(new Json(configString));
    CloudStorage cloudStorage = new CloudStorage();
    if (cloudStorageRequest.getIdentities() != null) {
        Optional<CloudIdentity> cloudIdentity = cloudStorageRequest.getIdentities().stream().map(this::identityRequestToCloudIdentity).filter(currCloudIdentity -> currCloudIdentity.getEfsIdentity() != null).findFirst();
        if (cloudIdentity != null && cloudIdentity.get() != null) {
            cloudStorage.setCloudIdentities(List.of(cloudIdentity.get()));
        }
    }
    cloudStorage.setAccountMapping(accountMappingRequestToAccountMapping(cloudStorageRequest.getAccountMapping()));
    fileSystem.setCloudStorage(cloudStorage);
    return fileSystem;
}
Also used : EfsIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.EfsIdentity) AwsStorageParameters(com.sequenceiq.common.api.cloudstorage.AwsStorageParameters) AwsEfsParameters(com.sequenceiq.common.api.cloudstorage.AwsEfsParameters) SpiFileSystem(com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem) HashMap(java.util.HashMap) JsonUtil(com.sequenceiq.cloudbreak.common.json.JsonUtil) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) StringUtils(org.apache.commons.lang3.StringUtils) AccountMapping(com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) AccountMappingBase(com.sequenceiq.common.api.cloudstorage.AccountMappingBase) CloudS3View(com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudS3View) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) Map(java.util.Map) AdlsGen2Identity(com.sequenceiq.cloudbreak.domain.cloudstorage.AdlsGen2Identity) GcsIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.GcsIdentity) CloudEfsConfiguration(com.sequenceiq.cloudbreak.cloud.model.filesystem.efs.CloudEfsConfiguration) BadRequestException(com.sequenceiq.cloudbreak.common.exception.BadRequestException) CloudStorageBase(com.sequenceiq.common.api.cloudstorage.CloudStorageBase) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) StorageIdentityBase(com.sequenceiq.common.api.cloudstorage.StorageIdentityBase) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) NullUtil(com.sequenceiq.cloudbreak.util.NullUtil) CloudStorageResponse(com.sequenceiq.common.api.cloudstorage.CloudStorageResponse) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) CloudFileSystemView(com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudFileSystemView) Collectors(java.util.stream.Collectors) Json(com.sequenceiq.cloudbreak.common.json.Json) S3Identity(com.sequenceiq.cloudbreak.domain.cloudstorage.S3Identity) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation) List(java.util.List) Component(org.springframework.stereotype.Component) WasbIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.WasbIdentity) WasbCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.WasbCloudStorageV1Parameters) MissingResourceNameGenerator(com.sequenceiq.cloudbreak.common.converter.MissingResourceNameGenerator) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) FileSystemType(com.sequenceiq.common.model.FileSystemType) AdlsGen2CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.AdlsGen2CloudStorageV1Parameters) Optional(java.util.Optional) FileSystemResolver(com.sequenceiq.cloudbreak.service.filesystem.FileSystemResolver) S3Guard(com.sequenceiq.common.api.cloudstorage.S3Guard) FILESYSTEM(com.sequenceiq.cloudbreak.common.type.APIResourceType.FILESYSTEM) CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) HashMap(java.util.HashMap) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) Json(com.sequenceiq.cloudbreak.common.json.Json) CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) SpiFileSystem(com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) FileSystemType(com.sequenceiq.common.model.FileSystemType) AwsEfsParameters(com.sequenceiq.common.api.cloudstorage.AwsEfsParameters) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Aggregations

CloudIdentity (com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity)7 FileSystem (com.sequenceiq.cloudbreak.domain.FileSystem)5 CloudStorage (com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage)5 CloudFileSystemView (com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudFileSystemView)3 S3Identity (com.sequenceiq.cloudbreak.domain.cloudstorage.S3Identity)3 StorageLocation (com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation)3 HashMap (java.util.HashMap)3 SpiFileSystem (com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem)2 BadRequestException (com.sequenceiq.cloudbreak.common.exception.BadRequestException)2 AccountMapping (com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping)2 AdlsGen2Identity (com.sequenceiq.cloudbreak.domain.cloudstorage.AdlsGen2Identity)2 EfsIdentity (com.sequenceiq.cloudbreak.domain.cloudstorage.EfsIdentity)2 GcsIdentity (com.sequenceiq.cloudbreak.domain.cloudstorage.GcsIdentity)2 WasbIdentity (com.sequenceiq.cloudbreak.domain.cloudstorage.WasbIdentity)2 InstanceGroup (com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup)2 HashSet (java.util.HashSet)2 Set (java.util.Set)2 Test (org.junit.Test)2 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 CloudS3View (com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudS3View)1