Search in sources :

Example 1 with CloudStorage

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage in project cloudbreak by hortonworks.

the class CloudStorageConverter method requestToFileSystem.

public FileSystem requestToFileSystem(CloudStorageBase cloudStorageRequest) {
    FileSystem fileSystem = new FileSystem();
    fileSystem.setName(nameGenerator.generateName(FILESYSTEM));
    FileSystemType fileSystemType = fileSystemResolver.determineFileSystemType(cloudStorageRequest);
    fileSystem.setType(fileSystemType);
    CloudStorage cloudStorage = new CloudStorage();
    String s3GuardDynamoTableName = getS3GuardDynamoTableName(cloudStorageRequest);
    cloudStorage.setS3GuardDynamoTableName(s3GuardDynamoTableName);
    List<StorageLocation> storageLocations = cloudStorageRequest.getLocations().stream().map(this::storageLocationRequestToStorageLocation).collect(Collectors.toList());
    cloudStorage.setLocations(storageLocations);
    if (cloudStorageRequest.getIdentities() != null) {
        List<CloudIdentity> cloudIdentities = cloudStorageRequest.getIdentities().stream().map(this::identityRequestToCloudIdentity).collect(Collectors.toList());
        cloudStorage.setCloudIdentities(cloudIdentities);
    }
    cloudStorage.setAccountMapping(accountMappingRequestToAccountMapping(cloudStorageRequest.getAccountMapping()));
    fileSystem.setCloudStorage(cloudStorage);
    return fileSystem;
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) SpiFileSystem(com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) FileSystemType(com.sequenceiq.common.model.FileSystemType) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation)

Example 2 with CloudStorage

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage in project cloudbreak by hortonworks.

the class RemoteDataContextWorkaroundService method prepareFilesystemIfNotPresentedButSdxHasIt.

private void prepareFilesystemIfNotPresentedButSdxHasIt(Cluster requestedCluster, Stack stack) {
    if (requestedCluster.getFileSystem() == null) {
        if (hasFilesystem(stack.getCluster().getFileSystem())) {
            FileSystem fileSystem = new FileSystem();
            fileSystem.setCloudStorage(new CloudStorage());
            fileSystem.setName(nameGenerator.generateName(FILESYSTEM));
            fileSystem.setType(stack.getCluster().getFileSystem().getType());
            requestedCluster.setFileSystem(fileSystem);
        }
    }
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem)

Example 3 with CloudStorage

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage in project cloudbreak by hortonworks.

the class RemoteDataContextWorkaroundService method prepareFilesytem.

public FileSystem prepareFilesytem(Cluster requestedCluster, Stack datalakeStack) {
    prepareFilesystemIfNotPresentedButSdxHasIt(requestedCluster, datalakeStack);
    if (hasFilesystem(requestedCluster.getFileSystem())) {
        FileSystem fileSystem = requestedCluster.getFileSystem();
        if (hasLocations(fileSystem)) {
            CloudStorage cloudStorage = fileSystem.getCloudStorage();
            FileSystem dlFileSystem = datalakeStack.getCluster().getFileSystem();
            if (hasFilesystem(dlFileSystem)) {
                if (hasLocations(dlFileSystem)) {
                    List<StorageLocation> dlStorageLocations = dlFileSystem.getCloudStorage().getLocations();
                    List<StorageLocation> hiveRelatedStorageConfigs = getHiveRelatedStorageConfigs(dlStorageLocations);
                    if (sdxHasHiveStorageConfiguration(hiveRelatedStorageConfigs)) {
                        List<StorageLocation> storageLocations = getStorageLocationsWithoutHiveRelatedLocations(cloudStorage.getLocations());
                        storageLocations.addAll(hiveRelatedStorageConfigs);
                        cloudStorage.setLocations(storageLocations);
                        fileSystem.setCloudStorage(cloudStorage);
                    }
                }
            }
        }
    }
    return requestedCluster.getFileSystem();
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation)

Example 4 with CloudStorage

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage in project cloudbreak by hortonworks.

the class ClouderaManagerStorageErrorMapperTest method setUp.

@BeforeEach
void setUp() {
    underTest = new ClouderaManagerStorageErrorMapper();
    exception = new CloudStorageConfigurationFailedException(EXCEPTION_MESSAGE);
    cluster = new Cluster();
    fileSystem = new FileSystem();
    cloudStorage = new CloudStorage();
    cloudIdentity = new CloudIdentity();
    cloudIdentity.setIdentityType(CloudIdentityType.ID_BROKER);
    cloudStorage.setCloudIdentities(List.of(cloudIdentity));
    accountMapping = new AccountMapping();
    accountMapping.setUserMappings(Map.ofEntries(entry("hive", "myDataAccessRole"), entry("solr", "myRangerAuditRole")));
    cloudStorage.setAccountMapping(accountMapping);
    StorageLocation locationRangerAudit = new StorageLocation();
    locationRangerAudit.setType(CloudStorageCdpService.RANGER_AUDIT);
    locationRangerAudit.setValue("myRangerAuditLocation");
    cloudStorage.setLocations(List.of(locationRangerAudit));
    fileSystem.setCloudStorage(cloudStorage);
    cluster.setFileSystem(fileSystem);
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) CloudIdentity(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity) Cluster(com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster) CloudStorageConfigurationFailedException(com.sequenceiq.cloudbreak.cm.exception.CloudStorageConfigurationFailedException) AccountMapping(com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping) StorageLocation(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 5 with CloudStorage

use of com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage in project cloudbreak by hortonworks.

the class StackToTemplatePreparationObjectConverterTest method testStackInputAccountMappings.

@Test
public void testStackInputAccountMappings() {
    FileSystem sourceFileSystem = mock(FileSystem.class);
    CloudStorage sourceCloudStorage = mock(CloudStorage.class);
    when(sourceCluster.getFileSystem()).thenReturn(sourceFileSystem);
    when(sourceFileSystem.getCloudStorage()).thenReturn(sourceCloudStorage);
    AccountMapping accountMapping = new AccountMapping();
    accountMapping.setGroupMappings(GROUP_MAPPINGS);
    accountMapping.setUserMappings(USER_MAPPINGS);
    when(sourceCloudStorage.getAccountMapping()).thenReturn(accountMapping);
    when(blueprintViewProvider.getBlueprintView(any())).thenReturn(getBlueprintView());
    TemplatePreparationObject result = underTest.convert(stackMock);
    AccountMappingView accountMappingView = result.getAccountMappingView();
    assertThat(accountMappingView).isNotNull();
    assertThat(accountMappingView.getGroupMappings()).isEqualTo(GROUP_MAPPINGS);
    assertThat(accountMappingView.getUserMappings()).isEqualTo(USER_MAPPINGS);
}
Also used : CloudStorage(com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage) TemplatePreparationObject(com.sequenceiq.cloudbreak.template.TemplatePreparationObject) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) AccountMapping(com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping) AccountMappingView(com.sequenceiq.cloudbreak.template.views.AccountMappingView) Test(org.junit.jupiter.api.Test)

Aggregations

CloudStorage (com.sequenceiq.cloudbreak.domain.cloudstorage.CloudStorage)13 FileSystem (com.sequenceiq.cloudbreak.domain.FileSystem)11 CloudIdentity (com.sequenceiq.cloudbreak.domain.cloudstorage.CloudIdentity)6 StorageLocation (com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation)6 CloudFileSystemView (com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudFileSystemView)4 AccountMapping (com.sequenceiq.cloudbreak.domain.cloudstorage.AccountMapping)4 HashMap (java.util.HashMap)4 SpiFileSystem (com.sequenceiq.cloudbreak.cloud.model.SpiFileSystem)3 Cluster (com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster)3 AwsStorageParameters (com.sequenceiq.common.api.cloudstorage.AwsStorageParameters)3 CloudStorageRequest (com.sequenceiq.common.api.cloudstorage.CloudStorageRequest)3 S3Guard (com.sequenceiq.common.api.cloudstorage.S3Guard)3 StorageIdentityBase (com.sequenceiq.common.api.cloudstorage.StorageIdentityBase)3 StorageLocationBase (com.sequenceiq.common.api.cloudstorage.StorageLocationBase)3 ArrayList (java.util.ArrayList)3 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)2 CloudS3View (com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudS3View)2 CloudEfsConfiguration (com.sequenceiq.cloudbreak.cloud.model.filesystem.efs.CloudEfsConfiguration)2 MissingResourceNameGenerator (com.sequenceiq.cloudbreak.common.converter.MissingResourceNameGenerator)2 BadRequestException (com.sequenceiq.cloudbreak.common.exception.BadRequestException)2