Search in sources :

Example 11 with StorageLocation

use of com.sequenceiq.cloudbreak.domain.StorageLocation in project cloudbreak by hortonworks.

the class FileSystemConfigurationsViewProvider method getLegacyStorageLocations.

private Set<StorageLocationView> getLegacyStorageLocations(FileSystem source) throws IOException {
    Set<StorageLocationView> locations = new HashSet<>();
    StorageLocations storageLocations = source.getLocations().get(StorageLocations.class);
    if (storageLocations != null) {
        for (StorageLocation location : storageLocations.getLocations()) {
            locations.add(new StorageLocationView(location));
        }
    }
    return locations;
}
Also used : StorageLocation(com.sequenceiq.cloudbreak.domain.StorageLocation) StorageLocations(com.sequenceiq.cloudbreak.domain.StorageLocations) HashSet(java.util.HashSet)

Example 12 with StorageLocation

use of com.sequenceiq.cloudbreak.domain.StorageLocation in project cloudbreak by hortonworks.

the class FileSystemConfigurationsViewProvider method storageLocationToView.

private Optional<StorageLocationView> storageLocationToView(com.sequenceiq.cloudbreak.domain.cloudstorage.StorageLocation storageLocation, ConfigQueryEntries configQueryEntries) {
    return configQueryEntries.getEntries().stream().filter(configQueryEntry -> configQueryEntry.getType().equals(storageLocation.getType())).findFirst().map(configQueryEntry -> {
        StorageLocation storage = new StorageLocation();
        storage.setValue(storageLocation.getValue());
        storage.setProperty(configQueryEntry.getPropertyName());
        storage.setConfigFile(configQueryEntry.getPropertyFile());
        return new StorageLocationView(storage);
    });
}
Also used : StorageLocation(com.sequenceiq.cloudbreak.domain.StorageLocation)

Example 13 with StorageLocation

use of com.sequenceiq.cloudbreak.domain.StorageLocation in project cloudbreak by hortonworks.

the class StackToTemplatePreparationObjectConverterTest method testConvertWhenEnvironmentBackupLocationDefinedThenBaseFileSystemConfigurationsViewShouldAddIt.

@Test
public void testConvertWhenEnvironmentBackupLocationDefinedThenBaseFileSystemConfigurationsViewShouldAddIt() throws IOException {
    String backupLocation = "s3a://test";
    FileSystem sourceFileSystem = new FileSystem();
    FileSystem clusterServiceFileSystem = new FileSystem();
    ConfigQueryEntries configQueryEntries = new ConfigQueryEntries();
    BaseFileSystemConfigurationsView expected = mock(BaseFileSystemConfigurationsView.class);
    List<StorageLocationView> storageLocationViews = mock(List.class);
    BackupResponse backupResponse = new BackupResponse();
    backupResponse.setStorageLocation(backupLocation);
    DetailedEnvironmentResponse environmentResponse = DetailedEnvironmentResponse.builder().withIdBrokerMappingSource(IdBrokerMappingSource.MOCK).withCredential(new CredentialResponse()).withAdminGroupName(ADMIN_GROUP_NAME).withCrn(TestConstants.CRN).withBackup(backupResponse).build();
    StorageLocation storageLocation = new StorageLocation();
    storageLocation.setValue(backupLocation);
    storageLocation.setProperty(RangerCloudStorageServiceConfigProvider.DEFAULT_BACKUP_DIR);
    StorageLocationView backupLocationView = new StorageLocationView(storageLocation);
    when(sourceCluster.getFileSystem()).thenReturn(sourceFileSystem);
    when(cluster.getFileSystem()).thenReturn(clusterServiceFileSystem);
    when(fileSystemConfigurationProvider.fileSystemConfiguration(eq(clusterServiceFileSystem), eq(stackMock), any(), eq(new Json("")), eq(configQueryEntries))).thenReturn(expected);
    when(cmCloudStorageConfigProvider.getConfigQueryEntries()).thenReturn(configQueryEntries);
    when(environmentClientService.getByCrn(anyString())).thenReturn(environmentResponse);
    when(blueprintViewProvider.getBlueprintView(any())).thenReturn(getBlueprintView());
    when(expected.getLocations()).thenReturn(storageLocationViews);
    TemplatePreparationObject result = underTest.convert(stackMock);
    assertThat(result.getFileSystemConfigurationView().isPresent()).isTrue();
    assertThat(result.getFileSystemConfigurationView().get()).isEqualTo(expected);
    verify(fileSystemConfigurationProvider, times(1)).fileSystemConfiguration(eq(clusterServiceFileSystem), eq(stackMock), any(), eq(new Json("")), eq(configQueryEntries));
    verify(expected, times(1)).getLocations();
    verify(storageLocationViews, times(1)).add(eq(backupLocationView));
}
Also used : TemplatePreparationObject(com.sequenceiq.cloudbreak.template.TemplatePreparationObject) StorageLocationView(com.sequenceiq.cloudbreak.template.filesystem.StorageLocationView) BaseFileSystemConfigurationsView(com.sequenceiq.cloudbreak.template.filesystem.BaseFileSystemConfigurationsView) FileSystem(com.sequenceiq.cloudbreak.domain.FileSystem) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) CredentialResponse(com.sequenceiq.environment.api.v1.credential.model.response.CredentialResponse) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Json(com.sequenceiq.cloudbreak.common.json.Json) BackupResponse(com.sequenceiq.common.api.backup.response.BackupResponse) StorageLocation(com.sequenceiq.cloudbreak.domain.StorageLocation) ConfigQueryEntries(com.sequenceiq.common.api.cloudstorage.query.ConfigQueryEntries) Test(org.junit.jupiter.api.Test)

Example 14 with StorageLocation

use of com.sequenceiq.cloudbreak.domain.StorageLocation in project cloudbreak by hortonworks.

the class HdfsConfigProviderTest method getStorageLocation.

protected StorageLocation getStorageLocation(String property, String value) {
    StorageLocation hmsWarehouseDir = new StorageLocation();
    hmsWarehouseDir.setProperty(property);
    hmsWarehouseDir.setValue(value);
    return hmsWarehouseDir;
}
Also used : StorageLocation(com.sequenceiq.cloudbreak.domain.StorageLocation)

Example 15 with StorageLocation

use of com.sequenceiq.cloudbreak.domain.StorageLocation in project cloudbreak by hortonworks.

the class HiveLLAPServiceConfigProviderTest method getTemplatePreparationObject.

private TemplatePreparationObject getTemplatePreparationObject(boolean includeLocations) {
    HostgroupView master = new HostgroupView("master", 1, InstanceGroupType.GATEWAY, 1);
    HostgroupView worker = new HostgroupView("worker", 2, InstanceGroupType.CORE, 2);
    List<StorageLocationView> locations = new ArrayList<>();
    if (includeLocations) {
        StorageLocation hmsExternalWarehouseDir = new StorageLocation();
        hmsExternalWarehouseDir.setProperty("hive.metastore.warehouse.external.dir");
        hmsExternalWarehouseDir.setValue("s3a://bucket/hive/warehouse/external");
        locations.add(new StorageLocationView(hmsExternalWarehouseDir));
    }
    S3FileSystemConfigurationsView fileSystemConfigurationsView = new S3FileSystemConfigurationsView(new S3FileSystem(), locations, false);
    return Builder.builder().withFileSystemConfigurationView(fileSystemConfigurationsView).withHostgroupViews(Set.of(master, worker)).build();
}
Also used : StorageLocationView(com.sequenceiq.cloudbreak.template.filesystem.StorageLocationView) ArrayList(java.util.ArrayList) HostgroupView(com.sequenceiq.cloudbreak.template.views.HostgroupView) StorageLocation(com.sequenceiq.cloudbreak.domain.StorageLocation) S3FileSystemConfigurationsView(com.sequenceiq.cloudbreak.template.filesystem.s3.S3FileSystemConfigurationsView) S3FileSystem(com.sequenceiq.common.api.filesystem.S3FileSystem)

Aggregations

StorageLocation (com.sequenceiq.cloudbreak.domain.StorageLocation)40 StorageLocationView (com.sequenceiq.cloudbreak.template.filesystem.StorageLocationView)16 ArrayList (java.util.ArrayList)13 S3FileSystemConfigurationsView (com.sequenceiq.cloudbreak.template.filesystem.s3.S3FileSystemConfigurationsView)11 S3FileSystem (com.sequenceiq.common.api.filesystem.S3FileSystem)11 HostgroupView (com.sequenceiq.cloudbreak.template.views.HostgroupView)7 Test (org.junit.Test)6 TemplatePreparationObject (com.sequenceiq.cloudbreak.template.TemplatePreparationObject)5 BaseFileSystemConfigurationsView (com.sequenceiq.cloudbreak.template.filesystem.BaseFileSystemConfigurationsView)5 CmTemplateProcessor (com.sequenceiq.cloudbreak.cmtemplate.CmTemplateProcessor)4 ApiClusterTemplate (com.cloudera.api.swagger.model.ApiClusterTemplate)2 BackupResponse (com.sequenceiq.common.api.backup.response.BackupResponse)2 HashSet (java.util.HashSet)2 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)2 ApiClusterTemplateConfig (com.cloudera.api.swagger.model.ApiClusterTemplateConfig)1 ClouderaManagerRepo (com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerRepo)1 CoreConfigProvider (com.sequenceiq.cloudbreak.cmtemplate.configproviders.core.CoreConfigProvider)1 HiveMetastoreConfigProvider (com.sequenceiq.cloudbreak.cmtemplate.configproviders.hive.HiveMetastoreConfigProvider)1 Json (com.sequenceiq.cloudbreak.common.json.Json)1 FileSystem (com.sequenceiq.cloudbreak.domain.FileSystem)1