Search in sources :

Example 1 with StorageLocationBase

use of com.sequenceiq.common.api.cloudstorage.StorageLocationBase in project cloudbreak by hortonworks.

the class AwsIDBrokerMappedRolePermissionValidator method collectPolicies.

List<Policy> collectPolicies(CloudS3View cloudFileSystem, List<String> policyFileNames) {
    List<Policy> policies = new ArrayList<>();
    for (StorageLocationBase location : cloudFileSystem.getLocations()) {
        if (checkLocation(location)) {
            Map<String, String> replacements = getPolicyJsonReplacements(location, cloudFileSystem);
            List<Policy> policiesWithReplacements = getPolicies(policyFileNames, replacements);
            policies.addAll(policiesWithReplacements);
        }
    }
    return policies;
}
Also used : Policy(com.amazonaws.auth.policy.Policy) ArrayList(java.util.ArrayList) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase)

Example 2 with StorageLocationBase

use of com.sequenceiq.common.api.cloudstorage.StorageLocationBase in project cloudbreak by hortonworks.

the class CloudStorageValidationUtilTest method testIsCloudStorageConfiguredWhenS3NotNull.

@Test
public void testIsCloudStorageConfiguredWhenS3NotNull() {
    CloudStorageRequest cloudStorageRequest = new CloudStorageRequest();
    StorageIdentityBase storageIdentityBase = new StorageIdentityBase();
    storageIdentityBase.setS3(new S3CloudStorageV1Parameters());
    cloudStorageRequest.setIdentities(List.of(storageIdentityBase));
    cloudStorageRequest.setLocations(List.of(new StorageLocationBase()));
    boolean actual = underTest.isCloudStorageConfigured(cloudStorageRequest);
    Assert.assertTrue(actual);
}
Also used : CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) StorageIdentityBase(com.sequenceiq.common.api.cloudstorage.StorageIdentityBase) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) Test(org.junit.Test)

Example 3 with StorageLocationBase

use of com.sequenceiq.common.api.cloudstorage.StorageLocationBase in project cloudbreak by hortonworks.

the class CloudStorageDecoratorTest method testUpdateCloudStorageLocationsWhenRequestContainsTemplatedLocationThenItShouldBeReplaced.

@Test
void testUpdateCloudStorageLocationsWhenRequestContainsTemplatedLocationThenItShouldBeReplaced() {
    CloudStorageCdpService storageLocationType = CloudStorageCdpService.DEFAULT_FS;
    String eStorageLocationValue = "s3a://some-dir/some-other-dir/{{{clusterName}}}";
    StorageLocationBase storageLocationBase = new StorageLocationBase();
    storageLocationBase.setType(storageLocationType);
    storageLocationBase.setValue(eStorageLocationValue);
    List<StorageLocationBase> storageLocations = new ArrayList<>(1);
    storageLocations.add(storageLocationBase);
    CloudStorageRequest request = new CloudStorageRequest();
    request.setLocations(storageLocations);
    SdxClusterResponse sdxResponse = new SdxClusterResponse();
    String storageLocationValue = "s3a://some-dir/some-other-dir/" + CLUSTER_NAME;
    sdxResponse.setCloudStorageBaseLocation(storageLocationValue);
    sdxResponse.setCloudStorageFileSystemType(FileSystemType.S3);
    ConfigQueryEntry cqe = new ConfigQueryEntry();
    cqe.setType(CloudStorageCdpService.DEFAULT_FS);
    cqe.setDefaultPath(storageLocationValue);
    Set<ConfigQueryEntry> cqes = new LinkedHashSet<>(1);
    cqes.add(cqe);
    when(blueprintService.queryFileSystemParameters(BLUEPRINT_NAME, CLUSTER_NAME, storageLocationValue, FileSystemType.S3.name(), "", true, false, 0L)).thenReturn(cqes);
    Pair<Blueprint, String> mockBt = mock(Pair.class);
    when(blueprintService.getBlueprintAndText(BLUEPRINT_NAME, 0L)).thenReturn(mockBt);
    FileSystemConfigQueryObject mockfscqo = mock(FileSystemConfigQueryObject.class);
    when(blueprintService.createFileSystemConfigQueryObject(mockBt, CLUSTER_NAME, sdxResponse.getCloudStorageBaseLocation(), sdxResponse.getCloudStorageFileSystemType().name(), "", true, false)).thenReturn(mockfscqo);
    when(cmCloudStorageConfigProvider.queryParameters(any(), eq(mockfscqo))).thenReturn(cqes);
    CloudStorageRequest result = underTest.updateCloudStorageLocations(BLUEPRINT_NAME, CLUSTER_NAME, request, List.of(sdxResponse));
    assertNotNull(result);
    assertEquals(0, result.getLocations().stream().filter(slb -> slb.getValue().contains("{{{") && slb.getValue().contains("}}}")).count());
}
Also used : LinkedHashSet(java.util.LinkedHashSet) ConfigQueryEntry(com.sequenceiq.common.api.cloudstorage.query.ConfigQueryEntry) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) Blueprint(com.sequenceiq.cloudbreak.domain.Blueprint) ArrayList(java.util.ArrayList) FileSystemConfigQueryObject(com.sequenceiq.cloudbreak.template.filesystem.FileSystemConfigQueryObject) SdxClusterResponse(com.sequenceiq.sdx.api.model.SdxClusterResponse) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) Test(org.junit.jupiter.api.Test)

Example 4 with StorageLocationBase

use of com.sequenceiq.common.api.cloudstorage.StorageLocationBase in project cloudbreak by hortonworks.

the class CloudStorageDecorator method filterConfigsWithTemplatePlaceholder.

private Set<ConfigQueryEntry> filterConfigsWithTemplatePlaceholder(CloudStorageRequest request, Set<ConfigQueryEntry> recommendations) {
    Set<ConfigQueryEntry> filtered = new LinkedHashSet<>();
    for (StorageLocationBase location : request.getLocations()) {
        recommendations.stream().filter(configQueryEntry -> configQueryEntry.getType().equals(location.getType())).findFirst().ifPresent(configQueryEntry -> {
            ConfigQueryEntry custom = configQueryEntry.copy();
            custom.setDefaultPath(location.getValue());
            filtered.add(custom);
        });
    }
    return filtered;
}
Also used : LinkedHashSet(java.util.LinkedHashSet) ConfigQueryEntry(com.sequenceiq.common.api.cloudstorage.query.ConfigQueryEntry) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase)

Example 5 with StorageLocationBase

use of com.sequenceiq.common.api.cloudstorage.StorageLocationBase in project cloudbreak by hortonworks.

the class CloudStorageManifesterTest method whenCloudStorageEnabledFromInternalRequest.

@Test
public void whenCloudStorageEnabledFromInternalRequest() {
    ClusterV4Request clusterV4Request = new ClusterV4Request();
    clusterV4Request.setBlueprintName(exampleBlueprintName);
    CloudStorageRequest cloudStorageRequest = new CloudStorageRequest();
    StorageLocationBase storageLocationBase = new StorageLocationBase();
    storageLocationBase.setType(CloudStorageCdpService.RANGER_AUDIT);
    storageLocationBase.setValue("s3a://ranger-audit");
    cloudStorageRequest.setLocations(List.of(storageLocationBase));
    clusterV4Request.setCloudStorage(cloudStorageRequest);
    CloudStorageRequest cloudStorageConfigReq = underTest.initCloudStorageRequest(new DetailedEnvironmentResponse(), clusterV4Request, null, new SdxClusterRequest());
    assertEquals(CloudStorageCdpService.RANGER_AUDIT, cloudStorageConfigReq.getLocations().get(0).getType());
}
Also used : ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) Test(org.junit.jupiter.api.Test)

Aggregations

StorageLocationBase (com.sequenceiq.common.api.cloudstorage.StorageLocationBase)34 CloudStorageRequest (com.sequenceiq.common.api.cloudstorage.CloudStorageRequest)19 Test (org.junit.jupiter.api.Test)19 CloudStorageCdpService (com.sequenceiq.common.model.CloudStorageCdpService)10 StorageIdentityBase (com.sequenceiq.common.api.cloudstorage.StorageIdentityBase)9 DetailedEnvironmentResponse (com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse)9 ArrayList (java.util.ArrayList)9 CloudS3View (com.sequenceiq.cloudbreak.cloud.model.filesystem.CloudS3View)7 LoggingResponse (com.sequenceiq.common.api.telemetry.response.LoggingResponse)7 FileSystemType (com.sequenceiq.common.model.FileSystemType)7 List (java.util.List)7 ConfigQueryEntry (com.sequenceiq.common.api.cloudstorage.query.ConfigQueryEntry)6 TelemetryResponse (com.sequenceiq.common.api.telemetry.response.TelemetryResponse)6 CloudIdentityType (com.sequenceiq.common.model.CloudIdentityType)6 LinkedHashSet (java.util.LinkedHashSet)6 ClusterV4Request (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request)5 S3CloudStorageV1Parameters (com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters)5 SdxCloudStorageRequest (com.sequenceiq.sdx.api.model.SdxCloudStorageRequest)5 SdxClusterRequest (com.sequenceiq.sdx.api.model.SdxClusterRequest)5 Assertions.assertEquals (org.junit.jupiter.api.Assertions.assertEquals)5