Search in sources :

Example 1 with LoggingResponse

use of com.sequenceiq.common.api.telemetry.response.LoggingResponse in project cloudbreak by hortonworks.

the class TelemetryConverterTest method testConvertFromEnvAndSdxResponseWithoutWAInput.

@Test
public void testConvertFromEnvAndSdxResponseWithoutWAInput() {
    // GIVEN
    TelemetryResponse response = new TelemetryResponse();
    LoggingResponse loggingResponse = new LoggingResponse();
    S3CloudStorageV1Parameters s3Params = new S3CloudStorageV1Parameters();
    s3Params.setInstanceProfile(INSTANCE_PROFILE_VALUE);
    loggingResponse.setS3(s3Params);
    response.setLogging(loggingResponse);
    SdxClusterResponse sdxClusterResponse = new SdxClusterResponse();
    sdxClusterResponse.setCrn("crn:cdp:cloudbreak:us-west-1:someone:sdxcluster:sdxId");
    sdxClusterResponse.setName("sdxName");
    // WHEN
    TelemetryRequest result = underTest.convert(response, sdxClusterResponse);
    // THEN
    assertTrue(result.getFeatures().getWorkloadAnalytics().isEnabled());
    assertEquals(INSTANCE_PROFILE_VALUE, result.getLogging().getS3().getInstanceProfile());
    assertEquals("sdxId", result.getWorkloadAnalytics().getAttributes().get("databus.header.sdx.id").toString());
    assertEquals("sdxName", result.getWorkloadAnalytics().getAttributes().get("databus.header.sdx.name").toString());
}
Also used : TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) TelemetryRequest(com.sequenceiq.common.api.telemetry.request.TelemetryRequest) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) SdxClusterResponse(com.sequenceiq.sdx.api.model.SdxClusterResponse) Test(org.junit.Test)

Example 2 with LoggingResponse

use of com.sequenceiq.common.api.telemetry.response.LoggingResponse in project cloudbreak by hortonworks.

the class TelemetryApiConverter method createLoggingResponseFromSource.

private LoggingResponse createLoggingResponseFromSource(EnvironmentLogging logging) {
    LoggingResponse loggingResponse = null;
    if (logging != null) {
        loggingResponse = new LoggingResponse();
        loggingResponse.setStorageLocation(logging.getStorageLocation());
        loggingResponse.setS3(convertS3(logging.getS3()));
        loggingResponse.setAdlsGen2(convertAdlsV2(logging.getAdlsGen2()));
        loggingResponse.setGcs(convertGcs(logging.getGcs()));
        loggingResponse.setCloudwatch(CloudwatchParams.copy(logging.getCloudwatch()));
    }
    return loggingResponse;
}
Also used : LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse)

Example 3 with LoggingResponse

use of com.sequenceiq.common.api.telemetry.response.LoggingResponse in project cloudbreak by hortonworks.

the class CloudStorageDecoratorTest method testConvertWhenCloudStorageLocationsIsNullAndEnvironmentHasTelemetryWithLogging.

@Test
void testConvertWhenCloudStorageLocationsIsNullAndEnvironmentHasTelemetryWithLogging() {
    DetailedEnvironmentResponse environment = new DetailedEnvironmentResponse();
    TelemetryResponse telemetry = new TelemetryResponse();
    telemetry.setLogging(new LoggingResponse());
    environment.setTelemetry(telemetry);
    CloudStorageRequest result = underTest.decorate(BLUEPRINT_NAME, CLUSTER_NAME, null, environment);
    assertNotNull(result);
    assertTrue(result.getIdentities().stream().anyMatch(id -> CloudIdentityType.LOG.equals(id.getType())));
}
Also used : TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) ArgumentMatchers.eq(org.mockito.ArgumentMatchers.eq) Mock(org.mockito.Mock) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) ArrayList(java.util.ArrayList) SdxClusterResponse(com.sequenceiq.sdx.api.model.SdxClusterResponse) Pair(org.apache.commons.lang3.tuple.Pair) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) FileSystemConfigQueryObject(com.sequenceiq.cloudbreak.template.filesystem.FileSystemConfigQueryObject) Blueprint(com.sequenceiq.cloudbreak.domain.Blueprint) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) LinkedHashSet(java.util.LinkedHashSet) InjectMocks(org.mockito.InjectMocks) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) MockitoExtension(org.mockito.junit.jupiter.MockitoExtension) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) Set(java.util.Set) Mockito.when(org.mockito.Mockito.when) Test(org.junit.jupiter.api.Test) List(java.util.List) CmCloudStorageConfigProvider(com.sequenceiq.cloudbreak.cmtemplate.cloudstorage.CmCloudStorageConfigProvider) SdxClientService(com.sequenceiq.cloudbreak.service.datalake.SdxClientService) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) FileSystemType(com.sequenceiq.common.model.FileSystemType) ConfigQueryEntry(com.sequenceiq.common.api.cloudstorage.query.ConfigQueryEntry) BlueprintService(com.sequenceiq.cloudbreak.service.blueprint.BlueprintService) CloudIdentityType(com.sequenceiq.common.model.CloudIdentityType) Mockito.mock(org.mockito.Mockito.mock) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) Test(org.junit.jupiter.api.Test)

Example 4 with LoggingResponse

use of com.sequenceiq.common.api.telemetry.response.LoggingResponse in project cloudbreak by hortonworks.

the class CloudStorageDecorator method decorate.

public CloudStorageRequest decorate(String blueprintName, String clusterName, CloudStorageRequest request, DetailedEnvironmentResponse environment) {
    if (environment != null) {
        if (request == null) {
            request = new CloudStorageRequest();
        }
        TelemetryResponse telemetry = environment.getTelemetry();
        if (telemetry != null && telemetry.getLogging() != null) {
            LoggingResponse logging = telemetry.getLogging();
            StorageIdentityBase identity = new StorageIdentityBase();
            identity.setType(CloudIdentityType.LOG);
            identity.setS3(logging.getS3());
            identity.setAdlsGen2(logging.getAdlsGen2());
            identity.setGcs(logging.getGcs());
            List<StorageIdentityBase> identities = request.getIdentities();
            if (identities == null) {
                identities = new ArrayList<>();
            }
            boolean logConfiguredInRequest = false;
            for (StorageIdentityBase identityBase : identities) {
                if (CloudIdentityType.LOG.equals(identityBase.getType())) {
                    logConfiguredInRequest = true;
                }
            }
            if (!logConfiguredInRequest) {
                identities.add(identity);
            }
            request.setIdentities(identities);
        }
        List<SdxClusterResponse> datalakes = sdxClientService.getByEnvironmentCrn(environment.getCrn());
        updateCloudStorageLocations(blueprintName, clusterName, request, datalakes);
        updateDynamoDBTable(request, environment);
    }
    return request;
}
Also used : TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) SdxClusterResponse(com.sequenceiq.sdx.api.model.SdxClusterResponse) StorageIdentityBase(com.sequenceiq.common.api.cloudstorage.StorageIdentityBase)

Example 5 with LoggingResponse

use of com.sequenceiq.common.api.telemetry.response.LoggingResponse in project cloudbreak by hortonworks.

the class CloudStorageManifesterTest method whenEnvironmentHasOnlyLoggingEnabledThenShouldApplyAsLogIdentityForGCS.

@Test
public void whenEnvironmentHasOnlyLoggingEnabledThenShouldApplyAsLogIdentityForGCS() {
    DetailedEnvironmentResponse environment = new DetailedEnvironmentResponse();
    environment.setCloudPlatform("GCP");
    TelemetryResponse telemetryResponse = new TelemetryResponse();
    LoggingResponse loggingResponse = new LoggingResponse();
    GcsCloudStorageV1Parameters gcsCloudStorageV1Parameters = new GcsCloudStorageV1Parameters();
    gcsCloudStorageV1Parameters.setServiceAccountEmail(EMAIL);
    loggingResponse.setGcs(gcsCloudStorageV1Parameters);
    telemetryResponse.setLogging(loggingResponse);
    environment.setTelemetry(telemetryResponse);
    ClusterV4Request clusterV4Request = new ClusterV4Request();
    clusterV4Request.setBlueprintName(exampleBlueprintName);
    CloudStorageRequest cloudStorageConfigReq = underTest.initCloudStorageRequest(environment, clusterV4Request, null, new SdxClusterRequest());
    assertEquals(1, cloudStorageConfigReq.getIdentities().size());
    assertEquals(1, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getType().equals(CloudIdentityType.LOG)).collect(Collectors.toSet()).size());
    assertEquals(EMAIL, cloudStorageConfigReq.getIdentities().get(0).getGcs().getServiceAccountEmail());
}
Also used : TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) GcpEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.gcp.GcpEnvironmentParameters) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) Mock(org.mockito.Mock) FileSystemParameterV4Responses(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Responses) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) ArgumentMatchers.anyBoolean(org.mockito.ArgumentMatchers.anyBoolean) AwsEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.AwsEnvironmentParameters) ArrayList(java.util.ArrayList) ThreadBasedUserCrnProvider(com.sequenceiq.cloudbreak.auth.ThreadBasedUserCrnProvider) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) FileSystemParameterV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Response) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) InjectMocks(org.mockito.InjectMocks) RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) MockitoExtension(org.mockito.junit.jupiter.MockitoExtension) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) Mockito.when(org.mockito.Mockito.when) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) FileSystemV4Endpoint(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.FileSystemV4Endpoint) RegionAwareInternalCrnGeneratorFactory(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGeneratorFactory) Test(org.junit.jupiter.api.Test) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) List(java.util.List) S3GuardRequestParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.S3GuardRequestParameters) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) FileSystemType(com.sequenceiq.common.model.FileSystemType) CloudIdentityType(com.sequenceiq.common.model.CloudIdentityType) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) Test(org.junit.jupiter.api.Test)

Aggregations

LoggingResponse (com.sequenceiq.common.api.telemetry.response.LoggingResponse)22 TelemetryResponse (com.sequenceiq.common.api.telemetry.response.TelemetryResponse)17 DetailedEnvironmentResponse (com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse)12 CloudStorageRequest (com.sequenceiq.common.api.cloudstorage.CloudStorageRequest)10 Test (org.junit.jupiter.api.Test)10 S3CloudStorageV1Parameters (com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters)9 StorageLocationBase (com.sequenceiq.common.api.cloudstorage.StorageLocationBase)8 CloudIdentityType (com.sequenceiq.common.model.CloudIdentityType)7 CloudStorageCdpService (com.sequenceiq.common.model.CloudStorageCdpService)7 FileSystemType (com.sequenceiq.common.model.FileSystemType)7 ArrayList (java.util.ArrayList)7 List (java.util.List)7 Assertions.assertEquals (org.junit.jupiter.api.Assertions.assertEquals)7 Assertions.assertNull (org.junit.jupiter.api.Assertions.assertNull)7 ExtendWith (org.junit.jupiter.api.extension.ExtendWith)7 InjectMocks (org.mockito.InjectMocks)7 Mock (org.mockito.Mock)7 Mockito.when (org.mockito.Mockito.when)7 MockitoExtension (org.mockito.junit.jupiter.MockitoExtension)7 ClusterV4Request (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request)5