Search in sources :

Example 1 with RegionAwareInternalCrnGenerator

use of com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator in project cloudbreak by hortonworks.

the class StackStatusCheckerJob method executeTracedJob.

@Override
protected void executeTracedJob(JobExecutionContext context) throws JobExecutionException {
    if (flowLogService.isOtherFlowRunning(getStackId())) {
        LOGGER.debug("StackStatusCheckerJob cannot run, because flow is running for stack: {}", getStackId());
        return;
    }
    try {
        measure(() -> {
            Stack stack = stackService.get(getStackId());
            Status stackStatus = stack.getStatus();
            if (Status.getUnschedulableStatuses().contains(stackStatus)) {
                LOGGER.debug("Stack sync will be unscheduled, stack state is {}", stackStatus);
                jobService.unschedule(getLocalId());
            } else if (shouldSwitchToLongSyncJob(stackStatus, context)) {
                LOGGER.debug("Stack sync will be scheduled to long polling, stack state is {}", stackStatus);
                jobService.unschedule(getLocalId());
                jobService.scheduleLongIntervalCheck(getStackId(), StackJobAdapter.class);
            } else if (null == stackStatus || ignoredStates().contains(stackStatus)) {
                LOGGER.debug("Stack sync is skipped, stack state is {}", stackStatus);
            } else if (syncableStates().contains(stackStatus)) {
                RegionAwareInternalCrnGenerator dataHub = regionAwareInternalCrnGeneratorFactory.datahub();
                ThreadBasedUserCrnProvider.doAs(dataHub.getInternalCrnForServiceAsString(), () -> doSync(stack));
                switchToShortSyncIfNecessary(context);
            } else {
                LOGGER.warn("Unhandled stack status, {}", stackStatus);
            }
        }, LOGGER, "Check status took {} ms for stack {}.", getStackId());
    } catch (Exception e) {
        LOGGER.info("Exception during cluster state check.", e);
    }
}
Also used : DetailedStackStatus(com.sequenceiq.cloudbreak.api.endpoint.v4.common.DetailedStackStatus) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) InstanceStatus(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.InstanceStatus) Status(com.sequenceiq.cloudbreak.api.endpoint.v4.common.Status) RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) JobExecutionException(org.quartz.JobExecutionException) Stack(com.sequenceiq.cloudbreak.domain.stack.Stack)

Example 2 with RegionAwareInternalCrnGenerator

use of com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator in project cloudbreak by hortonworks.

the class InternalCrnBuilderTest method generateCrnAsStringWhenFreeIpaIs.

@Test
public void generateCrnAsStringWhenFreeIpaIs() {
    RegionAwareInternalCrnGenerator regionAwareInternalCrnGenerator = regionalAwareInternalCrnGenerator(Crn.Service.FREEIPA, "cdp", "us-west-1");
    assertEquals("crn:cdp:freeipa:us-west-1:altus:user:__internal__actor__", regionAwareInternalCrnGenerator.getInternalCrnForServiceAsString());
}
Also used : RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) Test(org.junit.Test)

Example 3 with RegionAwareInternalCrnGenerator

use of com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator in project cloudbreak by hortonworks.

the class FreeIpaServiceTest method upgradeCcmFailureTest.

@Test
void upgradeCcmFailureTest() {
    when(freeIpaV1Endpoint.upgradeCcmInternal(ENVCRN, USERCRN)).thenThrow(new WebApplicationException("Houston..."));
    RegionAwareInternalCrnGenerator iamGenerator = mock(RegionAwareInternalCrnGenerator.class);
    when(regionAwareInternalCrnGeneratorFactory.iam()).thenReturn(iamGenerator);
    when(webApplicationExceptionMessageExtractor.getErrorMessage(any())).thenReturn("custom error");
    when(iamGenerator.getInternalCrnForServiceAsString()).thenReturn(USERCRN);
    assertThatThrownBy(() -> ThreadBasedUserCrnProvider.doAs(USERCRN, () -> underTest.upgradeCcm(ENVCRN))).hasMessage("custom error").isExactlyInstanceOf(FreeIpaOperationFailedException.class);
}
Also used : WebApplicationException(javax.ws.rs.WebApplicationException) RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) Test(org.junit.jupiter.api.Test)

Example 4 with RegionAwareInternalCrnGenerator

use of com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator in project cloudbreak by hortonworks.

the class CloudStorageManifesterTest method whenEnvironmentHasLoggingEnabledThenShouldApplyAsLogIdentityForGCS.

@Test
public void whenEnvironmentHasLoggingEnabledThenShouldApplyAsLogIdentityForGCS() {
    when(regionAwareInternalCrnGenerator.getInternalCrnForServiceAsString()).thenReturn("crn");
    when(regionAwareInternalCrnGeneratorFactory.iam()).thenReturn(regionAwareInternalCrnGenerator);
    mockFileSystemResponseForCloudbreakClient();
    SdxCluster sdxCluster = new SdxCluster();
    SdxClusterRequest sdxClusterRequest = new SdxClusterRequest();
    sdxCluster.setInitiatorUserCrn(USER_CRN);
    sdxCluster.setClusterName("sdx-cluster");
    SdxCloudStorageRequest cloudStorageRequest = new SdxCloudStorageRequest();
    cloudStorageRequest.setBaseLocation("gs://example-path");
    cloudStorageRequest.setFileSystemType(FileSystemType.GCS);
    GcsCloudStorageV1Parameters gcsCloudStorageV1Parameters = new GcsCloudStorageV1Parameters();
    gcsCloudStorageV1Parameters.setServiceAccountEmail(EMAIL);
    cloudStorageRequest.setGcs(gcsCloudStorageV1Parameters);
    sdxClusterRequest.setCloudStorage(cloudStorageRequest);
    DetailedEnvironmentResponse environment = new DetailedEnvironmentResponse();
    environment.setCloudPlatform("GCP");
    TelemetryResponse telemetryResponse = new TelemetryResponse();
    LoggingResponse loggingResponse = new LoggingResponse();
    loggingResponse.setGcs(gcsCloudStorageV1Parameters);
    telemetryResponse.setLogging(loggingResponse);
    GcpEnvironmentParameters gcpEnvironmentParameters = GcpEnvironmentParameters.builder().build();
    environment.setGcp(gcpEnvironmentParameters);
    environment.setTelemetry(telemetryResponse);
    ClusterV4Request clusterV4Request = new ClusterV4Request();
    clusterV4Request.setBlueprintName(exampleBlueprintName);
    CloudStorageRequest cloudStorageConfigReq = ThreadBasedUserCrnProvider.doAs(USER_CRN, () -> underTest.initCloudStorageRequest(environment, clusterV4Request, sdxCluster, sdxClusterRequest));
    StorageLocationBase singleRequest = cloudStorageConfigReq.getLocations().iterator().next();
    assertEquals(2, cloudStorageConfigReq.getIdentities().size());
    assertEquals(1, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getType().equals(CloudIdentityType.ID_BROKER)).collect(Collectors.toSet()).size());
    assertEquals(1, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getType().equals(CloudIdentityType.LOG)).collect(Collectors.toSet()).size());
    assertEquals(2, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getGcs().getServiceAccountEmail().equals(EMAIL)).collect(Collectors.toSet()).size());
    assertEquals(1, cloudStorageConfigReq.getLocations().size());
    assertEquals(CloudStorageCdpService.RANGER_AUDIT, singleRequest.getType());
    assertEquals("ranger/example-path", singleRequest.getValue());
    verify(storageValidationService).validateCloudStorage("GCP", cloudStorageRequest);
}
Also used : SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) GcpEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.gcp.GcpEnvironmentParameters) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) Mock(org.mockito.Mock) FileSystemParameterV4Responses(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Responses) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) ArgumentMatchers.anyBoolean(org.mockito.ArgumentMatchers.anyBoolean) AwsEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.AwsEnvironmentParameters) ArrayList(java.util.ArrayList) ThreadBasedUserCrnProvider(com.sequenceiq.cloudbreak.auth.ThreadBasedUserCrnProvider) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) FileSystemParameterV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Response) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) InjectMocks(org.mockito.InjectMocks) RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) MockitoExtension(org.mockito.junit.jupiter.MockitoExtension) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) Mockito.when(org.mockito.Mockito.when) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) FileSystemV4Endpoint(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.FileSystemV4Endpoint) RegionAwareInternalCrnGeneratorFactory(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGeneratorFactory) Test(org.junit.jupiter.api.Test) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) List(java.util.List) S3GuardRequestParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.S3GuardRequestParameters) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) FileSystemType(com.sequenceiq.common.model.FileSystemType) CloudIdentityType(com.sequenceiq.common.model.CloudIdentityType) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) GcpEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.gcp.GcpEnvironmentParameters) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) Test(org.junit.jupiter.api.Test)

Example 5 with RegionAwareInternalCrnGenerator

use of com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator in project cloudbreak by hortonworks.

the class CloudStorageManifesterTest method whenEnvironmentHasLoggingEnabledThenShouldApplyAsLogIdentity.

@Test
public void whenEnvironmentHasLoggingEnabledThenShouldApplyAsLogIdentity() {
    when(regionAwareInternalCrnGenerator.getInternalCrnForServiceAsString()).thenReturn("crn");
    when(regionAwareInternalCrnGeneratorFactory.iam()).thenReturn(regionAwareInternalCrnGenerator);
    mockFileSystemResponseForCloudbreakClient();
    SdxCluster sdxCluster = new SdxCluster();
    SdxClusterRequest sdxClusterRequest = new SdxClusterRequest();
    sdxCluster.setInitiatorUserCrn(USER_CRN);
    sdxCluster.setClusterName("sdx-cluster");
    SdxCloudStorageRequest cloudStorageRequest = new SdxCloudStorageRequest();
    cloudStorageRequest.setBaseLocation("s3a://example-path");
    cloudStorageRequest.setFileSystemType(FileSystemType.S3);
    S3CloudStorageV1Parameters s3Params = new S3CloudStorageV1Parameters();
    s3Params.setInstanceProfile("instance:profile");
    cloudStorageRequest.setS3(s3Params);
    sdxClusterRequest.setCloudStorage(cloudStorageRequest);
    DetailedEnvironmentResponse environment = new DetailedEnvironmentResponse();
    environment.setCloudPlatform("AWS");
    TelemetryResponse telemetryResponse = new TelemetryResponse();
    LoggingResponse loggingResponse = new LoggingResponse();
    S3CloudStorageV1Parameters s3CloudStorageV1Parameters = new S3CloudStorageV1Parameters();
    s3CloudStorageV1Parameters.setInstanceProfile("logprofile");
    loggingResponse.setS3(s3CloudStorageV1Parameters);
    telemetryResponse.setLogging(loggingResponse);
    AwsEnvironmentParameters awsEnvironmentParameters = new AwsEnvironmentParameters();
    S3GuardRequestParameters s3GuardRequestParameters = new S3GuardRequestParameters();
    s3GuardRequestParameters.setDynamoDbTableName("table");
    awsEnvironmentParameters.setS3guard(s3GuardRequestParameters);
    environment.setAws(awsEnvironmentParameters);
    environment.setTelemetry(telemetryResponse);
    ClusterV4Request clusterV4Request = new ClusterV4Request();
    clusterV4Request.setBlueprintName(exampleBlueprintName);
    CloudStorageRequest cloudStorageConfigReq = ThreadBasedUserCrnProvider.doAs(USER_CRN, () -> underTest.initCloudStorageRequest(environment, clusterV4Request, sdxCluster, sdxClusterRequest));
    StorageLocationBase singleRequest = cloudStorageConfigReq.getLocations().iterator().next();
    assertEquals(2, cloudStorageConfigReq.getIdentities().size());
    assertEquals(1, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getType().equals(CloudIdentityType.ID_BROKER)).collect(Collectors.toSet()).size());
    assertEquals(1, cloudStorageConfigReq.getIdentities().stream().filter(r -> r.getType().equals(CloudIdentityType.LOG)).collect(Collectors.toSet()).size());
    assertEquals("table", cloudStorageConfigReq.getAws().getS3Guard().getDynamoTableName());
    assertEquals(1, cloudStorageConfigReq.getLocations().size());
    assertEquals(CloudStorageCdpService.RANGER_AUDIT, singleRequest.getType());
    assertEquals("ranger/example-path", singleRequest.getValue());
    verify(storageValidationService).validateCloudStorage("AWS", cloudStorageRequest);
}
Also used : SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) GcpEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.gcp.GcpEnvironmentParameters) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) Mock(org.mockito.Mock) FileSystemParameterV4Responses(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Responses) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) GcsCloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters) ArgumentMatchers.anyBoolean(org.mockito.ArgumentMatchers.anyBoolean) AwsEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.AwsEnvironmentParameters) ArrayList(java.util.ArrayList) ThreadBasedUserCrnProvider(com.sequenceiq.cloudbreak.auth.ThreadBasedUserCrnProvider) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) FileSystemParameterV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Response) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) TelemetryResponse(com.sequenceiq.common.api.telemetry.response.TelemetryResponse) InjectMocks(org.mockito.InjectMocks) RegionAwareInternalCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) MockitoExtension(org.mockito.junit.jupiter.MockitoExtension) CloudStorageCdpService(com.sequenceiq.common.model.CloudStorageCdpService) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) Mockito.when(org.mockito.Mockito.when) Collectors(java.util.stream.Collectors) Mockito.verify(org.mockito.Mockito.verify) FileSystemV4Endpoint(com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.FileSystemV4Endpoint) RegionAwareInternalCrnGeneratorFactory(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGeneratorFactory) Test(org.junit.jupiter.api.Test) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) List(java.util.List) S3GuardRequestParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.S3GuardRequestParameters) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) FileSystemType(com.sequenceiq.common.model.FileSystemType) CloudIdentityType(com.sequenceiq.common.model.CloudIdentityType) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) S3CloudStorageV1Parameters(com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) StorageLocationBase(com.sequenceiq.common.api.cloudstorage.StorageLocationBase) ClusterV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request) S3GuardRequestParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.S3GuardRequestParameters) LoggingResponse(com.sequenceiq.common.api.telemetry.response.LoggingResponse) AwsEnvironmentParameters(com.sequenceiq.environment.api.v1.environment.model.request.aws.AwsEnvironmentParameters) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) Test(org.junit.jupiter.api.Test)

Aggregations

RegionAwareInternalCrnGenerator (com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGenerator)9 Test (org.junit.jupiter.api.Test)6 RegionAwareInternalCrnGeneratorFactory (com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGeneratorFactory)4 ClusterV4Request (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.cluster.ClusterV4Request)3 ThreadBasedUserCrnProvider (com.sequenceiq.cloudbreak.auth.ThreadBasedUserCrnProvider)3 Status (com.sequenceiq.cloudbreak.api.endpoint.v4.common.Status)2 FileSystemV4Endpoint (com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.FileSystemV4Endpoint)2 FileSystemParameterV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Response)2 FileSystemParameterV4Responses (com.sequenceiq.cloudbreak.api.endpoint.v4.filesystems.responses.FileSystemParameterV4Responses)2 CloudStorageRequest (com.sequenceiq.common.api.cloudstorage.CloudStorageRequest)2 StorageLocationBase (com.sequenceiq.common.api.cloudstorage.StorageLocationBase)2 GcsCloudStorageV1Parameters (com.sequenceiq.common.api.cloudstorage.old.GcsCloudStorageV1Parameters)2 S3CloudStorageV1Parameters (com.sequenceiq.common.api.cloudstorage.old.S3CloudStorageV1Parameters)2 LoggingResponse (com.sequenceiq.common.api.telemetry.response.LoggingResponse)2 TelemetryResponse (com.sequenceiq.common.api.telemetry.response.TelemetryResponse)2 CloudIdentityType (com.sequenceiq.common.model.CloudIdentityType)2 CloudStorageCdpService (com.sequenceiq.common.model.CloudStorageCdpService)2 FileSystemType (com.sequenceiq.common.model.FileSystemType)2 SdxCluster (com.sequenceiq.datalake.entity.SdxCluster)2 AwsEnvironmentParameters (com.sequenceiq.environment.api.v1.environment.model.request.aws.AwsEnvironmentParameters)2