Search in sources :

Example 1 with UseSpotInstances

use of com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances in project cloudbreak by hortonworks.

the class NewNetworkWithNoInternetEnvironmentTests method testCreateNewEnvironmentWithNewNetworkAndNoInternet.

@Test(dataProvider = TEST_CONTEXT)
@UseSpotInstances
@Description(given = "there is a running cloudbreak", when = "create an Environment with new network which CIDR is 10.0.0.0/16, outbound internet traffic is disabled with FreeIPA and SDX", then = "should create new Subnets and the number depends on the provider and should create instances without internet access.")
public void testCreateNewEnvironmentWithNewNetworkAndNoInternet(TestContext testContext) {
    String networkKey = "someOtherNetwork";
    String sdx = resourcePropertyProvider().getName();
    SdxDatabaseRequest database = new SdxDatabaseRequest();
    database.setCreate(false);
    testContext.given(CredentialTestDto.class).when(credentialTestClient.create()).given("telemetry", TelemetryTestDto.class).withLogging().withReportClusterLogs().given(EnvironmentTestDto.class).given(networkKey, EnvironmentNetworkTestDto.class).withNetworkCIDR("10.0.0.0/16").withPrivateSubnets().withNoOutboundInternetTraffic().withServiceEndpoints().given(EnvironmentTestDto.class).withNetwork(networkKey).withTelemetry("telemetry").withCreateFreeIpa(Boolean.TRUE).when(environmentTestClient.create()).await(EnvironmentStatus.AVAILABLE).then((tc, testDto, cc) -> environmentTestClient.describe().action(tc, testDto, cc)).then(EnvironmentNetworkTestAssertion.environmentContainsNeccessaryConfigs()).init(FreeIpaTestDto.class).when(freeIpaTestClient.describe()).then((tc, testDto, client) -> sshJClientActions.checkNoOutboundInternetTraffic(testDto, client)).given(sdx, SdxTestDto.class).withExternalDatabase(database).withCloudStorage().when(sdxTestClient.create(), RunningParameter.key(sdx)).await(SdxClusterStatusResponse.RUNNING).then((tc, testDto, client) -> sshJClientActions.checkNoOutboundInternetTraffic(testDto, getInstanceGroups(testDto, client), List.of(HostGroupType.MASTER.getName()))).validate();
}
Also used : TelemetryTestDto(com.sequenceiq.it.cloudbreak.dto.telemetry.TelemetryTestDto) CredentialTestDto(com.sequenceiq.it.cloudbreak.dto.credential.CredentialTestDto) FreeIpaTestDto(com.sequenceiq.it.cloudbreak.dto.freeipa.FreeIpaTestDto) Test(org.testng.annotations.Test) SdxDatabaseRequest(com.sequenceiq.sdx.api.model.SdxDatabaseRequest) Description(com.sequenceiq.it.cloudbreak.context.Description) EnvironmentNetworkTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentNetworkTestDto) Inject(javax.inject.Inject) CredentialTestClient(com.sequenceiq.it.cloudbreak.client.CredentialTestClient) SdxTestDto(com.sequenceiq.it.cloudbreak.dto.sdx.SdxTestDto) TestContext(com.sequenceiq.it.cloudbreak.context.TestContext) InstanceGroupV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response) SdxTestClient(com.sequenceiq.it.cloudbreak.client.SdxTestClient) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances) FreeIpaTestClient(com.sequenceiq.it.cloudbreak.client.FreeIpaTestClient) List(java.util.List) EnvironmentTestClient(com.sequenceiq.it.cloudbreak.client.EnvironmentTestClient) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) RunningParameter(com.sequenceiq.it.cloudbreak.context.RunningParameter) EnvironmentStatus(com.sequenceiq.environment.api.v1.environment.model.response.EnvironmentStatus) SdxClient(com.sequenceiq.it.cloudbreak.SdxClient) SdxClusterStatusResponse(com.sequenceiq.sdx.api.model.SdxClusterStatusResponse) EnvironmentNetworkTestAssertion(com.sequenceiq.it.cloudbreak.assertion.environment.EnvironmentNetworkTestAssertion) Collections(java.util.Collections) SshJClientActions(com.sequenceiq.it.cloudbreak.util.ssh.action.SshJClientActions) HostGroupType(com.sequenceiq.it.cloudbreak.cloud.HostGroupType) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) SdxDatabaseRequest(com.sequenceiq.sdx.api.model.SdxDatabaseRequest) FreeIpaTestDto(com.sequenceiq.it.cloudbreak.dto.freeipa.FreeIpaTestDto) Description(com.sequenceiq.it.cloudbreak.context.Description) Test(org.testng.annotations.Test) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)

Example 2 with UseSpotInstances

use of com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances in project cloudbreak by hortonworks.

the class AwsYcloudHybridCloudTest method testCreateSdxOnChildEnvironment.

@Test(dataProvider = TEST_CONTEXT)
@UseSpotInstances
@Description(given = "there is a running cloudbreak with parent-child environments ", when = "a valid SDX create request is sent to the child environment ", then = "SDX is created and instances are accessible via ssh by valid username and password ", and = "instances are not accessible via ssh by invalid username and password")
public void testCreateSdxOnChildEnvironment(TestContext testContext) {
    String sdxInternal = resourcePropertyProvider().getName(CHILD_CLOUD_PLATFORM);
    String clouderaManager = resourcePropertyProvider().getName(CHILD_CLOUD_PLATFORM);
    String cluster = resourcePropertyProvider().getName(CHILD_CLOUD_PLATFORM);
    String cmProduct = resourcePropertyProvider().getName(CHILD_CLOUD_PLATFORM);
    String stack = resourcePropertyProvider().getName(CHILD_CLOUD_PLATFORM);
    AtomicReference<String> cdhVersion = new AtomicReference<>();
    AtomicReference<String> cdhParcel = new AtomicReference<>();
    String runtimeVersion = commonClusterManagerProperties().getRuntimeVersion();
    testContext.given(StackMatrixTestDto.class, CHILD_CLOUD_PLATFORM).when(utilTestClient.stackMatrixV4()).then((tc, dto, client) -> {
        ClouderaManagerStackDescriptorV4Response response = dto.getResponse().getCdh().get(runtimeVersion);
        cdhVersion.set(response.getVersion());
        cdhParcel.set(response.getRepository().getStack().get(REDHAT7));
        return dto;
    }).validate();
    testContext.given("telemetry", TelemetryTestDto.class).withLogging(CHILD_CLOUD_PLATFORM).withReportClusterLogs().given(cmProduct, ClouderaManagerProductTestDto.class, CHILD_CLOUD_PLATFORM).withName(CDH).withVersion(cdhVersion.get()).withParcel(cdhParcel.get()).given(clouderaManager, ClouderaManagerTestDto.class, CHILD_CLOUD_PLATFORM).withClouderaManagerProduct(cmProduct).given(cluster, ClusterTestDto.class, CHILD_CLOUD_PLATFORM).withBlueprintName(getDefaultSDXBlueprintName()).withValidateBlueprint(Boolean.FALSE).withClouderaManager(clouderaManager).given(MASTER_INSTANCE_GROUP, InstanceGroupTestDto.class, CHILD_CLOUD_PLATFORM).withHostGroup(MASTER).withNodeCount(1).given(IDBROKER_INSTANCE_GROUP, InstanceGroupTestDto.class, CHILD_CLOUD_PLATFORM).withHostGroup(IDBROKER).withNodeCount(1).given(STACK_AUTHENTICATION, StackAuthenticationTestDto.class, CHILD_CLOUD_PLATFORM).given(stack, StackTestDto.class, CHILD_CLOUD_PLATFORM).withCluster(cluster).withInstanceGroups(MASTER_INSTANCE_GROUP, IDBROKER_INSTANCE_GROUP).withStackAuthentication(STACK_AUTHENTICATION).withTelemetry("telemetry").given(sdxInternal, SdxInternalTestDto.class, CHILD_CLOUD_PLATFORM).withStackRequest(key(cluster), key(stack)).withEnvironmentKey(RunningParameter.key(CHILD_ENVIRONMENT_KEY)).when(sdxTestClient.createInternal(), key(sdxInternal)).await(SdxClusterStatusResponse.RUNNING).awaitForHealthyInstances().then((tc, dto, client) -> {
        String environmentCrn = dto.getResponse().getEnvironmentCrn();
        com.sequenceiq.freeipa.api.client.FreeIpaClient freeIpaClient = tc.getMicroserviceClient(FreeIpaClient.class).getDefaultClient();
        checkUserSyncState(environmentCrn, freeIpaClient);
        String username = testContext.getActingUserCrn().getResource();
        String sanitizedUserName = SanitizerUtil.sanitizeWorkloadUsername(username);
        for (InstanceGroupV4Response ig : dto.getResponse().getStackV4Response().getInstanceGroups()) {
            for (InstanceMetaDataV4Response i : ig.getMetadata()) {
                String ip = i.getPublicIp();
                LOGGER.info("Trying to ssh with user {} into instance: {}", sanitizedUserName, OBJECT_MAPPER.writeValueAsString(i));
                testShhAuthenticationSuccessful(sanitizedUserName, ip);
                testShhAuthenticationFailure(sanitizedUserName, ip);
            }
        }
        return dto;
    }).given(CHILD_ENVIRONMENT_KEY, EnvironmentTestDto.class, CHILD_CLOUD_PLATFORM).when(environmentTestClient.cascadingDelete(), RunningParameter.key(CHILD_ENVIRONMENT_KEY)).await(EnvironmentStatus.ARCHIVED, RunningParameter.key(CHILD_ENVIRONMENT_KEY)).validate();
}
Also used : ClusterTestDto(com.sequenceiq.it.cloudbreak.dto.ClusterTestDto) LoggerFactory(org.slf4j.LoggerFactory) RunningParameter.key(com.sequenceiq.it.cloudbreak.context.RunningParameter.key) Test(org.testng.annotations.Test) Description(com.sequenceiq.it.cloudbreak.context.Description) EnvironmentNetworkTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentNetworkTestDto) InstanceMetaDataV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.instancemetadata.InstanceMetaDataV4Response) StackMatrixTestDto(com.sequenceiq.it.cloudbreak.dto.util.StackMatrixTestDto) UserV1Endpoint(com.sequenceiq.freeipa.api.v1.freeipa.user.UserV1Endpoint) InstanceGroupTestDto(com.sequenceiq.it.cloudbreak.dto.InstanceGroupTestDto) Map(java.util.Map) StackTestDto(com.sequenceiq.it.cloudbreak.dto.stack.StackTestDto) FreeIpaClient(com.sequenceiq.it.cloudbreak.FreeIpaClient) ClouderaManagerTestDto(com.sequenceiq.it.cloudbreak.dto.ClouderaManagerTestDto) InstanceStatus(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.InstanceStatus) InstanceGroupV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances) SanitizerUtil(com.sequenceiq.cloudbreak.util.SanitizerUtil) SSHClient(net.schmizz.sshj.SSHClient) UtilTestClient(com.sequenceiq.it.cloudbreak.client.UtilTestClient) EnvironmentStatus(com.sequenceiq.environment.api.v1.environment.model.response.EnvironmentStatus) ClouderaManagerProductTestDto(com.sequenceiq.it.cloudbreak.dto.ClouderaManagerProductTestDto) BlueprintTestDto(com.sequenceiq.it.cloudbreak.dto.blueprint.BlueprintTestDto) SdxClusterStatusResponse(com.sequenceiq.sdx.api.model.SdxClusterStatusResponse) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) TelemetryTestDto(com.sequenceiq.it.cloudbreak.dto.telemetry.TelemetryTestDto) CredentialTestDto(com.sequenceiq.it.cloudbreak.dto.credential.CredentialTestDto) EnvironmentUserSyncState(com.sequenceiq.freeipa.api.v1.freeipa.user.model.EnvironmentUserSyncState) HashMap(java.util.HashMap) PromiscuousVerifier(net.schmizz.sshj.transport.verification.PromiscuousVerifier) AtomicReference(java.util.concurrent.atomic.AtomicReference) CloudPlatform(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform) Inject(javax.inject.Inject) Value(org.springframework.beans.factory.annotation.Value) SdxInternalTestDto(com.sequenceiq.it.cloudbreak.dto.sdx.SdxInternalTestDto) StackAuthenticationTestDto(com.sequenceiq.it.cloudbreak.dto.StackAuthenticationTestDto) UserAuthException(net.schmizz.sshj.userauth.UserAuthException) TestFailException(com.sequenceiq.it.cloudbreak.exception.TestFailException) SyncOperationStatus(com.sequenceiq.freeipa.api.v1.freeipa.user.model.SyncOperationStatus) CredentialTestClient(com.sequenceiq.it.cloudbreak.client.CredentialTestClient) BlueprintTestClient(com.sequenceiq.it.cloudbreak.client.BlueprintTestClient) Logger(org.slf4j.Logger) TestContext(com.sequenceiq.it.cloudbreak.context.TestContext) MASTER(com.sequenceiq.it.cloudbreak.cloud.HostGroupType.MASTER) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) IDBROKER(com.sequenceiq.it.cloudbreak.cloud.HostGroupType.IDBROKER) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) SdxTestClient(com.sequenceiq.it.cloudbreak.client.SdxTestClient) ClouderaManagerStackDescriptorV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.util.responses.ClouderaManagerStackDescriptorV4Response) EnvironmentTestClient(com.sequenceiq.it.cloudbreak.client.EnvironmentTestClient) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) RunningParameter(com.sequenceiq.it.cloudbreak.context.RunningParameter) HostGroupType(com.sequenceiq.it.cloudbreak.cloud.HostGroupType) FreeIpaClient(com.sequenceiq.it.cloudbreak.FreeIpaClient) AtomicReference(java.util.concurrent.atomic.AtomicReference) ClouderaManagerStackDescriptorV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.util.responses.ClouderaManagerStackDescriptorV4Response) InstanceGroupTestDto(com.sequenceiq.it.cloudbreak.dto.InstanceGroupTestDto) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) StackTestDto(com.sequenceiq.it.cloudbreak.dto.stack.StackTestDto) InstanceMetaDataV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.instancemetadata.InstanceMetaDataV4Response) StackMatrixTestDto(com.sequenceiq.it.cloudbreak.dto.util.StackMatrixTestDto) TelemetryTestDto(com.sequenceiq.it.cloudbreak.dto.telemetry.TelemetryTestDto) InstanceGroupV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response) Description(com.sequenceiq.it.cloudbreak.context.Description) Test(org.testng.annotations.Test) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)

Example 3 with UseSpotInstances

use of com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances in project cloudbreak by hortonworks.

the class DistroXRepairTests method testEphemeralDistroXMasterRepairWithTerminatedEC2Instances.

@Test(dataProvider = TEST_CONTEXT)
@UseSpotInstances
@Description(given = "there is a running Cloudbreak, and an environment with SDX and DistroX cluster in available state", when = "recovery called on the MASTER host group of DistroX cluster, where the EC2 instance had been terminated", then = "DistroX recovery should be successful, the cluster should be up and running")
public void testEphemeralDistroXMasterRepairWithTerminatedEC2Instances(TestContext testContext) {
    String distrox = resourcePropertyProvider().getName();
    List<String> actualVolumeIds = new ArrayList<>();
    List<String> expectedVolumeIds = new ArrayList<>();
    String username = testContext.getActingUserCrn().getResource();
    String sanitizedUserName = SanitizerUtil.sanitizeWorkloadUsername(username);
    testContext.given(distrox, DistroXTestDto.class).withInstanceGroupsEntity(new DistroXInstanceGroupsBuilder(testContext).defaultHostGroup().withStorageOptimizedInstancetype().build()).when(distroXTestClient.create(), key(distrox)).await(STACK_AVAILABLE).awaitForHealthyInstances().then(this::verifyMountedDisks).then((tc, testDto, client) -> {
        CloudFunctionality cloudFunctionality = tc.getCloudProvider().getCloudFunctionality();
        List<String> instancesToDelete = distroxUtil.getInstanceIds(testDto, client, MASTER.getName());
        expectedVolumeIds.addAll(cloudFunctionality.listInstanceVolumeIds(testDto.getName(), instancesToDelete));
        cloudFunctionality.deleteInstances(testDto.getName(), instancesToDelete);
        return testDto;
    }).awaitForHostGroup(MASTER.getName(), InstanceStatus.DELETED_ON_PROVIDER_SIDE).when(distroXTestClient.repair(MASTER), key(distrox)).await(STACK_AVAILABLE, key(distrox)).awaitForHealthyInstances().then(this::verifyMountedDisks).then((tc, testDto, client) -> clouderaManagerUtil.checkClouderaManagerYarnNodemanagerRoleConfigGroups(testDto, sanitizedUserName, MOCK_UMS_PASSWORD)).then((tc, testDto, client) -> {
        CloudFunctionality cloudFunctionality = tc.getCloudProvider().getCloudFunctionality();
        List<String> instanceIds = distroxUtil.getInstanceIds(testDto, client, MASTER.getName());
        actualVolumeIds.addAll(cloudFunctionality.listInstanceVolumeIds(testDto.getName(), instanceIds));
        return testDto;
    }).then((tc, testDto, client) -> VolumeUtils.compareVolumeIdsAfterRepair(testDto, actualVolumeIds, expectedVolumeIds)).validate();
}
Also used : LoggerFactory(org.slf4j.LoggerFactory) RunningParameter.key(com.sequenceiq.it.cloudbreak.context.RunningParameter.key) Test(org.testng.annotations.Test) CloudPlatform(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform) ArrayList(java.util.ArrayList) Description(com.sequenceiq.it.cloudbreak.context.Description) Inject(javax.inject.Inject) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) Logger(org.slf4j.Logger) TestContext(com.sequenceiq.it.cloudbreak.context.TestContext) ClouderaManagerUtil(com.sequenceiq.it.cloudbreak.util.clouderamanager.ClouderaManagerUtil) MASTER(com.sequenceiq.it.cloudbreak.cloud.HostGroupType.MASTER) InstanceStatus(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.InstanceStatus) InstanceGroupV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances) DistroXTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto) SanitizerUtil(com.sequenceiq.cloudbreak.util.SanitizerUtil) List(java.util.List) DistroXTestClient(com.sequenceiq.it.cloudbreak.client.DistroXTestClient) DistroxUtil(com.sequenceiq.it.cloudbreak.util.DistroxUtil) CloudFunctionality(com.sequenceiq.it.cloudbreak.util.CloudFunctionality) CloudbreakClient(com.sequenceiq.it.cloudbreak.CloudbreakClient) VolumeUtils(com.sequenceiq.it.cloudbreak.util.VolumeUtils) HostGroupType(com.sequenceiq.it.cloudbreak.cloud.HostGroupType) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) DistroXTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto) CloudFunctionality(com.sequenceiq.it.cloudbreak.util.CloudFunctionality) ArrayList(java.util.ArrayList) ArrayList(java.util.ArrayList) List(java.util.List) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) Description(com.sequenceiq.it.cloudbreak.context.Description) Test(org.testng.annotations.Test) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)

Example 4 with UseSpotInstances

use of com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances in project cloudbreak by hortonworks.

the class DistroXStopStartTest method testCreateDistroXWithEphemeralTemporaryStorage.

@Test(dataProvider = TEST_CONTEXT)
@UseSpotInstances
@Description(given = "there is a running cloudbreak", when = "two valid DistroX create requests are sent one with ephemeral temporary storage and one without it", then = "clusters are created, device mount point are checked, and after stopping and starting the clusters ephemeral store handling checked again")
public void testCreateDistroXWithEphemeralTemporaryStorage(TestContext testContext) {
    String username = testContext.getActingUserCrn().getResource();
    String sanitizedUserName = SanitizerUtil.sanitizeWorkloadUsername(username);
    testContext.given("non_eph_dx", DistroXTestDto.class).withInstanceGroupsEntity(new DistroXInstanceGroupsBuilder(testContext).defaultHostGroup().build()).when(distroXTestClient.create(), RunningParameter.key("non_eph_dx")).given("eph_dx", DistroXTestDto.class).withInstanceGroupsEntity(new DistroXInstanceGroupsBuilder(testContext).defaultHostGroup().withStorageOptimizedInstancetype().build()).when(distroXTestClient.create(), RunningParameter.key("eph_dx")).given("non_eph_dx", DistroXTestDto.class).await(STACK_AVAILABLE, RunningParameter.key("non_eph_dx")).then((tc, testDto, client) -> verifyEphemeralVolumesShouldNotBeConfiguredInHdfs(tc, sanitizedUserName, testDto)).when(distroXTestClient.stop(), RunningParameter.key("non_eph_dx")).await(STACK_STOPPED, RunningParameter.key("non_eph_dx")).when(distroXTestClient.start(), RunningParameter.key("non_eph_dx")).await(STACK_AVAILABLE, RunningParameter.key("non_eph_dx")).then((tc, testDto, client) -> verifyEphemeralVolumesShouldNotBeConfiguredInHdfs(tc, sanitizedUserName, testDto)).given("eph_dx", DistroXTestDto.class).await(STACK_AVAILABLE, RunningParameter.key("eph_dx")).then(this::verifyMountedDisks).then((tc, testDto, client) -> verifyEphemeralVolumesShouldNotBeConfiguredInHdfs(tc, sanitizedUserName, testDto)).then((tc, testDto, client) -> clouderaManagerUtil.checkClouderaManagerYarnNodemanagerRoleConfigGroups(testDto, sanitizedUserName, MOCK_UMS_PASSWORD)).when(distroXTestClient.stop(), RunningParameter.key("eph_dx")).await(STACK_STOPPED, RunningParameter.key("eph_dx")).when(distroXTestClient.start(), RunningParameter.key("eph_dx")).await(STACK_AVAILABLE, RunningParameter.key("eph_dx")).then(this::verifyMountedDisks).then((tc, testDto, client) -> clouderaManagerUtil.checkClouderaManagerYarnNodemanagerRoleConfigGroups(testDto, sanitizedUserName, MOCK_UMS_PASSWORD)).validate();
}
Also used : Logger(org.slf4j.Logger) TestContext(com.sequenceiq.it.cloudbreak.context.TestContext) ClouderaManagerUtil(com.sequenceiq.it.cloudbreak.util.clouderamanager.ClouderaManagerUtil) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) Test(org.testng.annotations.Test) InstanceGroupV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances) TestParameter(com.sequenceiq.it.TestParameter) DistroXTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto) Description(com.sequenceiq.it.cloudbreak.context.Description) Inject(javax.inject.Inject) SanitizerUtil(com.sequenceiq.cloudbreak.util.SanitizerUtil) List(java.util.List) DistroXTestClient(com.sequenceiq.it.cloudbreak.client.DistroXTestClient) RunningParameter(com.sequenceiq.it.cloudbreak.context.RunningParameter) CloudFunctionality(com.sequenceiq.it.cloudbreak.util.CloudFunctionality) CloudbreakClient(com.sequenceiq.it.cloudbreak.CloudbreakClient) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) HostGroupType(com.sequenceiq.it.cloudbreak.cloud.HostGroupType) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) DistroXTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) Description(com.sequenceiq.it.cloudbreak.context.Description) Test(org.testng.annotations.Test) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)

Example 5 with UseSpotInstances

use of com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances in project cloudbreak by hortonworks.

the class DistroXEncryptedVolumeTest method testCreateDistroXWithEncryptedVolumesInSingleRG.

@Test(dataProvider = TEST_CONTEXT, groups = { "azure_singlerg" })
@UseSpotInstances
@Description(given = "there is a valid Azure credential and a new resource group", when = "environment with freeIpa should be created in the resource group with encrypted volume", and = "SDX then distroX also with encrypted volumes should be created for environment", then = "freeIpa, sdx and distroX volumes should be encrypted with the provided key")
public void testCreateDistroXWithEncryptedVolumesInSingleRG(TestContext testContext) {
    DistroXDatabaseRequest distroXDatabaseRequest = new DistroXDatabaseRequest();
    List<DistroXInstanceGroupTestDto> distroXInstanceGroupTestDtos = new DistroXInstanceGroupsBuilder(testContext).defaultHostGroup().withDiskEncryption().build();
    distroXDatabaseRequest.setAvailabilityType(DistroXDatabaseAvailabilityType.NON_HA);
    createResourceGroupForEnvironment(testContext);
    testContext.given(EnvironmentNetworkTestDto.class).withServiceEndpoints().given("telemetry", TelemetryTestDto.class).withLogging().withReportClusterLogs().given(EnvironmentTestDto.class).withNetwork().withResourceGroup(ResourceGroupTest.AZURE_RESOURCE_GROUP_USAGE_SINGLE, resourceGroupForTest).withResourceEncryption().withTelemetry("telemetry").withTunnel(Tunnel.CLUSTER_PROXY).withCreateFreeIpa(Boolean.FALSE).when(environmentTestClient.create()).then((context, dto, client) -> {
        context.getCloudProviderAssertion().assertServiceEndpoint(dto);
        return dto;
    }).await(EnvironmentStatus.AVAILABLE).given(FreeIpaTestDto.class).withEnvironment().withTelemetry("telemetry").withCatalog(commonCloudProperties().getImageValidation().getFreeIpaImageCatalog(), commonCloudProperties().getImageValidation().getFreeIpaImageUuid()).when(freeIpaTestClient.create()).await(Status.AVAILABLE).awaitForHealthyInstances().given(FreeIpaUserSyncTestDto.class).when(freeIpaTestClient.getLastSyncOperationStatus()).await(OperationState.COMPLETED).given(EnvironmentTestDto.class).when(environmentTestClient.describe()).then(this::verifyEnvironmentResponseDiskEncryptionKey).given(FreeIpaTestDto.class).then(this::verifyFreeIpaVolumeEncryptionKey).given(SdxTestDto.class).withCloudStorage().when(sdxTestClient.create()).await(SdxClusterStatusResponse.RUNNING).awaitForHealthyInstances().then(this::verifySdxVolumeEncryptionKey).given(DistroXTestDto.class).withInstanceGroupsEntity(distroXInstanceGroupTestDtos).withExternalDatabase(distroXDatabaseRequest).when(distroXTestClient.create()).await(STACK_AVAILABLE).awaitForHealthyInstances().then(this::verifyDistroxVolumeEncryptionKey).then(this::verifyAwsEnaDriver).then(new AwsAvailabilityZoneAssertion()).then(validateTemplateContainsExternalDatabaseHostname()).validate();
}
Also used : DistroXInstanceGroupTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupTestDto) AwsCloudProvider(com.sequenceiq.it.cloudbreak.cloud.v4.aws.AwsCloudProvider) FreeIpaTestDto(com.sequenceiq.it.cloudbreak.dto.freeipa.FreeIpaTestDto) LoggerFactory(org.slf4j.LoggerFactory) Test(org.testng.annotations.Test) AfterMethod(org.testng.annotations.AfterMethod) Description(com.sequenceiq.it.cloudbreak.context.Description) EnvironmentNetworkTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentNetworkTestDto) Map(java.util.Map) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) SdxUtil(com.sequenceiq.it.cloudbreak.util.SdxUtil) Tunnel(com.sequenceiq.common.api.type.Tunnel) Status(com.sequenceiq.freeipa.api.v1.freeipa.stack.model.common.Status) FreeIpaClient(com.sequenceiq.it.cloudbreak.FreeIpaClient) SdxTestDto(com.sequenceiq.it.cloudbreak.dto.sdx.SdxTestDto) OperationState(com.sequenceiq.freeipa.api.v1.operation.model.OperationState) GcpCloudProvider(com.sequenceiq.it.cloudbreak.cloud.v4.gcp.GcpCloudProvider) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances) List(java.util.List) DistroXTestClient(com.sequenceiq.it.cloudbreak.client.DistroXTestClient) DistroXExternalDatabaseAssertion.validateTemplateContainsExternalDatabaseHostname(com.sequenceiq.it.cloudbreak.assertion.distrox.DistroXExternalDatabaseAssertion.validateTemplateContainsExternalDatabaseHostname) EnvironmentClient(com.sequenceiq.it.cloudbreak.EnvironmentClient) EnvironmentStatus(com.sequenceiq.environment.api.v1.environment.model.response.EnvironmentStatus) SdxClusterStatusResponse(com.sequenceiq.sdx.api.model.SdxClusterStatusResponse) CloudbreakClient(com.sequenceiq.it.cloudbreak.CloudbreakClient) FreeIpaUserSyncTestDto(com.sequenceiq.it.cloudbreak.dto.freeipa.FreeIpaUserSyncTestDto) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) TelemetryTestDto(com.sequenceiq.it.cloudbreak.dto.telemetry.TelemetryTestDto) DistroXDatabaseAvailabilityType(com.sequenceiq.distrox.api.v1.distrox.model.database.DistroXDatabaseAvailabilityType) AzureCloudFunctionality(com.sequenceiq.it.cloudbreak.util.azure.AzureCloudFunctionality) HashMap(java.util.HashMap) AzureCloudProvider(com.sequenceiq.it.cloudbreak.cloud.v4.azure.AzureCloudProvider) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) ResourceGroup(com.microsoft.azure.management.resources.ResourceGroup) ResourceGroupTest(com.sequenceiq.it.cloudbreak.ResourceGroupTest) DistroXInstanceGroupTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupTestDto) AwsAvailabilityZoneAssertion(com.sequenceiq.it.cloudbreak.assertion.distrox.AwsAvailabilityZoneAssertion) Logger(org.slf4j.Logger) TestContext(com.sequenceiq.it.cloudbreak.context.TestContext) MASTER(com.sequenceiq.it.cloudbreak.cloud.HostGroupType.MASTER) SdxTestClient(com.sequenceiq.it.cloudbreak.client.SdxTestClient) SshEnaDriverCheckActions(com.sequenceiq.it.cloudbreak.util.ssh.action.SshEnaDriverCheckActions) FreeIpaInstanceUtil(com.sequenceiq.it.cloudbreak.util.FreeIpaInstanceUtil) DistroXTestDto(com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto) FreeIpaTestClient(com.sequenceiq.it.cloudbreak.client.FreeIpaTestClient) EnvironmentTestClient(com.sequenceiq.it.cloudbreak.client.EnvironmentTestClient) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) DistroxUtil(com.sequenceiq.it.cloudbreak.util.DistroxUtil) SdxClient(com.sequenceiq.it.cloudbreak.SdxClient) DistroXDatabaseRequest(com.sequenceiq.distrox.api.v1.distrox.model.database.DistroXDatabaseRequest) EnvironmentTestDto(com.sequenceiq.it.cloudbreak.dto.environment.EnvironmentTestDto) AwsAvailabilityZoneAssertion(com.sequenceiq.it.cloudbreak.assertion.distrox.AwsAvailabilityZoneAssertion) DistroXDatabaseRequest(com.sequenceiq.distrox.api.v1.distrox.model.database.DistroXDatabaseRequest) DistroXInstanceGroupsBuilder(com.sequenceiq.it.cloudbreak.dto.distrox.instancegroup.DistroXInstanceGroupsBuilder) FreeIpaTestDto(com.sequenceiq.it.cloudbreak.dto.freeipa.FreeIpaTestDto) Description(com.sequenceiq.it.cloudbreak.context.Description) Test(org.testng.annotations.Test) AbstractE2ETest(com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest) ResourceGroupTest(com.sequenceiq.it.cloudbreak.ResourceGroupTest) UseSpotInstances(com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)

Aggregations

Description (com.sequenceiq.it.cloudbreak.context.Description)18 UseSpotInstances (com.sequenceiq.it.cloudbreak.util.spot.UseSpotInstances)18 Test (org.testng.annotations.Test)18 TestContext (com.sequenceiq.it.cloudbreak.context.TestContext)15 Inject (javax.inject.Inject)15 RunningParameter.key (com.sequenceiq.it.cloudbreak.context.RunningParameter.key)11 AbstractE2ETest (com.sequenceiq.it.cloudbreak.testcase.e2e.AbstractE2ETest)11 SdxTestClient (com.sequenceiq.it.cloudbreak.client.SdxTestClient)10 SdxClusterStatusResponse (com.sequenceiq.sdx.api.model.SdxClusterStatusResponse)10 List (java.util.List)9 Logger (org.slf4j.Logger)9 LoggerFactory (org.slf4j.LoggerFactory)9 DistroXTestClient (com.sequenceiq.it.cloudbreak.client.DistroXTestClient)8 HostGroupType (com.sequenceiq.it.cloudbreak.cloud.HostGroupType)8 DistroXTestDto (com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto)8 SdxDatabaseRequest (com.sequenceiq.sdx.api.model.SdxDatabaseRequest)8 SdxTestDto (com.sequenceiq.it.cloudbreak.dto.sdx.SdxTestDto)7 TestFailException (com.sequenceiq.it.cloudbreak.exception.TestFailException)7 InstanceGroupV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response)6 MASTER (com.sequenceiq.it.cloudbreak.cloud.HostGroupType.MASTER)6