Search in sources :

Example 1 with Credentials

use of com.amazonaws.services.securitytoken.model.Credentials in project hadoop by apache.

the class ITestS3ATemporaryCredentials method testSTS.

/**
   * Test use of STS for requesting temporary credentials.
   *
   * The property test.sts.endpoint can be set to point this at different
   * STS endpoints. This test will use the AWS credentials (if provided) for
   * S3A tests to request temporary credentials, then attempt to use those
   * credentials instead.
   *
   * @throws IOException
   */
@Test
public void testSTS() throws IOException {
    Configuration conf = getContract().getConf();
    if (!conf.getBoolean(TEST_STS_ENABLED, true)) {
        skip("STS functional tests disabled");
    }
    S3xLoginHelper.Login login = S3AUtils.getAWSAccessKeys(URI.create("s3a://foobar"), conf);
    if (!login.hasLogin()) {
        skip("testSTS disabled because AWS credentials not configured");
    }
    AWSCredentialsProvider parentCredentials = new BasicAWSCredentialsProvider(login.getUser(), login.getPassword());
    String stsEndpoint = conf.getTrimmed(TEST_STS_ENDPOINT, "");
    AWSSecurityTokenServiceClient stsClient;
    stsClient = new AWSSecurityTokenServiceClient(parentCredentials);
    if (!stsEndpoint.isEmpty()) {
        LOG.debug("STS Endpoint ={}", stsEndpoint);
        stsClient.setEndpoint(stsEndpoint);
    }
    GetSessionTokenRequest sessionTokenRequest = new GetSessionTokenRequest();
    sessionTokenRequest.setDurationSeconds(900);
    GetSessionTokenResult sessionTokenResult;
    sessionTokenResult = stsClient.getSessionToken(sessionTokenRequest);
    Credentials sessionCreds = sessionTokenResult.getCredentials();
    String childAccessKey = sessionCreds.getAccessKeyId();
    conf.set(ACCESS_KEY, childAccessKey);
    String childSecretKey = sessionCreds.getSecretAccessKey();
    conf.set(SECRET_KEY, childSecretKey);
    String sessionToken = sessionCreds.getSessionToken();
    conf.set(SESSION_TOKEN, sessionToken);
    conf.set(AWS_CREDENTIALS_PROVIDER, PROVIDER_CLASS);
    try (S3AFileSystem fs = S3ATestUtils.createTestFileSystem(conf)) {
        createAndVerifyFile(fs, path("testSTS"), TEST_FILE_SIZE);
    }
    // now create an invalid set of credentials by changing the session
    // token
    conf.set(SESSION_TOKEN, "invalid-" + sessionToken);
    try (S3AFileSystem fs = S3ATestUtils.createTestFileSystem(conf)) {
        createAndVerifyFile(fs, path("testSTSInvalidToken"), TEST_FILE_SIZE);
        fail("Expected an access exception, but file access to " + fs.getUri() + " was allowed: " + fs);
    } catch (AWSS3IOException ex) {
        LOG.info("Expected Exception: {}", ex.toString());
        LOG.debug("Expected Exception: {}", ex, ex);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) GetSessionTokenResult(com.amazonaws.services.securitytoken.model.GetSessionTokenResult) AWSSecurityTokenServiceClient(com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient) S3xLoginHelper(org.apache.hadoop.fs.s3native.S3xLoginHelper) GetSessionTokenRequest(com.amazonaws.services.securitytoken.model.GetSessionTokenRequest) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider) Credentials(com.amazonaws.services.securitytoken.model.Credentials) AWSCredentials(com.amazonaws.auth.AWSCredentials) Test(org.junit.Test)

Example 2 with Credentials

use of com.amazonaws.services.securitytoken.model.Credentials in project SimianArmy by Netflix.

the class STSAssumeRoleSessionCredentialsProvider method startSession.

/**
     * Starts a new session by sending a request to the AWS Security Token
     * Service (STS) to assume a Role using the long lived AWS credentials. This
     * class then vends the short lived session credentials for the assumed Role
     * sent back from STS.
     */
private void startSession() {
    AssumeRoleResult assumeRoleResult = securityTokenService.assumeRole(new AssumeRoleRequest().withRoleArn(roleArn).withDurationSeconds(DEFAULT_DURATION_SECONDS).withRoleSessionName("SimianArmy"));
    Credentials stsCredentials = assumeRoleResult.getCredentials();
    sessionCredentials = new BasicSessionCredentials(stsCredentials.getAccessKeyId(), stsCredentials.getSecretAccessKey(), stsCredentials.getSessionToken());
    sessionCredentialsExpiration = stsCredentials.getExpiration();
}
Also used : AssumeRoleRequest(com.amazonaws.services.securitytoken.model.AssumeRoleRequest) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) AssumeRoleResult(com.amazonaws.services.securitytoken.model.AssumeRoleResult) AWSSessionCredentials(com.amazonaws.auth.AWSSessionCredentials) Credentials(com.amazonaws.services.securitytoken.model.Credentials) AWSCredentials(com.amazonaws.auth.AWSCredentials) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials)

Example 3 with Credentials

use of com.amazonaws.services.securitytoken.model.Credentials in project eureka by Netflix.

the class AwsAsgUtil method retrieveAutoScalingGroupCrossAccount.

private AutoScalingGroup retrieveAutoScalingGroupCrossAccount(String asgAccount, String asgName) {
    logger.debug("Getting cross account ASG for asgName: " + asgName + ", asgAccount: " + asgAccount);
    Credentials credentials = stsCredentials.get(asgAccount);
    if (credentials == null || credentials.getExpiration().getTime() < System.currentTimeMillis() + 1000) {
        stsCredentials.put(asgAccount, initializeStsSession(asgAccount));
        credentials = stsCredentials.get(asgAccount);
    }
    ClientConfiguration clientConfiguration = new ClientConfiguration().withConnectionTimeout(serverConfig.getASGQueryTimeoutMs());
    AmazonAutoScaling autoScalingClient = new AmazonAutoScalingClient(new BasicSessionCredentials(credentials.getAccessKeyId(), credentials.getSecretAccessKey(), credentials.getSessionToken()), clientConfiguration);
    String region = clientConfig.getRegion();
    if (!region.equals("us-east-1")) {
        autoScalingClient.setEndpoint("autoscaling." + region + ".amazonaws.com");
    }
    DescribeAutoScalingGroupsRequest request = new DescribeAutoScalingGroupsRequest().withAutoScalingGroupNames(asgName);
    DescribeAutoScalingGroupsResult result = autoScalingClient.describeAutoScalingGroups(request);
    List<AutoScalingGroup> asgs = result.getAutoScalingGroups();
    if (asgs.isEmpty()) {
        return null;
    } else {
        return asgs.get(0);
    }
}
Also used : DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) AmazonAutoScaling(com.amazonaws.services.autoscaling.AmazonAutoScaling) DescribeAutoScalingGroupsResult(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult) Credentials(com.amazonaws.services.securitytoken.model.Credentials) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) BasicAWSCredentials(com.amazonaws.auth.BasicAWSCredentials) ClientConfiguration(com.amazonaws.ClientConfiguration)

Example 4 with Credentials

use of com.amazonaws.services.securitytoken.model.Credentials in project ice by Netflix.

the class AwsUtils method downloadFileIfChangedSince.

public static boolean downloadFileIfChangedSince(String bucketName, String bucketFilePrefix, File file, long milles, String accountId, String assumeRole, String externalId) {
    AmazonS3Client s3Client = AwsUtils.s3Client;
    try {
        if (!StringUtils.isEmpty(accountId) && !StringUtils.isEmpty(assumeRole)) {
            Credentials assumedCredentials = getAssumedCredentials(accountId, assumeRole, externalId);
            s3Client = new AmazonS3Client(new BasicSessionCredentials(assumedCredentials.getAccessKeyId(), assumedCredentials.getSecretAccessKey(), assumedCredentials.getSessionToken()), clientConfig);
        }
        ObjectMetadata metadata = s3Client.getObjectMetadata(bucketName, bucketFilePrefix + file.getName());
        boolean download = !file.exists() || metadata.getLastModified().getTime() > milles;
        if (download) {
            return download(s3Client, bucketName, bucketFilePrefix + file.getName(), file);
        } else
            return download;
    } finally {
        if (s3Client != AwsUtils.s3Client)
            s3Client.shutdown();
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) Credentials(com.amazonaws.services.securitytoken.model.Credentials)

Example 5 with Credentials

use of com.amazonaws.services.securitytoken.model.Credentials in project ice by Netflix.

the class AwsUtils method listAllObjects.

/**
     * List all object summary with given prefix in the s3 bucket.
     * @param bucket
     * @param prefix
     * @return
     */
public static List<S3ObjectSummary> listAllObjects(String bucket, String prefix, String accountId, String assumeRole, String externalId) {
    AmazonS3Client s3Client = AwsUtils.s3Client;
    try {
        ListObjectsRequest request = new ListObjectsRequest().withBucketName(bucket).withPrefix(prefix);
        List<S3ObjectSummary> result = Lists.newLinkedList();
        if (!StringUtils.isEmpty(accountId) && !StringUtils.isEmpty(assumeRole)) {
            Credentials assumedCredentials = getAssumedCredentials(accountId, assumeRole, externalId);
            s3Client = new AmazonS3Client(new BasicSessionCredentials(assumedCredentials.getAccessKeyId(), assumedCredentials.getSecretAccessKey(), assumedCredentials.getSessionToken()), clientConfig);
        }
        ObjectListing page = null;
        do {
            if (page != null)
                request.setMarker(page.getNextMarker());
            page = s3Client.listObjects(request);
            result.addAll(page.getObjectSummaries());
        } while (page.isTruncated());
        return result;
    } finally {
        if (s3Client != AwsUtils.s3Client)
            s3Client.shutdown();
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) BasicSessionCredentials(com.amazonaws.auth.BasicSessionCredentials) Credentials(com.amazonaws.services.securitytoken.model.Credentials)

Aggregations

Credentials (com.amazonaws.services.securitytoken.model.Credentials)6 BasicSessionCredentials (com.amazonaws.auth.BasicSessionCredentials)4 AWSCredentials (com.amazonaws.auth.AWSCredentials)2 AWSSessionCredentials (com.amazonaws.auth.AWSSessionCredentials)2 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)2 ClientConfiguration (com.amazonaws.ClientConfiguration)1 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)1 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)1 AmazonAutoScaling (com.amazonaws.services.autoscaling.AmazonAutoScaling)1 AmazonAutoScalingClient (com.amazonaws.services.autoscaling.AmazonAutoScalingClient)1 AutoScalingGroup (com.amazonaws.services.autoscaling.model.AutoScalingGroup)1 DescribeAutoScalingGroupsRequest (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest)1 DescribeAutoScalingGroupsResult (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult)1 AmazonEC2Client (com.amazonaws.services.ec2.AmazonEC2Client)1 DescribeReservedInstancesResult (com.amazonaws.services.ec2.model.DescribeReservedInstancesResult)1 ReservedInstances (com.amazonaws.services.ec2.model.ReservedInstances)1 AWSSecurityTokenServiceClient (com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient)1 AssumeRoleRequest (com.amazonaws.services.securitytoken.model.AssumeRoleRequest)1 AssumeRoleResult (com.amazonaws.services.securitytoken.model.AssumeRoleResult)1 GetSessionTokenRequest (com.amazonaws.services.securitytoken.model.GetSessionTokenRequest)1