Search in sources :

Example 71 with AWSStaticCredentialsProvider

use of com.amazonaws.auth.AWSStaticCredentialsProvider in project syndesis by syndesisio.

the class AWSS3VerifierExtension method verifyConnectivity.

// *********************************
// Connectivity validation
// *********************************
@Override
@SuppressWarnings("PMD.AvoidCatchingGenericException")
protected Result verifyConnectivity(Map<String, Object> parameters) {
    ResultBuilder builder = ResultBuilder.withStatusAndScope(Result.Status.OK, Scope.CONNECTIVITY);
    try {
        S3Configuration configuration = setProperties(new S3Configuration(), parameters);
        AWSCredentials credentials = new BasicAWSCredentials(configuration.getAccessKey(), configuration.getSecretKey());
        AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
        AmazonS3 client = AmazonS3ClientBuilder.standard().withCredentials(credentialsProvider).withRegion(Regions.valueOf(configuration.getRegion())).build();
        client.listBuckets();
    } catch (SdkClientException e) {
        ResultErrorBuilder errorBuilder = ResultErrorBuilder.withCodeAndDescription(VerificationError.StandardCode.AUTHENTICATION, e.getMessage()).detail("aws_s3_exception_message", e.getMessage()).detail(VerificationError.ExceptionAttribute.EXCEPTION_CLASS, e.getClass().getName()).detail(VerificationError.ExceptionAttribute.EXCEPTION_INSTANCE, e);
        builder.error(errorBuilder.build());
    } catch (@SuppressWarnings("PMD.AvoidCatchingGenericException") Exception e) {
        builder.error(ResultErrorBuilder.withException(e).build());
    }
    return builder.build();
}
Also used : ResultBuilder(org.apache.camel.component.extension.verifier.ResultBuilder) AmazonS3(com.amazonaws.services.s3.AmazonS3) AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) S3Configuration(org.apache.camel.component.aws.s3.S3Configuration) SdkClientException(com.amazonaws.SdkClientException) ResultErrorBuilder(org.apache.camel.component.extension.verifier.ResultErrorBuilder) BasicAWSCredentials(com.amazonaws.auth.BasicAWSCredentials) AWSCredentials(com.amazonaws.auth.AWSCredentials) BasicAWSCredentials(com.amazonaws.auth.BasicAWSCredentials) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider) SdkClientException(com.amazonaws.SdkClientException)

Example 72 with AWSStaticCredentialsProvider

use of com.amazonaws.auth.AWSStaticCredentialsProvider in project flink by apache.

the class PrestoS3FileSystemTest method validateBasicCredentials.

// ------------------------------------------------------------------------
// utilities
// ------------------------------------------------------------------------
private static void validateBasicCredentials(FileSystem fs) throws Exception {
    assertTrue(fs instanceof FlinkS3FileSystem);
    org.apache.hadoop.fs.FileSystem hadoopFs = ((FlinkS3FileSystem) fs).getHadoopFileSystem();
    assertTrue(hadoopFs instanceof PrestoS3FileSystem);
    try (PrestoS3FileSystem prestoFs = (PrestoS3FileSystem) hadoopFs) {
        AWSCredentialsProvider provider = getAwsCredentialsProvider(prestoFs);
        assertTrue(provider instanceof AWSStaticCredentialsProvider);
    }
}
Also used : AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) FlinkS3FileSystem(org.apache.flink.fs.s3.common.FlinkS3FileSystem) PrestoS3FileSystem(com.facebook.presto.hive.s3.PrestoS3FileSystem) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider)

Example 73 with AWSStaticCredentialsProvider

use of com.amazonaws.auth.AWSStaticCredentialsProvider in project alluxio by Alluxio.

the class S3AUnderFileSystemTest method createCredentialsFromConf.

@Test
public void createCredentialsFromConf() throws Exception {
    Map<PropertyKey, Object> conf = new HashMap<>();
    conf.put(PropertyKey.S3A_ACCESS_KEY, "key1");
    conf.put(PropertyKey.S3A_SECRET_KEY, "key2");
    try (Closeable c = new ConfigurationRule(conf, sConf).toResource()) {
        UnderFileSystemConfiguration ufsConf = UnderFileSystemConfiguration.defaults(sConf);
        AWSCredentialsProvider credentialsProvider = S3AUnderFileSystem.createAwsCredentialsProvider(ufsConf);
        Assert.assertEquals("key1", credentialsProvider.getCredentials().getAWSAccessKeyId());
        Assert.assertEquals("key2", credentialsProvider.getCredentials().getAWSSecretKey());
        Assert.assertTrue(credentialsProvider instanceof AWSStaticCredentialsProvider);
    }
}
Also used : AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) HashMap(java.util.HashMap) UnderFileSystemConfiguration(alluxio.underfs.UnderFileSystemConfiguration) Closeable(java.io.Closeable) ConfigurationRule(alluxio.ConfigurationRule) PropertyKey(alluxio.conf.PropertyKey) AWSCredentialsProvider(com.amazonaws.auth.AWSCredentialsProvider) Test(org.junit.Test)

Example 74 with AWSStaticCredentialsProvider

use of com.amazonaws.auth.AWSStaticCredentialsProvider in project crate by crate.

the class S3Service method buildCredentials.

// pkg private for tests
static AWSCredentialsProvider buildCredentials(Logger logger, S3ClientSettings clientSettings) {
    final AWSCredentials credentials = clientSettings.credentials;
    if (credentials == null) {
        logger.debug("Using instance profile credentials");
        var ec2ContainerCredentialsProviderWrapper = new EC2ContainerCredentialsProviderWrapper();
        try {
            // Check if credentials are available
            ec2ContainerCredentialsProviderWrapper.getCredentials();
            return ec2ContainerCredentialsProviderWrapper;
        } catch (SdkClientException e) {
            throw new InvalidArgumentException("Cannot find required credentials to create a repository of type s3. " + "Credentials must be provided either as repository options access_key and secret_key or AWS IAM roles.");
        }
    } else {
        logger.debug("Using basic key/secret credentials");
        return new AWSStaticCredentialsProvider(credentials);
    }
}
Also used : AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) InvalidArgumentException(io.crate.exceptions.InvalidArgumentException) SdkClientException(com.amazonaws.SdkClientException) AWSCredentials(com.amazonaws.auth.AWSCredentials) EC2ContainerCredentialsProviderWrapper(com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper)

Example 75 with AWSStaticCredentialsProvider

use of com.amazonaws.auth.AWSStaticCredentialsProvider in project TOSCAna by StuPro-TOSCAna.

the class CapabilityMapper method mapOsCapabilityToImageId.

/**
 *     Requests the AWS server for ImageIds with filters which are filled based on the values of the {@link OsCapability}.
 *     <br>
 *     The image with the latest creation date is picked and its imageId returned.
 *
 *     @param osCapability the OsCapability to map
 *     @return a {@link String} that contains a valid ImageId that can be added to the properties of an EC2
 */
public String mapOsCapabilityToImageId(OsCapability osCapability) throws SdkClientException, ParseException, IllegalArgumentException {
    AmazonEC2 ec2 = AmazonEC2ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(awsCredentials)).withRegion(awsRegion).build();
    // need to set these
    DescribeImagesRequest describeImagesRequest = new DescribeImagesRequest().withFilters(new Filter("virtualization-type").withValues("hvm"), new Filter("root-device-type").withValues("ebs")).withOwners(// this is the ownerId of amazon itself
    "099720109477");
    if (osCapability.getType().isPresent() && osCapability.getType().get().equals(OsCapability.Type.WINDOWS)) {
        describeImagesRequest.withFilters(new Filter("platform").withValues("windows"));
    }
    if (osCapability.getDistribution().isPresent()) {
        if (osCapability.getDistribution().get().equals(OsCapability.Distribution.UBUNTU)) {
            // */ubuntu/images/* gets better results than plain *ubuntu*
            describeImagesRequest.withFilters(new Filter("name").withValues("*ubuntu/images/*"));
        } else {
            // just search for the string
            describeImagesRequest.withFilters(new Filter("name").withValues("*" + osCapability.getDistribution().toString() + "*"));
        }
    }
    if (osCapability.getVersion().isPresent()) {
        describeImagesRequest.withFilters(new Filter("name").withValues("*" + osCapability.getVersion().get() + "*"));
    }
    if (osCapability.getArchitecture().isPresent()) {
        if (osCapability.getArchitecture().get().equals(OsCapability.Architecture.x86_64)) {
            describeImagesRequest.withFilters(new Filter("architecture").withValues(ARCH_x86_64));
        } else if (osCapability.getArchitecture().get().equals(OsCapability.Architecture.x86_32)) {
            describeImagesRequest.withFilters(new Filter("architecture").withValues(ARCH_x86_32));
        } else {
            throw new UnsupportedOperationException("This architecture is not supported " + osCapability.getArchitecture());
        }
    } else {
        // defaulting to 64 bit architecture
        describeImagesRequest.withFilters(new Filter("architecture").withValues(ARCH_x86_64));
    }
    try {
        DescribeImagesResult describeImagesResult = ec2.describeImages(describeImagesRequest);
        String imageId = processResult(describeImagesResult);
        logger.debug("ImageId is: '{}'", imageId);
        return imageId;
    } catch (SdkClientException se) {
        logger.error("Cannot connect to AWS to request image Ids");
        throw se;
    } catch (ParseException pe) {
        logger.error("Error parsing date format of image creation dates");
        throw pe;
    } catch (IllegalArgumentException ie) {
        logger.error("With the filters created from the OsCapability there are no valid images received");
        throw ie;
    }
}
Also used : AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) SdkClientException(com.amazonaws.SdkClientException) Filter(com.amazonaws.services.ec2.model.Filter) DescribeImagesResult(com.amazonaws.services.ec2.model.DescribeImagesResult) DescribeImagesRequest(com.amazonaws.services.ec2.model.DescribeImagesRequest) AmazonEC2(com.amazonaws.services.ec2.AmazonEC2) ParseException(java.text.ParseException)

Aggregations

AWSStaticCredentialsProvider (com.amazonaws.auth.AWSStaticCredentialsProvider)79 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)57 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)18 AWSCredentials (com.amazonaws.auth.AWSCredentials)17 Test (org.junit.Test)14 ClientConfiguration (com.amazonaws.ClientConfiguration)13 ProfileCredentialsProvider (com.amazonaws.auth.profile.ProfileCredentialsProvider)11 AmazonS3 (com.amazonaws.services.s3.AmazonS3)11 AmazonS3ClientBuilder (com.amazonaws.services.s3.AmazonS3ClientBuilder)11 SdkClientException (com.amazonaws.SdkClientException)10 BasicSessionCredentials (com.amazonaws.auth.BasicSessionCredentials)9 AwsClientBuilder (com.amazonaws.client.builder.AwsClientBuilder)9 AmazonClientException (com.amazonaws.AmazonClientException)8 Regions (com.amazonaws.regions.Regions)8 Test (org.junit.jupiter.api.Test)7 EndpointConfiguration (com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration)6 File (java.io.File)6 AmazonServiceException (com.amazonaws.AmazonServiceException)5 AWSLambda (com.amazonaws.services.lambda.AWSLambda)4 Segment (com.amazonaws.xray.entities.Segment)4