Examples with AmazonS3 com.talend.shaded.com.amazonaws.services.s3.AmazonS3 used on opensource projects

Example 86 with AmazonS3

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3 in project components by Talend.

the class S3DatasetRuntime method listBuckets.

@Override
public Set<String> listBuckets() {
    AmazonS3 conn = S3Connection.createClient(properties.getDatastoreProperties());
    String region = properties.region.getValue().getValue();
    if (S3Region.OTHER.getValue().equals(region)) {
        region = properties.unknownRegion.getValue();
    }
    conn.setEndpoint(regionToEndpoint(region));
    LOG.debug("Start to find buckets in region {}", region);
    List<Bucket> buckets = conn.listBuckets();
    Set<String> bucketsName = new HashSet<>();
    for (Bucket bucket : buckets) {
        String bucketName = bucket.getName();
        try {
            String bucketLocation = conn.getBucketLocation(bucketName);
            String bucketRegion = locationToRegion(bucketLocation);
            LOG.debug("Bucket is {} and location is {}({})", bucketName, bucketLocation, bucketRegion);
            if (region.equals(bucketRegion)) {
                bucketsName.add(bucketName);
            }
        } catch (Exception e) {
            // Ignore any exception when calling getBucketLocation, try next
            LOG.debug("Exception when check bucket location: {}", e.getMessage());
        }
    }
    return bucketsName;
}

Example 87 with AmazonS3

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3 in project syndesis by syndesisio.

the class AWSS3VerifierExtension method verifyConnectivity.

// *********************************
// Connectivity validation
// *********************************
@Override
@SuppressWarnings("PMD.AvoidCatchingGenericException")
protected Result verifyConnectivity(Map<String, Object> parameters) {
    ResultBuilder builder = ResultBuilder.withStatusAndScope(Result.Status.OK, Scope.CONNECTIVITY);
    try {
        S3Configuration configuration = setProperties(new S3Configuration(), parameters);
        AWSCredentials credentials = new BasicAWSCredentials(configuration.getAccessKey(), configuration.getSecretKey());
        AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(credentials);
        AmazonS3 client = AmazonS3ClientBuilder.standard().withCredentials(credentialsProvider).withRegion(Regions.valueOf(configuration.getRegion())).build();
        client.listBuckets();
    } catch (SdkClientException e) {
        ResultErrorBuilder errorBuilder = ResultErrorBuilder.withCodeAndDescription(VerificationError.StandardCode.AUTHENTICATION, e.getMessage()).detail("aws_s3_exception_message", e.getMessage()).detail(VerificationError.ExceptionAttribute.EXCEPTION_CLASS, e.getClass().getName()).detail(VerificationError.ExceptionAttribute.EXCEPTION_INSTANCE, e);
        builder.error(errorBuilder.build());
    } catch (@SuppressWarnings("PMD.AvoidCatchingGenericException") Exception e) {
        builder.error(ResultErrorBuilder.withException(e).build());
    }
    return builder.build();
}

Example 88 with AmazonS3

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3 in project qpp-conversion-tool by CMSgov.

the class QrdaApiAcceptance method getS3ObjectCount.

private long getS3ObjectCount() {
    AmazonS3 s3Client = AwsTestHelper.getS3Client();
    ObjectListing objectListing = s3Client.listObjects(TEST_S3_BUCKET_NAME);
    long objectCount = objectListing.getObjectSummaries().size();
    while (objectListing.isTruncated()) {
        objectListing = s3Client.listNextBatchOfObjects(objectListing);
        objectCount += objectListing.getObjectSummaries().size();
    }
    return objectCount;
}