Examples with AmazonS3Client com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client used on opensource projects

Example 56 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project ice by Netflix.

the class AwsUtils method listAllObjects.

/**
     * List all object summary with given prefix in the s3 bucket.
     * @param bucket
     * @param prefix
     * @return
     */
public static List<S3ObjectSummary> listAllObjects(String bucket, String prefix, String accountId, String assumeRole, String externalId) {
    AmazonS3Client s3Client = AwsUtils.s3Client;
    try {
        ListObjectsRequest request = new ListObjectsRequest().withBucketName(bucket).withPrefix(prefix);
        List<S3ObjectSummary> result = Lists.newLinkedList();
        if (!StringUtils.isEmpty(accountId) && !StringUtils.isEmpty(assumeRole)) {
            Credentials assumedCredentials = getAssumedCredentials(accountId, assumeRole, externalId);
            s3Client = new AmazonS3Client(new BasicSessionCredentials(assumedCredentials.getAccessKeyId(), assumedCredentials.getSecretAccessKey(), assumedCredentials.getSessionToken()), clientConfig);
        }
        ObjectListing page = null;
        do {
            if (page != null)
                request.setMarker(page.getNextMarker());
            page = s3Client.listObjects(request);
            result.addAll(page.getObjectSummaries());
        } while (page.isTruncated());
        return result;
    } finally {
        if (s3Client != AwsUtils.s3Client)
            s3Client.shutdown();
    }
}

Example 57 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project ice by Netflix.

the class BillingFileProcessor method getLastMillis.

private Long getLastMillis(String filename) {
    AmazonS3Client s3Client = AwsUtils.getAmazonS3Client();
    InputStream in = null;
    try {
        in = s3Client.getObject(config.workS3BucketName, config.workS3BucketPrefix + filename).getObjectContent();
        return Long.parseLong(IOUtils.toString(in));
    } catch (Exception e) {
        logger.error("Error reading from file " + filename, e);
        return 0L;
    } finally {
        if (in != null)
            try {
                in.close();
            } catch (Exception e) {
            }
    }
}

Example 58 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project exhibitor by soabase.

the class S3ClientImpl method listNextBatchOfObjects.

@Override
public ObjectListing listNextBatchOfObjects(ObjectListing previousObjectListing) throws Exception {
    RefCountedClient holder = client.get();
    AmazonS3Client amazonS3Client = holder.useClient();
    try {
        return amazonS3Client.listNextBatchOfObjects(previousObjectListing);
    } finally {
        holder.release();
    }
}

Example 59 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project exhibitor by soabase.

the class S3ClientImpl method deleteObject.

@Override
public void deleteObject(String bucket, String key) throws Exception {
    RefCountedClient holder = client.get();
    AmazonS3Client amazonS3Client = holder.useClient();
    try {
        amazonS3Client.deleteObject(bucket, key);
    } finally {
        holder.release();
    }
}

Example 60 with AmazonS3Client

use of com.talend.shaded.com.amazonaws.services.s3.AmazonS3Client in project exhibitor by soabase.

the class S3ClientImpl method getObjectMetadata.

@Override
public ObjectMetadata getObjectMetadata(String bucket, String key) throws Exception {
    RefCountedClient holder = client.get();
    AmazonS3Client amazonS3Client = holder.useClient();
    try {
        return amazonS3Client.getObjectMetadata(bucket, key);
    } finally {
        holder.release();
    }
}