Search in sources :

Example 6 with StaticCredentialsProvider

use of com.amazonaws.internal.StaticCredentialsProvider in project archaius by Netflix.

the class S3ConfigurationSourceTest method setup.

@Before
public void setup() throws Exception {
    fakeS3 = createHttpServer();
    client = new AmazonS3Client(new StaticCredentialsProvider(new AnonymousAWSCredentials()));
    client.setS3ClientOptions(new S3ClientOptions().withPathStyleAccess(true));
    client.setEndpoint("http://localhost:8069");
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) StaticCredentialsProvider(com.amazonaws.internal.StaticCredentialsProvider) S3ClientOptions(com.amazonaws.services.s3.S3ClientOptions) AnonymousAWSCredentials(com.amazonaws.auth.AnonymousAWSCredentials) Before(org.junit.Before)

Example 7 with StaticCredentialsProvider

use of com.amazonaws.internal.StaticCredentialsProvider in project beam by apache.

the class KinesisUploader method uploadAll.

public static void uploadAll(List<String> data, KinesisTestOptions options) {
    AmazonKinesisClient client = new AmazonKinesisClient(new StaticCredentialsProvider(new BasicAWSCredentials(options.getAwsAccessKey(), options.getAwsSecretKey()))).withRegion(Regions.fromName(options.getAwsKinesisRegion()));
    List<List<String>> partitions = Lists.partition(data, MAX_NUMBER_OF_RECORDS_IN_BATCH);
    for (List<String> partition : partitions) {
        List<PutRecordsRequestEntry> allRecords = newArrayList();
        for (String row : partition) {
            allRecords.add(new PutRecordsRequestEntry().withData(ByteBuffer.wrap(row.getBytes(Charsets.UTF_8))).withPartitionKey(Integer.toString(row.hashCode())));
        }
        PutRecordsResult result;
        do {
            result = client.putRecords(new PutRecordsRequest().withStreamName(options.getAwsKinesisStream()).withRecords(allRecords));
            List<PutRecordsRequestEntry> failedRecords = newArrayList();
            int i = 0;
            for (PutRecordsResultEntry row : result.getRecords()) {
                if (row.getErrorCode() != null) {
                    failedRecords.add(allRecords.get(i));
                }
                ++i;
            }
            allRecords = failedRecords;
        } while (result.getFailedRecordCount() > 0);
    }
}
Also used : AmazonKinesisClient(com.amazonaws.services.kinesis.AmazonKinesisClient) PutRecordsRequestEntry(com.amazonaws.services.kinesis.model.PutRecordsRequestEntry) StaticCredentialsProvider(com.amazonaws.internal.StaticCredentialsProvider) BasicAWSCredentials(com.amazonaws.auth.BasicAWSCredentials) PutRecordsResult(com.amazonaws.services.kinesis.model.PutRecordsResult) List(java.util.List) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) PutRecordsResultEntry(com.amazonaws.services.kinesis.model.PutRecordsResultEntry) PutRecordsRequest(com.amazonaws.services.kinesis.model.PutRecordsRequest)

Aggregations

StaticCredentialsProvider (com.amazonaws.internal.StaticCredentialsProvider)7 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)4 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)4 DefaultAWSCredentialsProviderChain (com.amazonaws.auth.DefaultAWSCredentialsProviderChain)3 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)3 AWSCredentialsProviderChain (com.amazonaws.auth.AWSCredentialsProviderChain)2 ArrayList (java.util.ArrayList)2 List (java.util.List)2 HerdAWSCredentialsProvider (org.finra.herd.model.dto.HerdAWSCredentialsProvider)2 AnonymousAWSCredentials (com.amazonaws.auth.AnonymousAWSCredentials)1 Region (com.amazonaws.regions.Region)1 AmazonKinesisClient (com.amazonaws.services.kinesis.AmazonKinesisClient)1 PutRecordsRequest (com.amazonaws.services.kinesis.model.PutRecordsRequest)1 PutRecordsRequestEntry (com.amazonaws.services.kinesis.model.PutRecordsRequestEntry)1 PutRecordsResult (com.amazonaws.services.kinesis.model.PutRecordsResult)1 PutRecordsResultEntry (com.amazonaws.services.kinesis.model.PutRecordsResultEntry)1 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 AmazonS3EncryptionClient (com.amazonaws.services.s3.AmazonS3EncryptionClient)1 S3ClientOptions (com.amazonaws.services.s3.S3ClientOptions)1 CryptoConfiguration (com.amazonaws.services.s3.model.CryptoConfiguration)1