Search in sources :

Example 1 with Bucket

use of software.amazon.awssdk.services.s3.model.Bucket in project camel by apache.

the class AmazonS3ClientMock method listBuckets.

@Override
public List<Bucket> listBuckets() throws AmazonClientException, AmazonServiceException {
    ArrayList<Bucket> list = new ArrayList<Bucket>();
    Bucket bucket = new Bucket("camel-bucket");
    bucket.setOwner(new Owner("Camel", "camel"));
    bucket.setCreationDate(new Date());
    list.add(bucket);
    return list;
}
Also used : Owner(com.amazonaws.services.s3.model.Owner) Bucket(com.amazonaws.services.s3.model.Bucket) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) ArrayList(java.util.ArrayList) Date(java.util.Date)

Example 2 with Bucket

use of software.amazon.awssdk.services.s3.model.Bucket in project uploader by smoketurner.

the class Uploader method upload.

/**
 * Upload a batch to S3
 *
 * @param batch
 *            Batch to upload
 */
public void upload(@Nonnull final Batch batch) {
    batchSize.update(batch.size());
    batchCount.update(batch.getCount());
    final ImmutableMap.Builder<String, String> builder = ImmutableMap.<String, String>builder().put("count", String.valueOf(batch.getCount()));
    batch.getCustomerId().ifPresent(id -> builder.put("customer_id", id));
    final Map<String, String> metadata = builder.build();
    final String key;
    if (configuration.getPrefix().isPresent()) {
        key = configuration.getPrefix().get() + "/" + batch.getKey();
    } else {
        key = batch.getKey();
    }
    LOGGER.debug("Customer: {}, S3 key: {}", batch.getCustomerId().orElse(null), key);
    final PutObjectRequest request = PutObjectRequest.builder().bucket(configuration.getBucketName()).key(key).metadata(metadata).contentLength(batch.size()).contentType(MediaType.TEXT_PLAIN).contentEncoding("gzip").serverSideEncryption(ServerSideEncryption.AES256).build();
    final long start = currentTimeProvider.get();
    final CompletableFuture<PutObjectResponse> future = s3.putObject(request, new BatchRequestProvider(batch));
    future.whenComplete((resp, err) -> {
        if (resp != null) {
            final long took = currentTimeProvider.get() - start;
            uploadTime.update(took, TimeUnit.NANOSECONDS);
            successCounter.inc();
            LOGGER.info("Finished uploading \"{}\" ({} events, {} bytes) in {}ms", key, batch.getCount(), batch.size(), (took / NANOS_IN_MILLIS));
        } else {
            failedCounter.inc();
            LOGGER.error(String.format("Failed to upload \"%s\"", key), err);
        }
    });
}
Also used : PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) ImmutableMap(com.google.common.collect.ImmutableMap) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 3 with Bucket

use of software.amazon.awssdk.services.s3.model.Bucket in project tutorials by eugenp.

the class S3Application method main.

public static void main(String[] args) throws IOException {
    // set-up the client
    AmazonS3 s3client = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion(Regions.US_EAST_2).build();
    AWSS3Service awsService = new AWSS3Service(s3client);
    bucketName = "baeldung-bucket";
    // creating a bucket
    if (awsService.doesBucketExist(bucketName)) {
        System.out.println("Bucket name is not available." + " Try again with a different Bucket name.");
        return;
    }
    awsService.createBucket(bucketName);
    // list all the buckets
    for (Bucket s : awsService.listBuckets()) {
        System.out.println(s.getName());
    }
    // deleting bucket
    awsService.deleteBucket("baeldung-bucket-test2");
    // uploading object
    awsService.putObject(bucketName, "Document/hello.txt", new File("/Users/user/Document/hello.txt"));
    // listing objects
    ObjectListing objectListing = awsService.listObjects(bucketName);
    for (S3ObjectSummary os : objectListing.getObjectSummaries()) {
        System.out.println(os.getKey());
    }
    // downloading an object
    S3Object s3object = awsService.getObject(bucketName, "Document/hello.txt");
    S3ObjectInputStream inputStream = s3object.getObjectContent();
    FileUtils.copyInputStreamToFile(inputStream, new File("/Users/user/Desktop/hello.txt"));
    // copying an object
    awsService.copyObject("baeldung-bucket", "picture/pic.png", "baeldung-bucket2", "Document/picture.png");
    // deleting an object
    awsService.deleteObject(bucketName, "Document/hello.txt");
    // deleting multiple objects
    String[] objkeyArr = { "Document/hello2.txt", "Document/picture.png" };
    DeleteObjectsRequest delObjReq = new DeleteObjectsRequest("baeldung-bucket").withKeys(objkeyArr);
    awsService.deleteObjects(delObjReq);
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) AWSStaticCredentialsProvider(com.amazonaws.auth.AWSStaticCredentialsProvider) Bucket(com.amazonaws.services.s3.model.Bucket) ObjectListing(com.amazonaws.services.s3.model.ObjectListing) S3ObjectSummary(com.amazonaws.services.s3.model.S3ObjectSummary) S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) S3Object(com.amazonaws.services.s3.model.S3Object) File(java.io.File) DeleteObjectsRequest(com.amazonaws.services.s3.model.DeleteObjectsRequest)

Example 4 with Bucket

use of software.amazon.awssdk.services.s3.model.Bucket in project sandbox by irof.

the class S3Test method setup.

@Before
public void setup() {
    // bucketがなかったら作る
    if (!s3.doesBucketExist("irof-sandbox")) {
        Bucket bucket = s3.createBucket("irof-sandbox");
        logger.info("create {}", bucket.getName());
    }
}
Also used : Bucket(com.amazonaws.services.s3.model.Bucket) Before(org.junit.Before)

Example 5 with Bucket

use of software.amazon.awssdk.services.s3.model.Bucket in project testcontainers-java by testcontainers.

the class SimpleLocalstackS3Test method s3Test.

@Test
public void s3Test() throws IOException {
    AmazonS3 s3 = AmazonS3ClientBuilder.standard().withEndpointConfiguration(localstack.getEndpointConfiguration(S3)).withCredentials(localstack.getDefaultCredentialsProvider()).build();
    s3.createBucket("foo");
    s3.putObject("foo", "bar", "baz");
    final List<Bucket> buckets = s3.listBuckets();
    assertEquals("The created bucket is present", 1, buckets.size());
    final Bucket bucket = buckets.get(0);
    assertEquals("The created bucket has the right name", "foo", bucket.getName());
    assertEquals("The created bucket has the right name", "foo", bucket.getName());
    final ObjectListing objectListing = s3.listObjects("foo");
    assertEquals("The created bucket has 1 item in it", 1, objectListing.getObjectSummaries().size());
    final S3Object object = s3.getObject("foo", "bar");
    final String content = IOUtils.toString(object.getObjectContent(), Charset.forName("UTF-8"));
    assertEquals("The object can be retrieved", "baz", content);
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) Bucket(com.amazonaws.services.s3.model.Bucket) ObjectListing(com.amazonaws.services.s3.model.ObjectListing) S3Object(com.amazonaws.services.s3.model.S3Object) Test(org.junit.Test)

Aggregations

S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)60 S3Client (software.amazon.awssdk.services.s3.S3Client)53 Region (software.amazon.awssdk.regions.Region)47 Bucket (com.amazonaws.services.s3.model.Bucket)32 ArrayList (java.util.ArrayList)24 Test (org.junit.Test)22 IOException (java.io.IOException)18 GetObjectRequest (software.amazon.awssdk.services.s3.model.GetObjectRequest)18 S3Object (software.amazon.awssdk.services.s3.model.S3Object)17 GetObjectResponse (software.amazon.awssdk.services.s3.model.GetObjectResponse)14 HeadObjectResponse (software.amazon.awssdk.services.s3.model.HeadObjectResponse)14 PutObjectRequest (software.amazon.awssdk.services.s3.model.PutObjectRequest)14 ListObjectsV2Response (software.amazon.awssdk.services.s3.model.ListObjectsV2Response)11 AmazonS3 (com.amazonaws.services.s3.AmazonS3)10 ListObjectsV2Request (software.amazon.awssdk.services.s3.model.ListObjectsV2Request)10 CompleteMultipartUploadRequest (software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest)9 S3TestUtils.buildMockedS3FileSystem (org.apache.beam.sdk.io.aws2.s3.S3TestUtils.buildMockedS3FileSystem)8 MatchResult (org.apache.beam.sdk.io.fs.MatchResult)8 Date (java.util.Date)7 CompletedPart (software.amazon.awssdk.services.s3.model.CompletedPart)7