Search in sources :

Example 1 with PutObjectResponse

use of software.amazon.awssdk.services.s3.model.PutObjectResponse in project uploader by smoketurner.

the class Uploader method upload.

/**
 * Upload a batch to S3
 *
 * @param batch
 *            Batch to upload
 */
public void upload(@Nonnull final Batch batch) {
    batchSize.update(batch.size());
    batchCount.update(batch.getCount());
    final ImmutableMap.Builder<String, String> builder = ImmutableMap.<String, String>builder().put("count", String.valueOf(batch.getCount()));
    batch.getCustomerId().ifPresent(id -> builder.put("customer_id", id));
    final Map<String, String> metadata = builder.build();
    final String key;
    if (configuration.getPrefix().isPresent()) {
        key = configuration.getPrefix().get() + "/" + batch.getKey();
    } else {
        key = batch.getKey();
    }
    LOGGER.debug("Customer: {}, S3 key: {}", batch.getCustomerId().orElse(null), key);
    final PutObjectRequest request = PutObjectRequest.builder().bucket(configuration.getBucketName()).key(key).metadata(metadata).contentLength(batch.size()).contentType(MediaType.TEXT_PLAIN).contentEncoding("gzip").serverSideEncryption(ServerSideEncryption.AES256).build();
    final long start = currentTimeProvider.get();
    final CompletableFuture<PutObjectResponse> future = s3.putObject(request, new BatchRequestProvider(batch));
    future.whenComplete((resp, err) -> {
        if (resp != null) {
            final long took = currentTimeProvider.get() - start;
            uploadTime.update(took, TimeUnit.NANOSECONDS);
            successCounter.inc();
            LOGGER.info("Finished uploading \"{}\" ({} events, {} bytes) in {}ms", key, batch.getCount(), batch.size(), (took / NANOS_IN_MILLIS));
        } else {
            failedCounter.inc();
            LOGGER.error(String.format("Failed to upload \"%s\"", key), err);
        }
    });
}
Also used : PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) ImmutableMap(com.google.common.collect.ImmutableMap) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 2 with PutObjectResponse

use of software.amazon.awssdk.services.s3.model.PutObjectResponse in project aws-doc-sdk-examples by awsdocs.

the class S3AsyncOps method main.

public static void main(String[] args) {
    final String USAGE = "\n" + "Usage:\n" + "    S3AsyncOps <bucketName> <key> <path>\n\n" + "Where:\n" + "    bucketName - the name of the Amazon S3 bucket (for example, bucket1). \n\n" + "    key - the name of the object (for example, book.pdf). \n" + "    path - the local path to the file (for example, C:/AWS/book.pdf). \n";
    if (args.length != 3) {
        System.out.println(USAGE);
        System.exit(1);
    }
    String bucketName = args[0];
    String key = args[1];
    String path = args[2];
    Region region = Region.US_WEST_2;
    S3AsyncClient client = S3AsyncClient.builder().region(region).build();
    PutObjectRequest objectRequest = PutObjectRequest.builder().bucket(bucketName).key(key).build();
    // Put the object into the bucket
    CompletableFuture<PutObjectResponse> future = client.putObject(objectRequest, AsyncRequestBody.fromFile(Paths.get(path)));
    future.whenComplete((resp, err) -> {
        try {
            if (resp != null) {
                System.out.println("Object uploaded. Details: " + resp);
            } else {
                // Handle error
                err.printStackTrace();
            }
        } finally {
            // Only close the client when you are completely done with it
            client.close();
        }
    });
    future.join();
}
Also used : PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) Region(software.amazon.awssdk.regions.Region) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest) S3AsyncClient(software.amazon.awssdk.services.s3.S3AsyncClient)

Example 3 with PutObjectResponse

use of software.amazon.awssdk.services.s3.model.PutObjectResponse in project aws-doc-sdk-examples by awsdocs.

the class PutObjectMetadata method putS3Object.

// snippet-start:[s3.java2.s3_object_upload.metadata.main]
public static String putS3Object(S3Client s3, String bucketName, String objectKey, String objectPath) {
    try {
        // Define the metadata
        Map<String, String> metadata = new HashMap<>();
        metadata.put("author", "Mary Doe");
        metadata.put("version", "1.0.0.0");
        PutObjectRequest putOb = PutObjectRequest.builder().bucket(bucketName).key(objectKey).metadata(metadata).build();
        PutObjectResponse response = s3.putObject(putOb, RequestBody.fromBytes(getObjectFile(objectPath)));
        return response.eTag();
    } catch (S3Exception e) {
        System.err.println(e.getMessage());
        System.exit(1);
    }
    return "";
}
Also used : HashMap(java.util.HashMap) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 4 with PutObjectResponse

use of software.amazon.awssdk.services.s3.model.PutObjectResponse in project aws-doc-sdk-examples by awsdocs.

the class PutObject method putS3Object.

// snippet-start:[s3.java2.s3_object_upload.main]
public static String putS3Object(S3Client s3, String bucketName, String objectKey, String objectPath) {
    try {
        Map<String, String> metadata = new HashMap<>();
        metadata.put("x-amz-meta-myVal", "test");
        PutObjectRequest putOb = PutObjectRequest.builder().bucket(bucketName).key(objectKey).metadata(metadata).build();
        PutObjectResponse response = s3.putObject(putOb, RequestBody.fromBytes(getObjectFile(objectPath)));
        return response.eTag();
    } catch (S3Exception e) {
        System.err.println(e.getMessage());
        System.exit(1);
    }
    return "";
}
Also used : HashMap(java.util.HashMap) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Example 5 with PutObjectResponse

use of software.amazon.awssdk.services.s3.model.PutObjectResponse in project data-transfer-project by google.

the class BackblazeDataTransferClient method uploadFile.

public String uploadFile(String fileKey, File file) throws IOException {
    if (s3Client == null || bucketName == null) {
        throw new IllegalStateException("BackblazeDataTransferClient has not been initialised");
    }
    try {
        long contentLength = file.length();
        monitor.debug(() -> String.format("Uploading '%s' with file size %d bytes", fileKey, contentLength));
        if (contentLength >= sizeThresholdForMultipartUpload) {
            monitor.debug(() -> String.format("File size is larger than %d bytes, so using multipart upload", sizeThresholdForMultipartUpload));
            return uploadFileUsingMultipartUpload(fileKey, file, contentLength);
        }
        PutObjectRequest putObjectRequest = PutObjectRequest.builder().bucket(bucketName).key(fileKey).build();
        PutObjectResponse putObjectResponse = s3Client.putObject(putObjectRequest, RequestBody.fromFile(file));
        return putObjectResponse.versionId();
    } catch (AwsServiceException | SdkClientException e) {
        throw new IOException(String.format("Error while uploading file, fileKey: %s", fileKey), e);
    }
}
Also used : SdkClientException(software.amazon.awssdk.core.exception.SdkClientException) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) AwsServiceException(software.amazon.awssdk.awscore.exception.AwsServiceException) IOException(java.io.IOException) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest)

Aggregations

PutObjectRequest (software.amazon.awssdk.services.s3.model.PutObjectRequest)5 PutObjectResponse (software.amazon.awssdk.services.s3.model.PutObjectResponse)5 HashMap (java.util.HashMap)2 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 IOException (java.io.IOException)1 AwsServiceException (software.amazon.awssdk.awscore.exception.AwsServiceException)1 SdkClientException (software.amazon.awssdk.core.exception.SdkClientException)1 Region (software.amazon.awssdk.regions.Region)1 S3AsyncClient (software.amazon.awssdk.services.s3.S3AsyncClient)1