Search in sources :

Example 86 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project xm-ms-entity by xm-online.

the class AmazonS3Template method save.

/**
 * Save a file using authenticated session credentials.
 *
 * @param key is the name of the file to save in the bucket
 * @param inputStream is the file that will be saved
 */
public void save(String key, InputStream inputStream) throws IOException {
    ObjectMetadata metadata = new ObjectMetadata();
    metadata.setContentType(URLConnection.guessContentTypeFromStream(inputStream));
    PutObjectRequest request = new PutObjectRequest(bucket, key, inputStream, metadata);
    request.setCannedAcl(CannedAccessControlList.PublicRead);
    request.getRequestClientOptions().setReadLimit(Integer.MAX_VALUE);
    Upload upload = getTransferManager().upload(request);
    try {
        upload.waitForUploadResult();
    } catch (AmazonClientException ex) {
        throw new IOException(ex);
    } catch (InterruptedException ex) {
        // reset interrupted status
        Thread.currentThread().interrupt();
        // continue interrupt
        throw new IllegalStateException(ex);
    }
}
Also used : AmazonClientException(com.amazonaws.AmazonClientException) Upload(com.amazonaws.services.s3.transfer.Upload) IOException(java.io.IOException) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest)

Example 87 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project spring-integration-aws by spring-projects.

the class S3MessageHandlerTests method testUploadByteArray.

@Test
public void testUploadByteArray() throws IOException {
    byte[] payload = "b".getBytes("UTF-8");
    Message<?> message = MessageBuilder.withPayload(payload).setHeader("s3Command", S3MessageHandler.Command.UPLOAD.name()).setHeader("key", "myStream").build();
    this.s3SendChannel.send(message);
    ArgumentCaptor<PutObjectRequest> putObjectRequestArgumentCaptor = ArgumentCaptor.forClass(PutObjectRequest.class);
    verify(this.amazonS3, atLeastOnce()).putObject(putObjectRequestArgumentCaptor.capture());
    PutObjectRequest putObjectRequest = putObjectRequestArgumentCaptor.getValue();
    assertThat(putObjectRequest.getBucketName()).isEqualTo(S3_BUCKET_NAME);
    assertThat(putObjectRequest.getKey()).isEqualTo("myStream");
    assertThat(putObjectRequest.getFile()).isNull();
    assertThat(putObjectRequest.getInputStream()).isNotNull();
    ObjectMetadata metadata = putObjectRequest.getMetadata();
    assertThat(metadata.getContentMD5()).isEqualTo(Md5Utils.md5AsBase64(payload));
    assertThat(metadata.getContentLength()).isEqualTo(1);
    assertThat(metadata.getContentType()).isEqualTo(MediaType.APPLICATION_JSON_VALUE);
    assertThat(metadata.getContentDisposition()).isEqualTo("test.json");
}
Also used : ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) Test(org.junit.Test)

Example 88 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project spring-integration-aws by spring-projects.

the class S3MessageHandlerTests method testUploadInputStream.

@Test
public void testUploadInputStream() throws IOException {
    Expression actualKeyExpression = TestUtils.getPropertyValue(this.s3MessageHandler, "keyExpression", Expression.class);
    this.s3MessageHandler.setKeyExpression(null);
    InputStream payload = new StringInputStream("a");
    Message<?> message = MessageBuilder.withPayload(payload).setHeader("s3Command", S3MessageHandler.Command.UPLOAD.name()).setHeader("key", "myStream").build();
    assertThatThrownBy(() -> this.s3SendChannel.send(message)).hasCauseExactlyInstanceOf(IllegalStateException.class).hasMessageContaining("Specify a 'keyExpression' for non-java.io.File payloads");
    this.s3MessageHandler.setKeyExpression(actualKeyExpression);
    this.s3SendChannel.send(message);
    ArgumentCaptor<PutObjectRequest> putObjectRequestArgumentCaptor = ArgumentCaptor.forClass(PutObjectRequest.class);
    verify(this.amazonS3, atLeastOnce()).putObject(putObjectRequestArgumentCaptor.capture());
    PutObjectRequest putObjectRequest = putObjectRequestArgumentCaptor.getValue();
    assertThat(putObjectRequest.getBucketName()).isEqualTo(S3_BUCKET_NAME);
    assertThat(putObjectRequest.getKey()).isEqualTo("myStream");
    assertThat(putObjectRequest.getFile()).isNull();
    assertThat(putObjectRequest.getInputStream()).isNotNull();
    ObjectMetadata metadata = putObjectRequest.getMetadata();
    assertThat(metadata.getContentMD5()).isEqualTo(Md5Utils.md5AsBase64(payload));
    assertThat(metadata.getContentLength()).isEqualTo(1);
    assertThat(metadata.getContentType()).isEqualTo(MediaType.APPLICATION_JSON_VALUE);
    assertThat(metadata.getContentDisposition()).isEqualTo("test.json");
}
Also used : StringInputStream(com.amazonaws.util.StringInputStream) ValueExpression(org.springframework.integration.expression.ValueExpression) Expression(org.springframework.expression.Expression) ByteArrayInputStream(java.io.ByteArrayInputStream) S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) StringInputStream(com.amazonaws.util.StringInputStream) FileInputStream(java.io.FileInputStream) InputStream(java.io.InputStream) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) Test(org.junit.Test)

Example 89 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project qpp-conversion-tool by CMSgov.

the class StorageServiceImpl method store.

/**
 * Stores the {@link InputStream} as an object in the S3 bucket.
 *
 * @param keyName The requested key name for the object.
 * @param inStream The {@link InputStream} to write out to an object in S3.
 * @param size The size of the {@link InputStream}.
 * @return A {@link CompletableFuture} that will eventually contain the S3 object key.
 */
@Override
public CompletableFuture<String> store(String keyName, Supplier<InputStream> inStream, long size) {
    final String bucketName = environment.getProperty(Constants.BUCKET_NAME_ENV_VARIABLE);
    final String kmsKey = environment.getProperty(Constants.KMS_KEY_ENV_VARIABLE);
    if (Strings.isNullOrEmpty(bucketName) || Strings.isNullOrEmpty(kmsKey)) {
        API_LOG.warn("No bucket name is specified or no KMS key specified.");
        return CompletableFuture.completedFuture("");
    }
    ObjectMetadata s3ObjectMetadata = new ObjectMetadata();
    s3ObjectMetadata.setContentLength(size);
    Supplier<PutObjectRequest> putObjectRequest = () -> new PutObjectRequest(bucketName, keyName, inStream.get(), s3ObjectMetadata).withSSEAwsKeyManagementParams(new SSEAwsKeyManagementParams(kmsKey));
    API_LOG.info("Writing object {} to S3 bucket {}", keyName, bucketName);
    return actOnItem(putObjectRequest);
}
Also used : ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest) SSEAwsKeyManagementParams(com.amazonaws.services.s3.model.SSEAwsKeyManagementParams)

Example 90 with ObjectMetadata

use of com.talend.shaded.com.amazonaws.services.s3.model.ObjectMetadata in project elasticsearch by elastic.

the class DefaultS3OutputStream method doInitialize.

protected String doInitialize(S3BlobStore blobStore, String bucketName, String blobName, boolean serverSideEncryption) {
    InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, blobName).withCannedACL(blobStore.getCannedACL()).withStorageClass(blobStore.getStorageClass());
    if (serverSideEncryption) {
        ObjectMetadata md = new ObjectMetadata();
        md.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
        request.setObjectMetadata(md);
    }
    return blobStore.client().initiateMultipartUpload(request).getUploadId();
}
Also used : InitiateMultipartUploadRequest(com.amazonaws.services.s3.model.InitiateMultipartUploadRequest) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata)

Aggregations

ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)163 PutObjectRequest (com.amazonaws.services.s3.model.PutObjectRequest)76 ByteArrayInputStream (java.io.ByteArrayInputStream)52 Test (org.junit.Test)47 IOException (java.io.IOException)33 File (java.io.File)27 AmazonClientException (com.amazonaws.AmazonClientException)25 AmazonServiceException (com.amazonaws.AmazonServiceException)22 S3FileTransferRequestParamsDto (org.finra.herd.model.dto.S3FileTransferRequestParamsDto)21 InputStream (java.io.InputStream)20 DataStoreException (org.apache.jackrabbit.core.data.DataStoreException)18 PutObjectResult (com.amazonaws.services.s3.model.PutObjectResult)15 Upload (com.amazonaws.services.s3.transfer.Upload)15 CopyObjectRequest (com.amazonaws.services.s3.model.CopyObjectRequest)11 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)10 Date (java.util.Date)9 BusinessObjectDataKey (org.finra.herd.model.api.xml.BusinessObjectDataKey)9 Copy (com.amazonaws.services.s3.transfer.Copy)8 S3Object (com.amazonaws.services.s3.model.S3Object)7 InterruptedIOException (java.io.InterruptedIOException)7