Search in sources :

Example 1 with RequestPayer

use of software.amazon.awssdk.services.s3.model.RequestPayer in project aws-sdk-java-v2 by aws.

the class S3PresignerIntegrationTest method browserIncompatiblePresignedUrlWorksWithAdditionalHeaders.

@Test
public void browserIncompatiblePresignedUrlWorksWithAdditionalHeaders() throws IOException {
    PresignedGetObjectRequest presigned = presigner.presignGetObject(r -> r.signatureDuration(Duration.ofMinutes(5)).getObjectRequest(gor -> gor.bucket(testBucket).key(testGetObjectKey).requestPayer(RequestPayer.REQUESTER)));
    assertThat(presigned.isBrowserExecutable()).isFalse();
    HttpURLConnection connection = (HttpURLConnection) presigned.url().openConnection();
    presigned.httpRequest().headers().forEach((header, values) -> {
        values.forEach(value -> {
            connection.addRequestProperty(header, value);
        });
    });
    try (InputStream content = connection.getInputStream()) {
        assertThat(IoUtils.toUtf8String(content)).isEqualTo(testObjectContent);
    }
}
Also used : HttpURLConnection(java.net.HttpURLConnection) BeforeClass(org.junit.BeforeClass) ApacheHttpClient(software.amazon.awssdk.http.apache.ApacheHttpClient) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) CreateMultipartUploadResponse(software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse) SdkBytes(software.amazon.awssdk.core.SdkBytes) PresignedAbortMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedAbortMultipartUploadRequest) RequestPayer(software.amazon.awssdk.services.s3.model.RequestPayer) S3Presigner(software.amazon.awssdk.services.s3.presigner.S3Presigner) Duration(java.time.Duration) After(org.junit.After) PresignedPutObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest) Before(org.junit.Before) UploadPartRequest(software.amazon.awssdk.services.s3.model.UploadPartRequest) AfterClass(org.junit.AfterClass) MultipartUpload(software.amazon.awssdk.services.s3.model.MultipartUpload) PresignedGetObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest) ContentStreamProvider(software.amazon.awssdk.http.ContentStreamProvider) PresignedUploadPartRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedUploadPartRequest) IOException(java.io.IOException) Test(org.junit.Test) PresignedRequest(software.amazon.awssdk.awscore.presigner.PresignedRequest) UUID(java.util.UUID) SdkHttpClient(software.amazon.awssdk.http.SdkHttpClient) CompleteMultipartUploadRequest(software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest) PresignedCompleteMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedCompleteMultipartUploadRequest) S3TestUtils(software.amazon.awssdk.services.s3.utils.S3TestUtils) Consumer(java.util.function.Consumer) AbortableInputStream(software.amazon.awssdk.http.AbortableInputStream) StringInputStream(software.amazon.awssdk.utils.StringInputStream) PresignedCreateMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedCreateMultipartUploadRequest) Optional(java.util.Optional) RequestBody(software.amazon.awssdk.core.sync.RequestBody) HttpExecuteResponse(software.amazon.awssdk.http.HttpExecuteResponse) CreateMultipartUploadRequest(software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest) AbortMultipartUploadRequest(software.amazon.awssdk.services.s3.model.AbortMultipartUploadRequest) IoUtils(software.amazon.awssdk.utils.IoUtils) InputStream(java.io.InputStream) HttpExecuteRequest(software.amazon.awssdk.http.HttpExecuteRequest) UploadPartResponse(software.amazon.awssdk.services.s3.model.UploadPartResponse) HttpURLConnection(java.net.HttpURLConnection) AbortableInputStream(software.amazon.awssdk.http.AbortableInputStream) StringInputStream(software.amazon.awssdk.utils.StringInputStream) InputStream(java.io.InputStream) PresignedGetObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest) Test(org.junit.Test)

Example 2 with RequestPayer

use of software.amazon.awssdk.services.s3.model.RequestPayer in project hedera-mirror-node by hashgraph.

the class Downloader method listFiles.

private List<S3Object> listFiles(String lastFilename, String nodeAccountId) throws ExecutionException, InterruptedException {
    // batchSize (number of items we plan do download in a single batch) times 2 for file + sig.
    int listSize = (downloaderProperties.getBatchSize() * 2);
    String s3Prefix = getS3Prefix(nodeAccountId);
    // Not using ListObjectsV2Request because it does not work with GCP.
    ListObjectsRequest listRequest = ListObjectsRequest.builder().bucket(commonDownloaderProperties.getBucketName()).prefix(s3Prefix).delimiter("/").marker(s3Prefix + lastFilename).maxKeys(listSize).requestPayer(RequestPayer.REQUESTER).build();
    return s3Client.listObjects(listRequest).get().contents();
}
Also used : ListObjectsRequest(software.amazon.awssdk.services.s3.model.ListObjectsRequest)

Example 3 with RequestPayer

use of software.amazon.awssdk.services.s3.model.RequestPayer in project spring-cloud-aws by awspring.

the class ObjectMetadataTests method mapsEnumsToString.

@Test
void mapsEnumsToString() {
    ObjectMetadata metadata = ObjectMetadata.builder().acl(ObjectCannedACL.AUTHENTICATED_READ).storageClass(StorageClass.ONEZONE_IA).objectLockLegalHoldStatus(ObjectLockLegalHoldStatus.OFF).requestPayer(RequestPayer.REQUESTER).serverSideEncryption(ServerSideEncryption.AES256).checksumAlgorithm(ChecksumAlgorithm.CRC32).build();
    PutObjectRequest.Builder builder = PutObjectRequest.builder();
    metadata.apply(builder);
    PutObjectRequest result = builder.build();
    assertThat(result.acl()).isEqualTo(ObjectCannedACL.AUTHENTICATED_READ);
    assertThat(result.storageClass()).isEqualTo(StorageClass.ONEZONE_IA);
    assertThat(result.objectLockLegalHoldStatus()).isEqualTo(ObjectLockLegalHoldStatus.OFF);
    assertThat(result.requestPayer()).isEqualTo(RequestPayer.REQUESTER);
    assertThat(result.serverSideEncryption()).isEqualTo(ServerSideEncryption.AES256);
    assertThat(result.checksumAlgorithm()).isEqualTo(ChecksumAlgorithm.CRC32);
}
Also used : PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest) Test(org.junit.jupiter.api.Test)

Example 4 with RequestPayer

use of software.amazon.awssdk.services.s3.model.RequestPayer in project spring-cloud-aws by awspring.

the class ObjectMetadataTests method canBeAppliedToPutObjectRequest.

@Test
void canBeAppliedToPutObjectRequest() {
    Instant now = Instant.now();
    ObjectMetadata objectMetadata = ObjectMetadata.builder().acl("acl").cacheControl("cacheControl").contentDisposition("contentDisposition").contentEncoding("contentEncoding").contentLanguage("contentLanguage").contentType("contentType").expires(now).grantFullControl("grantFullControl").grantRead("grantRead").grantReadACP("grantReadACP").grantWriteACP("grantWriteACP").metadata("key1", "value1").metadata("key2", "value2").serverSideEncryption("serverSideEncryption").storageClass("storageClass").websiteRedirectLocation("websiteRedirectLocation").sseCustomerAlgorithm("sseCustomerAlgorithm").sseCustomerKey("sseCustomerKey").sseCustomerKeyMD5("sseCustomerKeyMD5").ssekmsKeyId("ssekmsKeyId").ssekmsEncryptionContext("ssekmsEncryptionContext").bucketKeyEnabled(true).requestPayer("requestPayer").tagging("tagging").objectLockMode("objectLockMode").objectLockRetainUntilDate(now).objectLockLegalHoldStatus("objectLockLegalHoldStatus").expectedBucketOwner("expectedBucketOwner").checksumAlgorithm("checksumAlgorithm").build();
    PutObjectRequest.Builder builder = PutObjectRequest.builder();
    objectMetadata.apply(builder);
    assertThat(builder).usingRecursiveComparison().ignoringFields("awsRequestOverrideConfig", "checksumCRC32C", "checksumSHA1", "checksumSHA256", "key", "contentMD5", "bucket", "checksumCRC32", "contentLength").isEqualTo(objectMetadata);
}
Also used : Instant(java.time.Instant) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest) Test(org.junit.jupiter.api.Test)

Example 5 with RequestPayer

use of software.amazon.awssdk.services.s3.model.RequestPayer in project aws-sdk-java-v2 by aws.

the class S3PresignerIntegrationTest method browserIncompatiblePresignedUrlDoesNotWorkWithoutAdditionalHeaders.

@Test
public void browserIncompatiblePresignedUrlDoesNotWorkWithoutAdditionalHeaders() throws IOException {
    PresignedGetObjectRequest presigned = presigner.presignGetObject(r -> r.signatureDuration(Duration.ofMinutes(5)).getObjectRequest(gor -> gor.bucket(testBucket).key(testGetObjectKey).requestPayer(RequestPayer.REQUESTER)));
    assertThat(presigned.isBrowserExecutable()).isFalse();
    HttpURLConnection connection = (HttpURLConnection) presigned.url().openConnection();
    connection.connect();
    try {
        assertThat(connection.getResponseCode()).isEqualTo(403);
    } finally {
        connection.disconnect();
    }
}
Also used : HttpURLConnection(java.net.HttpURLConnection) BeforeClass(org.junit.BeforeClass) ApacheHttpClient(software.amazon.awssdk.http.apache.ApacheHttpClient) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) CreateMultipartUploadResponse(software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse) SdkBytes(software.amazon.awssdk.core.SdkBytes) PresignedAbortMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedAbortMultipartUploadRequest) RequestPayer(software.amazon.awssdk.services.s3.model.RequestPayer) S3Presigner(software.amazon.awssdk.services.s3.presigner.S3Presigner) Duration(java.time.Duration) After(org.junit.After) PresignedPutObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest) Before(org.junit.Before) UploadPartRequest(software.amazon.awssdk.services.s3.model.UploadPartRequest) AfterClass(org.junit.AfterClass) MultipartUpload(software.amazon.awssdk.services.s3.model.MultipartUpload) PresignedGetObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest) ContentStreamProvider(software.amazon.awssdk.http.ContentStreamProvider) PresignedUploadPartRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedUploadPartRequest) IOException(java.io.IOException) Test(org.junit.Test) PresignedRequest(software.amazon.awssdk.awscore.presigner.PresignedRequest) UUID(java.util.UUID) SdkHttpClient(software.amazon.awssdk.http.SdkHttpClient) CompleteMultipartUploadRequest(software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest) PresignedCompleteMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedCompleteMultipartUploadRequest) S3TestUtils(software.amazon.awssdk.services.s3.utils.S3TestUtils) Consumer(java.util.function.Consumer) AbortableInputStream(software.amazon.awssdk.http.AbortableInputStream) StringInputStream(software.amazon.awssdk.utils.StringInputStream) PresignedCreateMultipartUploadRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedCreateMultipartUploadRequest) Optional(java.util.Optional) RequestBody(software.amazon.awssdk.core.sync.RequestBody) HttpExecuteResponse(software.amazon.awssdk.http.HttpExecuteResponse) CreateMultipartUploadRequest(software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest) AbortMultipartUploadRequest(software.amazon.awssdk.services.s3.model.AbortMultipartUploadRequest) IoUtils(software.amazon.awssdk.utils.IoUtils) InputStream(java.io.InputStream) HttpExecuteRequest(software.amazon.awssdk.http.HttpExecuteRequest) UploadPartResponse(software.amazon.awssdk.services.s3.model.UploadPartResponse) HttpURLConnection(java.net.HttpURLConnection) PresignedGetObjectRequest(software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest) Test(org.junit.Test)

Aggregations

Duration (java.time.Duration)4 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)4 After (org.junit.After)4 Before (org.junit.Before)4 Test (org.junit.Test)4 RequestPayer (software.amazon.awssdk.services.s3.model.RequestPayer)4 S3Presigner (software.amazon.awssdk.services.s3.presigner.S3Presigner)4 PresignedGetObjectRequest (software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest)4 PresignedPutObjectRequest (software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest)4 IOException (java.io.IOException)3 InputStream (java.io.InputStream)3 HttpURLConnection (java.net.HttpURLConnection)3 Optional (java.util.Optional)3 UUID (java.util.UUID)3 Consumer (java.util.function.Consumer)3 AfterClass (org.junit.AfterClass)3 BeforeClass (org.junit.BeforeClass)3 PresignedRequest (software.amazon.awssdk.awscore.presigner.PresignedRequest)3 SdkBytes (software.amazon.awssdk.core.SdkBytes)3 RequestBody (software.amazon.awssdk.core.sync.RequestBody)3