Search in sources :

Example 1 with CompletedUpload

use of software.amazon.awssdk.transfer.s3.CompletedUpload in project aws-sdk-java-v2 by aws.

the class DefaultS3TransferManager method upload.

@Override
public Upload upload(UploadRequest uploadRequest) {
    Validate.paramNotNull(uploadRequest, "uploadRequest");
    AsyncRequestBody requestBody = uploadRequest.requestBody();
    CompletableFuture<CompletedUpload> uploadFuture = new CompletableFuture<>();
    TransferProgressUpdater progressUpdater = new TransferProgressUpdater(uploadRequest, requestBody);
    progressUpdater.transferInitiated();
    requestBody = progressUpdater.wrapRequestBody(requestBody);
    progressUpdater.registerCompletion(uploadFuture);
    try {
        assertNotUnsupportedArn(uploadRequest.putObjectRequest().bucket(), "upload");
        CompletableFuture<PutObjectResponse> putObjFuture = s3CrtAsyncClient.putObject(uploadRequest.putObjectRequest(), requestBody);
        // Forward upload cancellation to CRT future
        CompletableFutureUtils.forwardExceptionTo(uploadFuture, putObjFuture);
        CompletableFutureUtils.forwardTransformedResultTo(putObjFuture, uploadFuture, r -> CompletedUpload.builder().response(r).build());
    } catch (Throwable throwable) {
        uploadFuture.completeExceptionally(throwable);
    }
    return new DefaultUpload(uploadFuture, progressUpdater.progress());
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) CompletedUpload(software.amazon.awssdk.transfer.s3.CompletedUpload) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) TransferProgressUpdater(software.amazon.awssdk.transfer.s3.internal.progress.TransferProgressUpdater) AsyncRequestBody(software.amazon.awssdk.core.async.AsyncRequestBody)

Example 2 with CompletedUpload

use of software.amazon.awssdk.transfer.s3.CompletedUpload in project aws-sdk-java-v2 by aws.

the class UploadDirectoryHelperParameterizedTest method uploadDirectory_withDelimiter_shouldHonor.

@Test
public void uploadDirectory_withDelimiter_shouldHonor() {
    ArgumentCaptor<UploadFileRequest> requestArgumentCaptor = ArgumentCaptor.forClass(UploadFileRequest.class);
    when(singleUploadFunction.apply(requestArgumentCaptor.capture())).thenReturn(completedUpload());
    DirectoryUpload uploadDirectory = uploadDirectoryHelper.uploadDirectory(UploadDirectoryRequest.builder().sourceDirectory(directory).bucket("bucket").delimiter(",").prefix("yolo").build());
    uploadDirectory.completionFuture().join();
    List<String> keys = requestArgumentCaptor.getAllValues().stream().map(u -> u.putObjectRequest().key()).collect(Collectors.toList());
    assertThat(keys.size()).isEqualTo(3);
    assertThat(keys).containsOnly("yolo,foo,2.txt", "yolo,foo,1.txt", "yolo,bar.txt");
}
Also used : UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) FileUtils(software.amazon.awssdk.testutils.FileUtils) Arrays(java.util.Arrays) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) CompletableFuture(java.util.concurrent.CompletableFuture) UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) Function(java.util.function.Function) Sets(org.assertj.core.util.Sets) FileUpload(software.amazon.awssdk.transfer.s3.FileUpload) DefaultTransferProgressSnapshot(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgressSnapshot) ArgumentCaptor(org.mockito.ArgumentCaptor) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Jimfs(com.google.common.jimfs.Jimfs) After(org.junit.After) UploadDirectoryRequest(software.amazon.awssdk.transfer.s3.UploadDirectoryRequest) Path(java.nio.file.Path) Parameterized(org.junit.runners.Parameterized) Before(org.junit.Before) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) Configuration(com.google.common.jimfs.Configuration) Files(java.nio.file.Files) Collection(java.util.Collection) Set(java.util.Set) DefaultTransferProgress(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgress) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) FileSystem(java.nio.file.FileSystem) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) List(java.util.List) Paths(java.nio.file.Paths) Assume.assumeTrue(org.junit.Assume.assumeTrue) CompletedFileUpload(software.amazon.awssdk.transfer.s3.CompletedFileUpload) IoUtils(software.amazon.awssdk.utils.IoUtils) Mockito.mock(org.mockito.Mockito.mock) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) Test(org.junit.Test)

Example 3 with CompletedUpload

use of software.amazon.awssdk.transfer.s3.CompletedUpload in project aws-sdk-java-v2 by aws.

the class S3TransferManagerTest method upload_returnsResponse.

@Test
public void upload_returnsResponse() {
    PutObjectResponse response = PutObjectResponse.builder().build();
    when(mockS3Crt.putObject(any(PutObjectRequest.class), any(AsyncRequestBody.class))).thenReturn(CompletableFuture.completedFuture(response));
    CompletedUpload completedUpload = tm.upload(u -> u.putObjectRequest(p -> p.bucket("bucket").key("key")).requestBody(AsyncRequestBody.fromString("foo"))).completionFuture().join();
    assertThat(completedUpload.response()).isEqualTo(response);
}
Also used : CompletedUpload(software.amazon.awssdk.transfer.s3.CompletedUpload) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) AsyncRequestBody(software.amazon.awssdk.core.async.AsyncRequestBody) PutObjectRequest(software.amazon.awssdk.services.s3.model.PutObjectRequest) Test(org.junit.jupiter.api.Test)

Example 4 with CompletedUpload

use of software.amazon.awssdk.transfer.s3.CompletedUpload in project aws-sdk-java-v2 by aws.

the class UploadDirectoryHelperParameterizedTest method uploadDirectory_maxLengthOne_shouldOnlyUploadTopLevel.

@Test
public void uploadDirectory_maxLengthOne_shouldOnlyUploadTopLevel() {
    ArgumentCaptor<UploadFileRequest> requestArgumentCaptor = ArgumentCaptor.forClass(UploadFileRequest.class);
    when(singleUploadFunction.apply(requestArgumentCaptor.capture())).thenReturn(completedUpload());
    DirectoryUpload uploadDirectory = uploadDirectoryHelper.uploadDirectory(UploadDirectoryRequest.builder().sourceDirectory(directory).bucket("bucket").overrideConfiguration(o -> o.maxDepth(1)).build());
    uploadDirectory.completionFuture().join();
    List<UploadFileRequest> actualRequests = requestArgumentCaptor.getAllValues();
    actualRequests.forEach(r -> assertThat(r.putObjectRequest().bucket()).isEqualTo("bucket"));
    assertThat(actualRequests.size()).isEqualTo(1);
    List<String> keys = actualRequests.stream().map(u -> u.putObjectRequest().key()).collect(Collectors.toList());
    assertThat(keys).containsOnly("bar.txt");
}
Also used : UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) FileUtils(software.amazon.awssdk.testutils.FileUtils) Arrays(java.util.Arrays) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) CompletableFuture(java.util.concurrent.CompletableFuture) UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) Function(java.util.function.Function) Sets(org.assertj.core.util.Sets) FileUpload(software.amazon.awssdk.transfer.s3.FileUpload) DefaultTransferProgressSnapshot(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgressSnapshot) ArgumentCaptor(org.mockito.ArgumentCaptor) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Jimfs(com.google.common.jimfs.Jimfs) After(org.junit.After) UploadDirectoryRequest(software.amazon.awssdk.transfer.s3.UploadDirectoryRequest) Path(java.nio.file.Path) Parameterized(org.junit.runners.Parameterized) Before(org.junit.Before) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) Configuration(com.google.common.jimfs.Configuration) Files(java.nio.file.Files) Collection(java.util.Collection) Set(java.util.Set) DefaultTransferProgress(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgress) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) FileSystem(java.nio.file.FileSystem) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) List(java.util.List) Paths(java.nio.file.Paths) Assume.assumeTrue(org.junit.Assume.assumeTrue) CompletedFileUpload(software.amazon.awssdk.transfer.s3.CompletedFileUpload) IoUtils(software.amazon.awssdk.utils.IoUtils) Mockito.mock(org.mockito.Mockito.mock) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) Test(org.junit.Test)

Example 5 with CompletedUpload

use of software.amazon.awssdk.transfer.s3.CompletedUpload in project aws-sdk-java-v2 by aws.

the class UploadDirectoryHelperParameterizedTest method uploadDirectory_FollowSymlinkTrue_shouldIncludeLinkedFiles.

@Test
public void uploadDirectory_FollowSymlinkTrue_shouldIncludeLinkedFiles() {
    // skip the test if we are using jimfs because it doesn't work well with symlink
    assumeTrue(configuration.equals(Configuration.forCurrentPlatform()));
    ArgumentCaptor<UploadFileRequest> requestArgumentCaptor = ArgumentCaptor.forClass(UploadFileRequest.class);
    when(singleUploadFunction.apply(requestArgumentCaptor.capture())).thenReturn(completedUpload());
    DirectoryUpload uploadDirectory = uploadDirectoryHelper.uploadDirectory(UploadDirectoryRequest.builder().sourceDirectory(directory).bucket("bucket").overrideConfiguration(o -> o.followSymbolicLinks(true)).build());
    uploadDirectory.completionFuture().join();
    List<UploadFileRequest> actualRequests = requestArgumentCaptor.getAllValues();
    actualRequests.forEach(r -> assertThat(r.putObjectRequest().bucket()).isEqualTo("bucket"));
    List<String> keys = actualRequests.stream().map(u -> u.putObjectRequest().key()).collect(Collectors.toList());
    assertThat(keys.size()).isEqualTo(5);
    assertThat(keys).containsOnly("bar.txt", "foo/1.txt", "foo/2.txt", "symlink/2.txt", "symlink2");
}
Also used : UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) FileUtils(software.amazon.awssdk.testutils.FileUtils) Arrays(java.util.Arrays) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) CompletableFuture(java.util.concurrent.CompletableFuture) UploadFileRequest(software.amazon.awssdk.transfer.s3.UploadFileRequest) Function(java.util.function.Function) Sets(org.assertj.core.util.Sets) FileUpload(software.amazon.awssdk.transfer.s3.FileUpload) DefaultTransferProgressSnapshot(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgressSnapshot) ArgumentCaptor(org.mockito.ArgumentCaptor) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Jimfs(com.google.common.jimfs.Jimfs) After(org.junit.After) UploadDirectoryRequest(software.amazon.awssdk.transfer.s3.UploadDirectoryRequest) Path(java.nio.file.Path) Parameterized(org.junit.runners.Parameterized) Before(org.junit.Before) PutObjectResponse(software.amazon.awssdk.services.s3.model.PutObjectResponse) Configuration(com.google.common.jimfs.Configuration) Files(java.nio.file.Files) Collection(java.util.Collection) Set(java.util.Set) DefaultTransferProgress(software.amazon.awssdk.transfer.s3.internal.progress.DefaultTransferProgress) IOException(java.io.IOException) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) FileSystem(java.nio.file.FileSystem) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) List(java.util.List) Paths(java.nio.file.Paths) Assume.assumeTrue(org.junit.Assume.assumeTrue) CompletedFileUpload(software.amazon.awssdk.transfer.s3.CompletedFileUpload) IoUtils(software.amazon.awssdk.utils.IoUtils) Mockito.mock(org.mockito.Mockito.mock) DirectoryUpload(software.amazon.awssdk.transfer.s3.DirectoryUpload) Test(org.junit.Test)

Aggregations

PutObjectResponse (software.amazon.awssdk.services.s3.model.PutObjectResponse)9 CompletableFuture (java.util.concurrent.CompletableFuture)8 Test (org.junit.Test)8 DirectoryUpload (software.amazon.awssdk.transfer.s3.DirectoryUpload)8 UploadFileRequest (software.amazon.awssdk.transfer.s3.UploadFileRequest)8 Configuration (com.google.common.jimfs.Configuration)7 Jimfs (com.google.common.jimfs.Jimfs)7 IOException (java.io.IOException)7 StandardCharsets (java.nio.charset.StandardCharsets)7 FileSystem (java.nio.file.FileSystem)7 Files (java.nio.file.Files)7 Path (java.nio.file.Path)7 Paths (java.nio.file.Paths)7 Arrays (java.util.Arrays)7 Collection (java.util.Collection)7 List (java.util.List)7 Set (java.util.Set)7 Function (java.util.function.Function)7 Collectors (java.util.stream.Collectors)7 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)7