Search in sources :

Example 6 with ListObjectsResponse

use of software.amazon.awssdk.services.s3.model.ListObjectsResponse in project aws-doc-sdk-examples by awsdocs.

the class VPCS3Example method listBucketObjects.

// snippet-start:[s3.java2.vpc.example.main]
public static void listBucketObjects(S3Client s3, String bucketName) {
    try {
        ListObjectsRequest listObjects = ListObjectsRequest.builder().bucket(bucketName).build();
        ListObjectsResponse res = s3.listObjects(listObjects);
        List<S3Object> objects = res.contents();
        for (S3Object s3Object : objects) {
            System.out.print("\n The name of the key is " + s3Object.key());
            System.out.print("\n The object is " + convertBToKb(s3Object.size()) + " KBs");
            System.out.print("\n The owner is " + s3Object.owner());
        }
    } catch (S3Exception e) {
        System.err.println(e.awsErrorDetails().errorMessage());
        System.exit(1);
    }
}
Also used : ListObjectsRequest(software.amazon.awssdk.services.s3.model.ListObjectsRequest) S3Exception(software.amazon.awssdk.services.s3.model.S3Exception) ListObjectsResponse(software.amazon.awssdk.services.s3.model.ListObjectsResponse) S3Object(software.amazon.awssdk.services.s3.model.S3Object)

Example 7 with ListObjectsResponse

use of software.amazon.awssdk.services.s3.model.ListObjectsResponse in project beam by apache.

the class S3FileSystemTest method matchVariousInvokeThreadPool.

@Test
public void matchVariousInvokeThreadPool() throws IOException {
    S3FileSystem s3FileSystem = buildMockedS3FileSystem(s3Config("mys3"));
    SdkServiceException notFoundException = S3Exception.builder().message("mock exception").statusCode(404).build();
    S3ResourceId pathNotExist = S3ResourceId.fromUri("mys3://testbucket/testdirectory/nonexistentfile");
    HeadObjectRequest headObjectRequestNotExist = HeadObjectRequest.builder().bucket(pathNotExist.getBucket()).key(pathNotExist.getKey()).build();
    when(s3FileSystem.getS3Client().headObject(argThat(new GetHeadObjectRequestMatcher(headObjectRequestNotExist)))).thenThrow(notFoundException);
    SdkServiceException forbiddenException = SdkServiceException.builder().message("mock exception").statusCode(403).build();
    S3ResourceId pathForbidden = S3ResourceId.fromUri("mys3://testbucket/testdirectory/forbiddenfile");
    HeadObjectRequest headObjectRequestForbidden = HeadObjectRequest.builder().bucket(pathForbidden.getBucket()).key(pathForbidden.getKey()).build();
    when(s3FileSystem.getS3Client().headObject(argThat(new GetHeadObjectRequestMatcher(headObjectRequestForbidden)))).thenThrow(forbiddenException);
    S3ResourceId pathExist = S3ResourceId.fromUri("mys3://testbucket/testdirectory/filethatexists");
    HeadObjectRequest headObjectRequestExist = HeadObjectRequest.builder().bucket(pathExist.getBucket()).key(pathExist.getKey()).build();
    HeadObjectResponse s3ObjectMetadata = HeadObjectResponse.builder().contentLength(100L).contentEncoding("not-gzip").lastModified(Instant.ofEpochMilli(1540000000000L)).build();
    when(s3FileSystem.getS3Client().headObject(argThat(new GetHeadObjectRequestMatcher(headObjectRequestExist)))).thenReturn(s3ObjectMetadata);
    S3ResourceId pathGlob = S3ResourceId.fromUri("mys3://testbucket/path/part*");
    S3Object foundListObject = S3Object.builder().key("path/part-0").size(200L).lastModified(Instant.ofEpochMilli(1541000000000L)).build();
    ListObjectsV2Response listObjectsResponse = ListObjectsV2Response.builder().continuationToken(null).contents(foundListObject).build();
    when(s3FileSystem.getS3Client().listObjectsV2((ListObjectsV2Request) notNull())).thenReturn(listObjectsResponse);
    HeadObjectResponse headObjectResponse = HeadObjectResponse.builder().contentEncoding("").build();
    when(s3FileSystem.getS3Client().headObject(argThat(new GetHeadObjectRequestMatcher(HeadObjectRequest.builder().bucket(pathGlob.getBucket()).key("path/part-0").build())))).thenReturn(headObjectResponse);
    assertThat(s3FileSystem.match(ImmutableList.of(pathNotExist.toString(), pathForbidden.toString(), pathExist.toString(), pathGlob.toString())), contains(MatchResultMatcher.create(MatchResult.Status.NOT_FOUND, new FileNotFoundException()), MatchResultMatcher.create(MatchResult.Status.ERROR, new IOException(forbiddenException)), MatchResultMatcher.create(100, 1540000000000L, pathExist, true), MatchResultMatcher.create(200, 1541000000000L, S3ResourceId.fromComponents("mys3", pathGlob.getBucket(), foundListObject.key()), true)));
}
Also used : SdkServiceException(software.amazon.awssdk.core.exception.SdkServiceException) HeadObjectResponse(software.amazon.awssdk.services.s3.model.HeadObjectResponse) FileNotFoundException(java.io.FileNotFoundException) HeadObjectRequest(software.amazon.awssdk.services.s3.model.HeadObjectRequest) S3Object(software.amazon.awssdk.services.s3.model.S3Object) IOException(java.io.IOException) ListObjectsV2Response(software.amazon.awssdk.services.s3.model.ListObjectsV2Response) S3TestUtils.buildMockedS3FileSystem(org.apache.beam.sdk.io.aws2.s3.S3TestUtils.buildMockedS3FileSystem) Test(org.junit.Test)

Aggregations

ListObjectsRequest (software.amazon.awssdk.services.s3.model.ListObjectsRequest)6 ListObjectsResponse (software.amazon.awssdk.services.s3.model.ListObjectsResponse)6 S3Object (software.amazon.awssdk.services.s3.model.S3Object)6 S3Exception (software.amazon.awssdk.services.s3.model.S3Exception)4 S3Client (software.amazon.awssdk.services.s3.S3Client)3 ArrayList (java.util.ArrayList)2 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1 ListIterator (java.util.ListIterator)1 S3TestUtils.buildMockedS3FileSystem (org.apache.beam.sdk.io.aws2.s3.S3TestUtils.buildMockedS3FileSystem)1 Test (org.junit.Test)1 AwsSessionCredentials (software.amazon.awssdk.auth.credentials.AwsSessionCredentials)1 SdkServiceException (software.amazon.awssdk.core.exception.SdkServiceException)1 Region (software.amazon.awssdk.regions.Region)1 HeadObjectRequest (software.amazon.awssdk.services.s3.model.HeadObjectRequest)1 HeadObjectResponse (software.amazon.awssdk.services.s3.model.HeadObjectResponse)1 ListObjectsV2Response (software.amazon.awssdk.services.s3.model.ListObjectsV2Response)1 StsClient (software.amazon.awssdk.services.sts.StsClient)1 AssumeRoleRequest (software.amazon.awssdk.services.sts.model.AssumeRoleRequest)1 AssumeRoleResponse (software.amazon.awssdk.services.sts.model.AssumeRoleResponse)1