Search in sources :

Example 51 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project crate by crate.

the class S3FileInput method getStream.

@Override
public InputStream getStream(URI uri) throws IOException {
    S3URI s3URI = S3URI.toS3URI(uri);
    if (client == null) {
        client = clientBuilder.client(s3URI, protocolSetting);
    }
    S3Object object = client.getObject(s3URI.bucket(), s3URI.key());
    if (object != null) {
        return object.getObjectContent();
    }
    throw new IOException("Failed to load S3 URI: " + uri.toString());
}
Also used : S3URI(io.crate.copy.s3.common.S3URI) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException)

Example 52 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project ats-framework by Axway.

the class S3Operations method getFileMetadata.

/**
 * Get MD5, size, owner, storage class and last modification time for a desired file in the pointed bucket
 *
 * @param fileName the file name
 */
@PublicAtsApi
public S3ObjectInfo getFileMetadata(String fileName) {
    try {
        S3Object element = s3Client.getObject(bucketName, fileName);
        if (element != null) {
            ObjectMetadata metaData = element.getObjectMetadata();
            S3ObjectInfo s3Info = new S3ObjectInfo();
            s3Info.setBucketName(fileName);
            s3Info.setLastModified(metaData.getLastModified());
            s3Info.setMd5(metaData.getETag());
            s3Info.setName(element.getKey());
            s3Info.setSize(metaData.getContentLength());
            return s3Info;
        } else {
            throw new NoSuchElementException("File with name '" + fileName + "' does not exist!");
        }
    } catch (Exception e) {
        handleExeption(e, "Could not retrieve metadata for S3 object with key '" + fileName + "'");
    }
    return null;
}
Also used : S3Object(com.amazonaws.services.s3.model.S3Object) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) NoSuchElementException(java.util.NoSuchElementException) MultiObjectDeleteException(com.amazonaws.services.s3.model.MultiObjectDeleteException) NoSuchElementException(java.util.NoSuchElementException) AmazonServiceException(com.amazonaws.AmazonServiceException) AmazonClientException(com.amazonaws.AmazonClientException) PublicAtsApi(com.axway.ats.common.PublicAtsApi)

Example 53 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project gradle by gradle.

the class S3ResourceConnector method openResource.

@Override
public ExternalResourceReadResponse openResource(ExternalResourceName location, boolean revalidate) {
    LOGGER.debug("Attempting to get resource: {}", location);
    S3Object s3Object = s3Client.getResource(location.getUri());
    if (s3Object == null) {
        return null;
    }
    return new S3Resource(s3Object, location.getUri());
}
Also used : S3Object(com.amazonaws.services.s3.model.S3Object)

Example 54 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project hippo by NHS-digital-website.

the class S3SdkConnectorTest method retrievesS3Resource.

@Test
public void retrievesS3Resource() throws Exception {
    // given
    // includes negative values which doesn't matter here
    final long contentLength = newRandomLong();
    final S3ObjectInputStream expectedDownloadedFileInputStream = mock(S3ObjectInputStream.class);
    final ObjectMetadata objectMetadata = mock(ObjectMetadata.class);
    given(objectMetadata.getContentLength()).willReturn(contentLength);
    final S3Object s3ResponseObject = mock(S3Object.class);
    given(s3ResponseObject.getObjectContent()).willReturn(expectedDownloadedFileInputStream);
    given(s3ResponseObject.getObjectMetadata()).willReturn(objectMetadata);
    given(s3.getObject(bucketName, objectKey)).willReturn(s3ResponseObject);
    // when
    final S3File actualS3File = s3Connector.downloadFile(objectKey);
    // then
    assertThat("Returned content stream is the one returned by S3.", actualS3File.getContent(), is(sameInstance(expectedDownloadedFileInputStream)));
    assertThat("Returned content length is as returned by S3.", actualS3File.getLength(), is(contentLength));
}
Also used : S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) S3Object(com.amazonaws.services.s3.model.S3Object) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) Test(org.junit.Test)

Example 55 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project athenz by yahoo.

the class AwsPrivateKeyStoreTest method testGetEncryptedDataException.

@Test
public void testGetEncryptedDataException() {
    System.setProperty("athenz.aws.s3.region", "us-east-1");
    System.setProperty(ATHENZ_AWS_KMS_REGION, "us-east-1");
    String bucketName = "my_bucket";
    String keyName = "my_key";
    String expected = "my_value";
    AmazonS3 s3 = mock(AmazonS3.class);
    AWSKMS kms = mock(AWSKMS.class);
    S3Object s3Object = mock(S3Object.class);
    Mockito.when(s3.getObject(bucketName, keyName)).thenReturn(s3Object);
    given(s3Object.getObjectContent()).willAnswer(invocation -> {
        throw new IOException();
    });
    ByteBuffer buffer = ByteBuffer.wrap(expected.getBytes());
    DecryptResult decryptResult = mock(DecryptResult.class);
    Mockito.when(kms.decrypt(Mockito.any(DecryptRequest.class))).thenReturn(decryptResult);
    Mockito.when(decryptResult.getPlaintext()).thenReturn(buffer);
    System.setProperty("athenz.aws.store_kms_decrypt", "true");
    AwsPrivateKeyStore awsPrivateKeyStore = new AwsPrivateKeyStore();
    AwsPrivateKeyStore spyAWS = Mockito.spy(awsPrivateKeyStore);
    doReturn(s3).when(spyAWS).getS3();
    doReturn(kms).when(spyAWS).getKMS();
    assertEquals(spyAWS.getKMS(), kms);
    System.clearProperty("athenz.aws.s3.region");
    System.clearProperty(ATHENZ_AWS_KMS_REGION);
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) DecryptResult(com.amazonaws.services.kms.model.DecryptResult) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException) ByteBuffer(java.nio.ByteBuffer) DecryptRequest(com.amazonaws.services.kms.model.DecryptRequest) AWSKMS(com.amazonaws.services.kms.AWSKMS) Test(org.testng.annotations.Test)

Aggregations

S3Object (com.amazonaws.services.s3.model.S3Object)110 S3ObjectInputStream (com.amazonaws.services.s3.model.S3ObjectInputStream)34 InputStream (java.io.InputStream)28 IOException (java.io.IOException)24 GetObjectRequest (com.amazonaws.services.s3.model.GetObjectRequest)23 ByteArrayInputStream (java.io.ByteArrayInputStream)21 AmazonServiceException (com.amazonaws.AmazonServiceException)20 AmazonS3 (com.amazonaws.services.s3.AmazonS3)20 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)18 Test (org.junit.Test)18 Test (org.testng.annotations.Test)18 S3Object (software.amazon.awssdk.services.s3.model.S3Object)17 File (java.io.File)14 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)13 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)13 FileInputStream (java.io.FileInputStream)13 Date (java.util.Date)11 SignedDomain (com.yahoo.athenz.zms.SignedDomain)10 ListObjectsV2Response (software.amazon.awssdk.services.s3.model.ListObjectsV2Response)9 AmazonClientException (com.amazonaws.AmazonClientException)8