Search in sources :

Example 61 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project athenz by yahoo.

the class S3ChangeLogStoreTest method testGetJWSDomainException.

@Test
public void testGetJWSDomainException() throws IOException {
    MockS3ChangeLogStore store = new MockS3ChangeLogStore();
    InputStream is = new FileInputStream("src/test/resources/iaas.jws");
    MockS3ObjectInputStream s3Is = new MockS3ObjectInputStream(is, null);
    S3Object object = mock(S3Object.class);
    when(object.getObjectContent()).thenReturn(s3Is);
    // first call we return null, second call we return success
    when(store.awsS3Client.getObject("s3-unit-test-bucket-name", "iaas")).thenThrow(new AmazonServiceException("test")).thenReturn(object);
    JWSDomain jwsDomain = store.getLocalJWSDomain("iaas");
    assertNotNull(jwsDomain);
    is.close();
}
Also used : S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) JWSDomain(com.yahoo.athenz.zms.JWSDomain) AmazonServiceException(com.amazonaws.AmazonServiceException) S3Object(com.amazonaws.services.s3.model.S3Object) Test(org.testng.annotations.Test)

Example 62 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project athenz by yahoo.

the class S3ChangeLogStoreTest method testGetUpdatedSignedDomainsWithChange.

@Test
public void testGetUpdatedSignedDomainsWithChange() throws IOException {
    MockS3ChangeLogStore store = new MockS3ChangeLogStore();
    ArrayList<S3ObjectSummary> objectList = new ArrayList<>();
    S3ObjectSummary objectSummary = new S3ObjectSummary();
    objectSummary.setKey("iaas");
    objectSummary.setLastModified(new Date(100));
    objectList.add(objectSummary);
    objectSummary = new S3ObjectSummary();
    objectSummary.setKey("iaas.athenz");
    objectSummary.setLastModified(new Date(200));
    objectList.add(objectSummary);
    // we'll also include an invalid domain that should be skipped
    objectSummary = new S3ObjectSummary();
    objectSummary.setKey("unknown");
    objectSummary.setLastModified(new Date(200));
    objectList.add(objectSummary);
    ObjectListing objectListing = mock(ObjectListing.class);
    when(objectListing.getObjectSummaries()).thenReturn(objectList);
    when(objectListing.isTruncated()).thenReturn(false);
    when(store.awsS3Client.listObjects(any(ListObjectsRequest.class))).thenReturn(objectListing);
    InputStream is = new FileInputStream("src/test/resources/iaas.json");
    MockS3ObjectInputStream s3Is = new MockS3ObjectInputStream(is, null);
    S3Object object = mock(S3Object.class);
    when(object.getObjectContent()).thenReturn(s3Is);
    when(store.awsS3Client.getObject("s3-unit-test-bucket-name", "iaas")).thenReturn(object);
    when(store.awsS3Client.getObject("s3-unit-test-bucket-name", "iaas.athenz")).thenReturn(object);
    // set the last modification time to return one of the domains
    store.lastModTime = (new Date(150)).getTime();
    StringBuilder lastModTimeBuffer = new StringBuilder(512);
    SignedDomains signedDomains = store.getUpdatedSignedDomains(lastModTimeBuffer);
    assertTrue(lastModTimeBuffer.length() > 0);
    List<SignedDomain> domainList = signedDomains.getDomains();
    assertEquals(domainList.size(), 1);
    DomainData domainData = domainList.get(0).getDomain();
    assertNotNull(domainData);
    assertEquals(domainData.getName(), "iaas");
    is.close();
}
Also used : S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) DomainData(com.yahoo.athenz.zms.DomainData) ObjectListing(com.amazonaws.services.s3.model.ObjectListing) S3ObjectSummary(com.amazonaws.services.s3.model.S3ObjectSummary) SignedDomains(com.yahoo.athenz.zms.SignedDomains) ListObjectsRequest(com.amazonaws.services.s3.model.ListObjectsRequest) SignedDomain(com.yahoo.athenz.zms.SignedDomain) S3Object(com.amazonaws.services.s3.model.S3Object) Test(org.testng.annotations.Test)

Example 63 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project gradle-s3-build-cache by myniva.

the class AwsS3BuildCacheService method load.

@Override
public boolean load(BuildCacheKey key, BuildCacheEntryReader reader) {
    final String bucketPath = getBucketPath(key);
    if (s3.doesObjectExist(bucketName, bucketPath)) {
        logger.info("Found cache item '{}' in S3 bucket", bucketPath);
        S3Object object = s3.getObject(bucketName, bucketPath);
        try (InputStream is = object.getObjectContent()) {
            reader.readFrom(is);
            return true;
        } catch (IOException e) {
            throw new BuildCacheException("Error while reading cache object from S3 bucket", e);
        }
    } else {
        logger.info("Did not find cache item '{}' in S3 bucket", bucketPath);
        return false;
    }
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) BuildCacheException(org.gradle.caching.BuildCacheException) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException)

Example 64 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project gradle-s3-build-cache by myniva.

the class AwsS3BuildCacheServiceTest method loadGetsObjectsAndReturnsTrueIfItExistsInS3.

@Test
public void loadGetsObjectsAndReturnsTrueIfItExistsInS3() throws Exception {
    /**
     * Setup *
     */
    buildCacheService = new AwsS3BuildCacheService(s3, "bucketName", null, true);
    doReturn(true).when(s3).doesObjectExist("bucketName", "abcdefghijkl123456789");
    S3Object s3Object = mock(S3Object.class);
    doReturn(s3Object).when(s3).getObject("bucketName", "abcdefghijkl123456789");
    S3ObjectInputStream s3ObjectInputStream = mock(S3ObjectInputStream.class);
    doReturn(s3ObjectInputStream).when(s3Object).getObjectContent();
    /**
     * Run *
     */
    boolean result = buildCacheService.load(key, reader);
    /**
     * Check *
     */
    assertTrue(result);
    verify(reader).readFrom(s3ObjectInputStream);
}
Also used : S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) S3Object(com.amazonaws.services.s3.model.S3Object) Test(org.junit.Test)

Example 65 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project pentaho-kettle by pentaho.

the class S3ObjectsProviderTest method buildS3Object.

private static S3Object buildS3Object(Bucket bucket, String key, String dataString) {
    S3Object s3Object = new S3Object();
    s3Object.setKey(key);
    s3Object.setBucketName(bucket.getName());
    s3Object.setObjectContent(new ByteArrayInputStream(dataString.getBytes()));
    return s3Object;
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) S3Object(com.amazonaws.services.s3.model.S3Object)

Aggregations

S3Object (com.amazonaws.services.s3.model.S3Object)110 S3ObjectInputStream (com.amazonaws.services.s3.model.S3ObjectInputStream)34 InputStream (java.io.InputStream)28 IOException (java.io.IOException)24 GetObjectRequest (com.amazonaws.services.s3.model.GetObjectRequest)23 ByteArrayInputStream (java.io.ByteArrayInputStream)21 AmazonServiceException (com.amazonaws.AmazonServiceException)20 AmazonS3 (com.amazonaws.services.s3.AmazonS3)20 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)18 Test (org.junit.Test)18 Test (org.testng.annotations.Test)18 S3Object (software.amazon.awssdk.services.s3.model.S3Object)17 File (java.io.File)14 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)13 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)13 FileInputStream (java.io.FileInputStream)13 Date (java.util.Date)11 SignedDomain (com.yahoo.athenz.zms.SignedDomain)10 ListObjectsV2Response (software.amazon.awssdk.services.s3.model.ListObjectsV2Response)9 AmazonClientException (com.amazonaws.AmazonClientException)8