Search in sources :

Example 81 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project athenz by yahoo.

the class S3ChangeLogStoreTest method testGetSignedDomainInternal.

@Test
public void testGetSignedDomainInternal() throws IOException {
    MockS3ChangeLogStore store = new MockS3ChangeLogStore(null);
    InputStream is = new FileInputStream("src/test/resources/iaas.json");
    MockS3ObjectInputStream s3Is = new MockS3ObjectInputStream(is, null);
    S3Object object = mock(S3Object.class);
    when(object.getObjectContent()).thenReturn(s3Is);
    when(store.awsS3Client.getObject("athenz-domain-sys.auth", "iaas")).thenReturn(object);
    SignedDomain signedDomain = store.getSignedDomain(store.awsS3Client, "iaas");
    assertNotNull(signedDomain);
    DomainData domainData = signedDomain.getDomain();
    assertNotNull(domainData);
    assertEquals(domainData.getName(), "iaas");
    is.close();
}
Also used : FileInputStream(java.io.FileInputStream) S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) InputStream(java.io.InputStream) SignedDomain(com.yahoo.athenz.zms.SignedDomain) DomainData(com.yahoo.athenz.zms.DomainData) S3Object(com.amazonaws.services.s3.model.S3Object) FileInputStream(java.io.FileInputStream) Test(org.testng.annotations.Test)

Example 82 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project syndesis-qe by syndesisio.

the class S3Utils method readTextFileContentFromBucket.

/**
 * Gets specified text file content from specified S3 bucket.
 *
 * @param bucketName
 * @param fileName
 * @return
 */
public String readTextFileContentFromBucket(String bucketName, String fileName) {
    final S3Object s3object = s3client.getObject(bucketName, fileName);
    final S3ObjectInputStream inputStream = s3object.getObjectContent();
    final StringWriter writer = new StringWriter();
    try {
        IOUtils.copy(inputStream, writer, "UTF-8");
    } catch (IOException ex) {
        log.error("Error copying file from s3: " + ex);
    }
    return writer.toString();
}
Also used : StringWriter(java.io.StringWriter) S3ObjectInputStream(com.amazonaws.services.s3.model.S3ObjectInputStream) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException)

Example 83 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project Synapse-Stack-Builder by Sage-Bionetworks.

the class SSLSetup method getCertificateStringFromS3.

/**
 * Download a certificate file from S3 directly into a string, skipping an intermediate file.
 * @param key
 * @return
 */
public String getCertificateStringFromS3(String key) {
    // For this case we do not write to file first
    S3Object s3Object = s3Client.getObject(new GetObjectRequest(config.getDefaultS3BucketName(), key));
    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    try {
        byte[] buffer = new byte[1024 * 10];
        int bytesRead;
        while ((bytesRead = s3Object.getObjectContent().read(buffer)) > -1) {
            outputStream.write(buffer, 0, bytesRead);
        }
        // Go right to string.
        return new String(outputStream.toByteArray(), "UTF-8");
    } catch (IOException e) {
        s3Object.getObjectContent().abort();
        throw new AmazonClientException("Unable to store object contents to disk: " + e.getMessage(), e);
    } finally {
        try {
            outputStream.close();
        } catch (Exception e) {
        }
        try {
            s3Object.getObjectContent().close();
        } catch (Exception e) {
        }
    }
}
Also used : AmazonClientException(com.amazonaws.AmazonClientException) S3Object(com.amazonaws.services.s3.model.S3Object) ByteArrayOutputStream(java.io.ByteArrayOutputStream) IOException(java.io.IOException) GetObjectRequest(com.amazonaws.services.s3.model.GetObjectRequest) IOException(java.io.IOException) AmazonClientException(com.amazonaws.AmazonClientException)

Example 84 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project carina by qaprosoft.

the class AbstractTest method updateS3AppPath.

/**
 * Method to update MOBILE_APP path in case if apk is located in s3 bucket.
 */
private void updateS3AppPath() {
    Pattern S3_BUCKET_PATTERN = Pattern.compile("s3:\\/\\/([a-zA-Z-0-9][^\\/]*)\\/(.*)");
    // get app path to be sure that we need(do not need) to download app from s3 bucket
    String mobileAppPath = Configuration.getMobileApp();
    Matcher matcher = S3_BUCKET_PATTERN.matcher(mobileAppPath);
    LOGGER.info("Analyzing if mobile app is located on S3...");
    if (matcher.find()) {
        LOGGER.info("app artifact is located on s3...");
        String bucketName = matcher.group(1);
        String key = matcher.group(2);
        Pattern pattern = Pattern.compile(key);
        // analyze if we have any pattern inside mobile_app to make extra
        // search in AWS
        int position = key.indexOf(".*");
        if (position > 0) {
            // /android/develop/dfgdfg.*/Mapmyrun.apk
            int slashPosition = key.substring(0, position).lastIndexOf("/");
            if (slashPosition > 0) {
                key = key.substring(0, slashPosition);
                S3ObjectSummary lastBuild = AmazonS3Manager.getInstance().getLatestBuildArtifact(bucketName, key, pattern);
                key = lastBuild.getKey();
            }
        }
        S3Object objBuild = AmazonS3Manager.getInstance().get(bucketName, key);
        String s3LocalStorage = Configuration.get(Parameter.S3_LOCAL_STORAGE);
        // download file from AWS to local storage
        String fileName = s3LocalStorage + "/" + StringUtils.substringAfterLast(objBuild.getKey(), "/");
        File file = new File(fileName);
        // download
        if (file.exists() && file.length() == objBuild.getObjectMetadata().getContentLength()) {
            LOGGER.info("build artifact with the same size already downloaded: " + file.getAbsolutePath());
        } else {
            LOGGER.info(String.format("Following data was extracted: bucket: %s, key: %s, local file: %s", bucketName, key, file.getAbsolutePath()));
            AmazonS3Manager.getInstance().download(bucketName, key, new File(fileName));
        }
        Configuration.setMobileApp(file.getAbsolutePath());
        // try to redefine app_version if it's value is latest or empty
        String appVersion = Configuration.get(Parameter.APP_VERSION);
        if (appVersion.equals("latest") || appVersion.isEmpty()) {
            R.CONFIG.put(Parameter.APP_VERSION.getKey(), file.getName());
        }
    }
}
Also used : Pattern(java.util.regex.Pattern) Matcher(java.util.regex.Matcher) S3ObjectSummary(com.amazonaws.services.s3.model.S3ObjectSummary) S3Object(com.amazonaws.services.s3.model.S3Object) File(java.io.File)

Example 85 with S3Object

use of software.amazon.awssdk.services.s3.model.S3Object in project herd by FINRAOS.

the class MockS3OperationsImpl method getS3Object.

@Override
public S3Object getS3Object(GetObjectRequest getObjectRequest, AmazonS3 s3) {
    MockS3Object mockS3Object = getMockS3Object(getObjectRequest.getBucketName(), getObjectRequest.getKey());
    S3Object s3Object = new S3Object();
    s3Object.setBucketName(getObjectRequest.getBucketName());
    s3Object.setKey(getObjectRequest.getKey());
    s3Object.setObjectContent(new ByteArrayInputStream(mockS3Object.getData()));
    s3Object.setObjectMetadata(mockS3Object.getObjectMetadata());
    return s3Object;
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) S3Object(com.amazonaws.services.s3.model.S3Object)

Aggregations

S3Object (com.amazonaws.services.s3.model.S3Object)110 S3ObjectInputStream (com.amazonaws.services.s3.model.S3ObjectInputStream)34 InputStream (java.io.InputStream)28 IOException (java.io.IOException)24 GetObjectRequest (com.amazonaws.services.s3.model.GetObjectRequest)23 ByteArrayInputStream (java.io.ByteArrayInputStream)21 AmazonServiceException (com.amazonaws.AmazonServiceException)20 AmazonS3 (com.amazonaws.services.s3.AmazonS3)20 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)18 Test (org.junit.Test)18 Test (org.testng.annotations.Test)18 S3Object (software.amazon.awssdk.services.s3.model.S3Object)17 File (java.io.File)14 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)13 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)13 FileInputStream (java.io.FileInputStream)13 Date (java.util.Date)11 SignedDomain (com.yahoo.athenz.zms.SignedDomain)10 ListObjectsV2Response (software.amazon.awssdk.services.s3.model.ListObjectsV2Response)9 AmazonClientException (com.amazonaws.AmazonClientException)8