Search in sources :

Example 96 with AmazonS3Client

use of com.amazonaws.services.s3.AmazonS3Client in project SimpleContentPortlet by Jasig.

the class AmazonS3PersistenceStrategy method persistAttachmentBinary.

@Override
public String persistAttachmentBinary(HttpServletRequest request, Attachment attachment) throws PersistenceException {
    // The data import is only going to update the attachment metadata in the database.
    if (request == null && attachment.getData() == null) {
        return null;
    }
    AmazonS3 s3 = new AmazonS3Client();
    String key = PATH_FORMAT.format(new Object[] { s3BucketPath, attachment.getGuid(), attachment.getFilename() });
    ObjectMetadata metadata = new ObjectMetadata();
    metadata.setContentType(attachment.getContentType());
    metadata.setContentLength(attachment.getData().length);
    metadata.setCacheControl(s3CacheControlString);
    // S3 chose base64-encoded hash not the typical 32-character hex string so convert accordingly.
    // Hex.decodeHex(attachment.getChecksum().toCharArray())
    metadata.setContentMD5(Base64.encodeBase64String(DatatypeConverter.parseHexBinary(attachment.getChecksum())));
    try {
        s3.putObject(new PutObjectRequest(s3BucketName, key, new ByteArrayInputStream(attachment.getData()), metadata));
        log.debug("Successfully sent {} to S3 bucket {} under key {}", attachment.getFilename(), s3BucketName, key);
    } catch (AmazonClientException e) {
        String message = String.format("Unable to persist attachment %1s to S3 bucket %2s, key %3s", attachment.getFilename(), s3BucketName, key);
        throw new PersistenceException(message, e);
    }
    return (s3BucketBaseUrl.endsWith("/") ? s3BucketBaseUrl : s3BucketBaseUrl + "/") + key;
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3) AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) ByteArrayInputStream(java.io.ByteArrayInputStream) AmazonClientException(com.amazonaws.AmazonClientException) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata) PutObjectRequest(com.amazonaws.services.s3.model.PutObjectRequest)

Example 97 with AmazonS3Client

use of com.amazonaws.services.s3.AmazonS3Client in project bender by Nextdoor.

the class BaseHandlerS3Test method setup.

@Before
public void setup() throws UnsupportedEncodingException, IOException {
    /*
     * Patch the handler to use this test's factory which produces a mock client.
     */
    S3MockClientFactory f;
    try {
        f = new S3MockClientFactory(tmpFolder);
    } catch (Exception e) {
        throw new RuntimeException("unable to start s3proxy", e);
    }
    AmazonS3Client client = f.newInstance();
    client.createBucket(S3_BUCKET);
    this.clientFactory = f;
    /*
     * Upload config file
     */
    String payload = IOUtils.toString(new InputStreamReader(this.getClass().getResourceAsStream("/config/handler_config.json"), "UTF-8"));
    client.putObject(S3_BUCKET, "bender/config.json", payload);
    /*
     * Export config file as env var
     */
    envVars.set("BENDER_CONFIG", "s3://" + S3_BUCKET + "/bender/config.json");
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) InputStreamReader(java.io.InputStreamReader) S3MockClientFactory(com.nextdoor.bender.aws.S3MockClientFactory) IOException(java.io.IOException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) Before(org.junit.Before)

Example 98 with AmazonS3Client

use of com.amazonaws.services.s3.AmazonS3Client in project bender by Nextdoor.

the class GeoIpOperationFactory method setConf.

@Override
public void setConf(AbstractConfig config) {
    this.config = (GeoIpOperationConfig) config;
    AmazonS3Client client = this.s3Factory.newInstance();
    AmazonS3URI uri = new AmazonS3URI(this.config.getGeoLiteDb());
    GetObjectRequest req = new GetObjectRequest(uri.getBucket(), uri.getKey());
    S3Object obj = client.getObject(req);
    try {
        this.databaseReader = new DatabaseReader.Builder(obj.getObjectContent()).withCache(new CHMCache()).build();
    } catch (IOException e) {
        throw new ConfigurationException("Unable to read " + this.config.getGeoLiteDb(), e);
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) ConfigurationException(com.nextdoor.bender.config.ConfigurationException) CHMCache(com.maxmind.db.CHMCache) DatabaseReader(com.maxmind.geoip2.DatabaseReader) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException) GetObjectRequest(com.amazonaws.services.s3.model.GetObjectRequest) AmazonS3URI(com.amazonaws.services.s3.AmazonS3URI)

Example 99 with AmazonS3Client

use of com.amazonaws.services.s3.AmazonS3Client in project bender by Nextdoor.

the class S3TransportFactory method setConf.

@Override
public void setConf(AbstractConfig config) {
    this.config = (S3TransportConfig) config;
    this.client = new AmazonS3Client();
    if (this.config.getRegion() != null) {
        this.client.withRegion(this.config.getRegion());
    }
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client)

Example 100 with AmazonS3Client

use of com.amazonaws.services.s3.AmazonS3Client in project bender by Nextdoor.

the class BenderConfig method load.

public static BenderConfig load(AmazonS3ClientFactory s3ClientFactory, AmazonS3URI s3Uri) {
    AmazonS3Client s3 = s3ClientFactory.newInstance();
    S3Object s3object = s3.getObject(s3Uri.getBucket(), s3Uri.getKey());
    StringWriter writer = new StringWriter();
    try {
        IOUtils.copy(s3object.getObjectContent(), writer, "UTF-8");
    } catch (IOException e) {
        throw new ConfigurationException("Unable to read file from s3", e);
    }
    BenderConfig config = load(s3Uri.getKey().toString(), writer.toString());
    config.setConfigFile(s3Uri.getURI().toString());
    return config;
}
Also used : AmazonS3Client(com.amazonaws.services.s3.AmazonS3Client) StringWriter(java.io.StringWriter) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException)

Aggregations

AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)109 Test (org.junit.Test)23 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)20 AmazonClientException (com.amazonaws.AmazonClientException)18 ClientConfiguration (com.amazonaws.ClientConfiguration)18 ArrayList (java.util.ArrayList)14 AWSCredentialsProvider (com.amazonaws.auth.AWSCredentialsProvider)13 HashMap (java.util.HashMap)13 AmazonS3 (com.amazonaws.services.s3.AmazonS3)12 File (java.io.File)12 InvocationOnMock (org.mockito.invocation.InvocationOnMock)12 PutObjectResult (com.amazonaws.services.s3.model.PutObjectResult)11 UploadPartRequest (com.amazonaws.services.s3.model.UploadPartRequest)11 AWSCredentials (com.amazonaws.auth.AWSCredentials)10 AmazonS3Exception (com.amazonaws.services.s3.model.AmazonS3Exception)10 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)10 S3Object (com.amazonaws.services.s3.model.S3Object)9 InternalEvent (com.nextdoor.bender.InternalEvent)9 TestContext (com.nextdoor.bender.aws.TestContext)9 IOException (java.io.IOException)9