Search in sources :

Example 1 with S3Object

use of com.amazonaws.services.s3.model.S3Object in project camel by apache.

the class AmazonS3ClientMock method putObject.

@SuppressWarnings("resource")
@Override
public PutObjectResult putObject(PutObjectRequest putObjectRequest) throws AmazonClientException, AmazonServiceException {
    putObjectRequests.add(putObjectRequest);
    S3Object s3Object = new S3Object();
    s3Object.setBucketName(putObjectRequest.getBucketName());
    s3Object.setKey(putObjectRequest.getKey());
    if (putObjectRequest.getFile() != null) {
        try {
            s3Object.setObjectContent(new FileInputStream(putObjectRequest.getFile()));
        } catch (FileNotFoundException e) {
            throw new AmazonServiceException("Cannot store the file object.", e);
        }
    } else {
        s3Object.setObjectContent(putObjectRequest.getInputStream());
    }
    objects.add(s3Object);
    PutObjectResult putObjectResult = new PutObjectResult();
    putObjectResult.setETag("3a5c8b1ad448bca04584ecb55b836264");
    return putObjectResult;
}
Also used : PutObjectResult(com.amazonaws.services.s3.model.PutObjectResult) FileNotFoundException(java.io.FileNotFoundException) AmazonServiceException(com.amazonaws.AmazonServiceException) S3Object(com.amazonaws.services.s3.model.S3Object) FileInputStream(java.io.FileInputStream)

Example 2 with S3Object

use of com.amazonaws.services.s3.model.S3Object in project camel by apache.

the class S3BatchConsumerMaxMessagesPerPollTest method createRegistry.

@Override
protected JndiRegistry createRegistry() throws Exception {
    JndiRegistry registry = super.createRegistry();
    AmazonS3ClientMock clientMock = new AmazonS3ClientMock();
    // add 20 messages
    for (int counter = 0; counter < 20; counter++) {
        S3Object s3Object = new S3Object();
        s3Object.setBucketName("mycamelbucket");
        s3Object.setKey("counter-" + counter);
        clientMock.objects.add(s3Object);
    }
    registry.bind("amazonS3Client", clientMock);
    return registry;
}
Also used : JndiRegistry(org.apache.camel.impl.JndiRegistry) S3Object(com.amazonaws.services.s3.model.S3Object) MockEndpoint(org.apache.camel.component.mock.MockEndpoint)

Example 3 with S3Object

use of com.amazonaws.services.s3.model.S3Object in project zeppelin by apache.

the class S3NotebookRepo method getNote.

private Note getNote(String key) throws IOException {
    GsonBuilder gsonBuilder = new GsonBuilder();
    gsonBuilder.setPrettyPrinting();
    Gson gson = gsonBuilder.registerTypeAdapter(Date.class, new NotebookImportDeserializer()).create();
    S3Object s3object;
    try {
        s3object = s3client.getObject(new GetObjectRequest(bucketName, key));
    } catch (AmazonClientException ace) {
        throw new IOException("Unable to retrieve object from S3: " + ace, ace);
    }
    Note note;
    try (InputStream ins = s3object.getObjectContent()) {
        String json = IOUtils.toString(ins, conf.getString(ConfVars.ZEPPELIN_ENCODING));
        note = gson.fromJson(json, Note.class);
    }
    for (Paragraph p : note.getParagraphs()) {
        if (p.getStatus() == Status.PENDING || p.getStatus() == Status.RUNNING) {
            p.setStatus(Status.ABORT);
        }
    }
    return note;
}
Also used : GsonBuilder(com.google.gson.GsonBuilder) InputStream(java.io.InputStream) AmazonClientException(com.amazonaws.AmazonClientException) Note(org.apache.zeppelin.notebook.Note) Gson(com.google.gson.Gson) S3Object(com.amazonaws.services.s3.model.S3Object) IOException(java.io.IOException) NotebookImportDeserializer(org.apache.zeppelin.notebook.NotebookImportDeserializer) GetObjectRequest(com.amazonaws.services.s3.model.GetObjectRequest) Date(java.util.Date) Paragraph(org.apache.zeppelin.notebook.Paragraph)

Example 4 with S3Object

use of com.amazonaws.services.s3.model.S3Object in project deeplearning4j by deeplearning4j.

the class S3Downloader method objectForKey.

/**
     * Returns an input stream for the given bucket and key
     * @param bucket the bucket to retrieve from
     * @param key the key of the objec  t
     * @return an input stream to the object
     */
public InputStream objectForKey(String bucket, String key) {
    AmazonS3 s3 = getClient();
    S3Object obj = s3.getObject(bucket, key);
    InputStream is = obj.getObjectContent();
    return is;
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3)

Example 5 with S3Object

use of com.amazonaws.services.s3.model.S3Object in project deeplearning4j by deeplearning4j.

the class S3Downloader method download.

public void download(String bucket, String key, OutputStream to) throws IOException {
    AmazonS3 s3 = getClient();
    S3Object obj = s3.getObject(bucket, key);
    InputStream is = obj.getObjectContent();
    BufferedOutputStream bos = new BufferedOutputStream(to);
    IOUtils.copy(is, bos);
    bos.close();
    is.close();
    obj.close();
}
Also used : AmazonS3(com.amazonaws.services.s3.AmazonS3)

Aggregations

S3Object (com.amazonaws.services.s3.model.S3Object)22 InputStream (java.io.InputStream)8 AmazonServiceException (com.amazonaws.AmazonServiceException)7 GetObjectRequest (com.amazonaws.services.s3.model.GetObjectRequest)7 AmazonS3 (com.amazonaws.services.s3.AmazonS3)6 ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)4 IOException (java.io.IOException)4 AmazonS3Exception (com.amazonaws.services.s3.model.AmazonS3Exception)3 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)3 ByteArrayInputStream (java.io.ByteArrayInputStream)3 Exchange (org.apache.camel.Exchange)3 DataStoreException (org.apache.jackrabbit.core.data.DataStoreException)3 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)2 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)2 PutObjectResult (com.amazonaws.services.s3.model.PutObjectResult)2 S3ObjectInputStream (com.amazonaws.services.s3.model.S3ObjectInputStream)2 DBException (com.yahoo.ycsb.DBException)2 FileNotFoundException (java.io.FileNotFoundException)2 Date (java.util.Date)2 HashMap (java.util.HashMap)2