Search in sources :

Example 1 with CopyObjectRequest

use of com.amazonaws.services.s3.model.CopyObjectRequest in project camel by apache.

the class AmazonS3ClientMock method copyObject.

@Override
public CopyObjectResult copyObject(CopyObjectRequest copyObjectRequest) throws AmazonClientException, AmazonServiceException {
    CopyObjectResult copyObjectResult = new CopyObjectResult();
    copyObjectResult.setETag("3a5c8b1ad448bca04584ecb55b836264");
    copyObjectResult.setVersionId("11192828ahsh2723");
    return copyObjectResult;
}
Also used : CopyObjectResult(com.amazonaws.services.s3.model.CopyObjectResult)

Example 2 with CopyObjectRequest

use of com.amazonaws.services.s3.model.CopyObjectRequest in project camel by apache.

the class S3Producer method copyObject.

private void copyObject(AmazonS3 s3Client, Exchange exchange) {
    String bucketNameDestination;
    String destinationKey;
    String sourceKey;
    String bucketName;
    String versionId;
    bucketName = exchange.getIn().getHeader(S3Constants.BUCKET_NAME, String.class);
    if (ObjectHelper.isEmpty(bucketName)) {
        bucketName = getConfiguration().getBucketName();
    }
    sourceKey = exchange.getIn().getHeader(S3Constants.KEY, String.class);
    destinationKey = exchange.getIn().getHeader(S3Constants.DESTINATION_KEY, String.class);
    bucketNameDestination = exchange.getIn().getHeader(S3Constants.BUCKET_DESTINATION_NAME, String.class);
    versionId = exchange.getIn().getHeader(S3Constants.VERSION_ID, String.class);
    if (ObjectHelper.isEmpty(bucketName)) {
        throw new IllegalArgumentException("Bucket Name must be specified for copyObject Operation");
    }
    if (ObjectHelper.isEmpty(bucketNameDestination)) {
        throw new IllegalArgumentException("Bucket Name Destination must be specified for copyObject Operation");
    }
    if (ObjectHelper.isEmpty(sourceKey)) {
        throw new IllegalArgumentException("Source Key must be specified for copyObject Operation");
    }
    if (ObjectHelper.isEmpty(destinationKey)) {
        throw new IllegalArgumentException("Destination Key must be specified for copyObject Operation");
    }
    CopyObjectRequest copyObjectRequest;
    if (ObjectHelper.isEmpty(versionId)) {
        copyObjectRequest = new CopyObjectRequest(bucketName, sourceKey, bucketNameDestination, destinationKey);
    } else {
        copyObjectRequest = new CopyObjectRequest(bucketName, sourceKey, versionId, bucketNameDestination, destinationKey);
    }
    CopyObjectResult copyObjectResult = s3Client.copyObject(copyObjectRequest);
    Message message = getMessageForResponse(exchange);
    message.setHeader(S3Constants.E_TAG, copyObjectResult.getETag());
    if (copyObjectResult.getVersionId() != null) {
        message.setHeader(S3Constants.VERSION_ID, copyObjectResult.getVersionId());
    }
}
Also used : CopyObjectRequest(com.amazonaws.services.s3.model.CopyObjectRequest) Message(org.apache.camel.Message) CopyObjectResult(com.amazonaws.services.s3.model.CopyObjectResult)

Example 3 with CopyObjectRequest

use of com.amazonaws.services.s3.model.CopyObjectRequest in project hadoop by apache.

the class S3AFileSystem method copyFile.

/**
   * Copy a single object in the bucket via a COPY operation.
   * @param srcKey source object path
   * @param dstKey destination object path
   * @param size object size
   * @throws AmazonClientException on failures inside the AWS SDK
   * @throws InterruptedIOException the operation was interrupted
   * @throws IOException Other IO problems
   */
private void copyFile(String srcKey, String dstKey, long size) throws IOException, InterruptedIOException, AmazonClientException {
    LOG.debug("copyFile {} -> {} ", srcKey, dstKey);
    try {
        ObjectMetadata srcom = getObjectMetadata(srcKey);
        ObjectMetadata dstom = cloneObjectMetadata(srcom);
        setOptionalObjectMetadata(dstom);
        CopyObjectRequest copyObjectRequest = new CopyObjectRequest(bucket, srcKey, bucket, dstKey);
        setOptionalCopyObjectRequestParameters(copyObjectRequest);
        copyObjectRequest.setCannedAccessControlList(cannedACL);
        copyObjectRequest.setNewObjectMetadata(dstom);
        ProgressListener progressListener = new ProgressListener() {

            public void progressChanged(ProgressEvent progressEvent) {
                switch(progressEvent.getEventType()) {
                    case TRANSFER_PART_COMPLETED_EVENT:
                        incrementWriteOperations();
                        break;
                    default:
                        break;
                }
            }
        };
        Copy copy = transfers.copy(copyObjectRequest);
        copy.addProgressListener(progressListener);
        try {
            copy.waitForCopyResult();
            incrementWriteOperations();
            instrumentation.filesCopied(1, size);
        } catch (InterruptedException e) {
            throw new InterruptedIOException("Interrupted copying " + srcKey + " to " + dstKey + ", cancelling");
        }
    } catch (AmazonClientException e) {
        throw translateException("copyFile(" + srcKey + ", " + dstKey + ")", srcKey, e);
    }
}
Also used : InterruptedIOException(java.io.InterruptedIOException) CopyObjectRequest(com.amazonaws.services.s3.model.CopyObjectRequest) ProgressListener(com.amazonaws.event.ProgressListener) Copy(com.amazonaws.services.s3.transfer.Copy) AmazonClientException(com.amazonaws.AmazonClientException) ProgressEvent(com.amazonaws.event.ProgressEvent) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata)

Example 4 with CopyObjectRequest

use of com.amazonaws.services.s3.model.CopyObjectRequest in project elasticsearch by elastic.

the class S3BlobContainer method move.

@Override
public void move(String sourceBlobName, String targetBlobName) throws IOException {
    try {
        CopyObjectRequest request = new CopyObjectRequest(blobStore.bucket(), buildKey(sourceBlobName), blobStore.bucket(), buildKey(targetBlobName));
        if (blobStore.serverSideEncryption()) {
            ObjectMetadata objectMetadata = new ObjectMetadata();
            objectMetadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
            request.setNewObjectMetadata(objectMetadata);
        }
        SocketAccess.doPrivilegedVoid(() -> {
            blobStore.client().copyObject(request);
            blobStore.client().deleteObject(blobStore.bucket(), buildKey(sourceBlobName));
        });
    } catch (AmazonS3Exception e) {
        throw new IOException(e);
    }
}
Also used : CopyObjectRequest(com.amazonaws.services.s3.model.CopyObjectRequest) AmazonS3Exception(com.amazonaws.services.s3.model.AmazonS3Exception) IOException(java.io.IOException) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata)

Example 5 with CopyObjectRequest

use of com.amazonaws.services.s3.model.CopyObjectRequest in project alluxio by Alluxio.

the class S3AUnderFileSystem method copyObject.

@Override
protected boolean copyObject(String src, String dst) {
    LOG.debug("Copying {} to {}", src, dst);
    // Retry copy for a few times, in case some AWS internal errors happened during copy.
    int retries = 3;
    for (int i = 0; i < retries; i++) {
        try {
            CopyObjectRequest request = new CopyObjectRequest(mBucketName, src, mBucketName, dst);
            if (Configuration.getBoolean(PropertyKey.UNDERFS_S3A_SERVER_SIDE_ENCRYPTION_ENABLED)) {
                ObjectMetadata meta = new ObjectMetadata();
                meta.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
                request.setNewObjectMetadata(meta);
            }
            mManager.copy(request).waitForCopyResult();
            return true;
        } catch (AmazonClientException | InterruptedException e) {
            LOG.error("Failed to copy file {} to {}", src, dst, e);
            if (i != retries - 1) {
                LOG.error("Retrying copying file {} to {}", src, dst);
            }
        }
    }
    LOG.error("Failed to copy file {} to {}, after {} retries", src, dst, retries);
    return false;
}
Also used : CopyObjectRequest(com.amazonaws.services.s3.model.CopyObjectRequest) AmazonClientException(com.amazonaws.AmazonClientException) ObjectMetadata(com.amazonaws.services.s3.model.ObjectMetadata)

Aggregations

ObjectMetadata (com.amazonaws.services.s3.model.ObjectMetadata)12 CopyObjectRequest (com.amazonaws.services.s3.model.CopyObjectRequest)11 AmazonClientException (com.amazonaws.AmazonClientException)9 Copy (com.amazonaws.services.s3.transfer.Copy)8 AmazonServiceException (com.amazonaws.AmazonServiceException)7 IOException (java.io.IOException)7 DataStoreException (org.apache.jackrabbit.core.data.DataStoreException)7 PutObjectRequest (com.amazonaws.services.s3.model.PutObjectRequest)3 Upload (com.amazonaws.services.s3.transfer.Upload)3 CopyObjectResult (com.amazonaws.services.s3.model.CopyObjectResult)2 AsyncUploadResult (org.apache.jackrabbit.core.data.AsyncUploadResult)2 ProgressEvent (com.amazonaws.event.ProgressEvent)1 ProgressListener (com.amazonaws.event.ProgressListener)1 AmazonS3Exception (com.amazonaws.services.s3.model.AmazonS3Exception)1 SSECustomerKey (com.amazonaws.services.s3.model.SSECustomerKey)1 InterruptedIOException (java.io.InterruptedIOException)1 UnsupportedEncodingException (java.io.UnsupportedEncodingException)1 Message (org.apache.camel.Message)1