Search in sources :

Example 1 with AbortMultipartUploadRequest

use of com.emc.object.s3.request.AbortMultipartUploadRequest in project pravega by pravega.

the class ExtendedS3ChunkStorage method doConcat.

@Override
public int doConcat(ConcatArgument[] chunks) throws ChunkStorageException {
    int totalBytesConcatenated = 0;
    String targetPath = getObjectPath(chunks[0].getName());
    String uploadId = null;
    boolean isCompleted = false;
    try {
        int partNumber = 1;
        SortedSet<MultipartPartETag> partEtags = new TreeSet<>();
        uploadId = client.initiateMultipartUpload(config.getBucket(), targetPath);
        // check whether the target exists
        if (!checkExists(chunks[0].getName())) {
            throw new ChunkNotFoundException(chunks[0].getName(), "doConcat - Target segment does not exist");
        }
        // Copy the parts
        for (int i = 0; i < chunks.length; i++) {
            if (0 != chunks[i].getLength()) {
                val sourceHandle = chunks[i];
                S3ObjectMetadata metadataResult = client.getObjectMetadata(config.getBucket(), getObjectPath(sourceHandle.getName()));
                // in bytes
                long objectSize = metadataResult.getContentLength();
                Preconditions.checkState(objectSize >= chunks[i].getLength());
                CopyPartRequest copyRequest = new CopyPartRequest(config.getBucket(), getObjectPath(sourceHandle.getName()), config.getBucket(), targetPath, uploadId, partNumber++).withSourceRange(Range.fromOffsetLength(0, chunks[i].getLength()));
                CopyPartResult copyResult = client.copyPart(copyRequest);
                partEtags.add(new MultipartPartETag(copyResult.getPartNumber(), copyResult.getETag()));
                totalBytesConcatenated += chunks[i].getLength();
            }
        }
        // Close the upload
        client.completeMultipartUpload(new CompleteMultipartUploadRequest(config.getBucket(), targetPath, uploadId).withParts(partEtags));
        isCompleted = true;
    } catch (RuntimeException e) {
        // Error message is REC_CATCH_EXCEPTION: Exception is caught when Exception is not thrown
        throw convertException(chunks[0].getName(), "doConcat", e);
    } catch (Exception e) {
        throw convertException(chunks[0].getName(), "doConcat", e);
    } finally {
        if (!isCompleted && null != uploadId) {
            client.abortMultipartUpload(new AbortMultipartUploadRequest(config.getBucket(), targetPath, uploadId));
        }
    }
    return totalBytesConcatenated;
}
Also used : ChunkNotFoundException(io.pravega.segmentstore.storage.chunklayer.ChunkNotFoundException) lombok.val(lombok.val) AbortMultipartUploadRequest(com.emc.object.s3.request.AbortMultipartUploadRequest) InvalidOffsetException(io.pravega.segmentstore.storage.chunklayer.InvalidOffsetException) S3Exception(com.emc.object.s3.S3Exception) ChunkStorageException(io.pravega.segmentstore.storage.chunklayer.ChunkStorageException) ChunkNotFoundException(io.pravega.segmentstore.storage.chunklayer.ChunkNotFoundException) ChunkAlreadyExistsException(io.pravega.segmentstore.storage.chunklayer.ChunkAlreadyExistsException) CopyPartRequest(com.emc.object.s3.request.CopyPartRequest) TreeSet(java.util.TreeSet) MultipartPartETag(com.emc.object.s3.bean.MultipartPartETag) S3ObjectMetadata(com.emc.object.s3.S3ObjectMetadata) CopyPartResult(com.emc.object.s3.bean.CopyPartResult) CompleteMultipartUploadRequest(com.emc.object.s3.request.CompleteMultipartUploadRequest)

Aggregations

S3Exception (com.emc.object.s3.S3Exception)1 S3ObjectMetadata (com.emc.object.s3.S3ObjectMetadata)1 CopyPartResult (com.emc.object.s3.bean.CopyPartResult)1 MultipartPartETag (com.emc.object.s3.bean.MultipartPartETag)1 AbortMultipartUploadRequest (com.emc.object.s3.request.AbortMultipartUploadRequest)1 CompleteMultipartUploadRequest (com.emc.object.s3.request.CompleteMultipartUploadRequest)1 CopyPartRequest (com.emc.object.s3.request.CopyPartRequest)1 ChunkAlreadyExistsException (io.pravega.segmentstore.storage.chunklayer.ChunkAlreadyExistsException)1 ChunkNotFoundException (io.pravega.segmentstore.storage.chunklayer.ChunkNotFoundException)1 ChunkStorageException (io.pravega.segmentstore.storage.chunklayer.ChunkStorageException)1 InvalidOffsetException (io.pravega.segmentstore.storage.chunklayer.InvalidOffsetException)1 TreeSet (java.util.TreeSet)1 lombok.val (lombok.val)1