Search in sources :

Example 1 with UploadPartResult

use of com.aliyun.oss.model.UploadPartResult in project hadoop by apache.

the class AliyunOSSFileSystemStore method multipartUploadObject.

/**
   * Upload a file as an OSS object, using multipart upload.
   *
   * @param key object key.
   * @param file local file to upload.
   * @throws IOException if failed to upload object.
   */
public void multipartUploadObject(String key, File file) throws IOException {
    File object = file.getAbsoluteFile();
    long dataLen = object.length();
    long realPartSize = AliyunOSSUtils.calculatePartSize(dataLen, partSize);
    int partNum = (int) (dataLen / realPartSize);
    if (dataLen % realPartSize != 0) {
        partNum += 1;
    }
    InitiateMultipartUploadRequest initiateMultipartUploadRequest = new InitiateMultipartUploadRequest(bucketName, key);
    ObjectMetadata meta = new ObjectMetadata();
    if (StringUtils.isNotEmpty(serverSideEncryptionAlgorithm)) {
        meta.setServerSideEncryption(serverSideEncryptionAlgorithm);
    }
    initiateMultipartUploadRequest.setObjectMetadata(meta);
    InitiateMultipartUploadResult initiateMultipartUploadResult = ossClient.initiateMultipartUpload(initiateMultipartUploadRequest);
    List<PartETag> partETags = new ArrayList<PartETag>();
    String uploadId = initiateMultipartUploadResult.getUploadId();
    try {
        for (int i = 0; i < partNum; i++) {
            // TODO: Optimize this, avoid opening the object multiple times
            FileInputStream fis = new FileInputStream(object);
            try {
                long skipBytes = realPartSize * i;
                AliyunOSSUtils.skipFully(fis, skipBytes);
                long size = (realPartSize < dataLen - skipBytes) ? realPartSize : dataLen - skipBytes;
                UploadPartRequest uploadPartRequest = new UploadPartRequest();
                uploadPartRequest.setBucketName(bucketName);
                uploadPartRequest.setKey(key);
                uploadPartRequest.setUploadId(uploadId);
                uploadPartRequest.setInputStream(fis);
                uploadPartRequest.setPartSize(size);
                uploadPartRequest.setPartNumber(i + 1);
                UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
                statistics.incrementWriteOps(1);
                partETags.add(uploadPartResult.getPartETag());
            } finally {
                fis.close();
            }
        }
        CompleteMultipartUploadRequest completeMultipartUploadRequest = new CompleteMultipartUploadRequest(bucketName, key, uploadId, partETags);
        CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest);
        LOG.debug(completeMultipartUploadResult.getETag());
    } catch (OSSException | ClientException e) {
        AbortMultipartUploadRequest abortMultipartUploadRequest = new AbortMultipartUploadRequest(bucketName, key, uploadId);
        ossClient.abortMultipartUpload(abortMultipartUploadRequest);
    }
}
Also used : InitiateMultipartUploadResult(com.aliyun.oss.model.InitiateMultipartUploadResult) InitiateMultipartUploadRequest(com.aliyun.oss.model.InitiateMultipartUploadRequest) ArrayList(java.util.ArrayList) UploadPartRequest(com.aliyun.oss.model.UploadPartRequest) OSSException(com.aliyun.oss.OSSException) AbortMultipartUploadRequest(com.aliyun.oss.model.AbortMultipartUploadRequest) CompleteMultipartUploadResult(com.aliyun.oss.model.CompleteMultipartUploadResult) PartETag(com.aliyun.oss.model.PartETag) FileInputStream(java.io.FileInputStream) UploadPartResult(com.aliyun.oss.model.UploadPartResult) ClientException(com.aliyun.oss.ClientException) File(java.io.File) ObjectMetadata(com.aliyun.oss.model.ObjectMetadata) CompleteMultipartUploadRequest(com.aliyun.oss.model.CompleteMultipartUploadRequest)

Aggregations

ClientException (com.aliyun.oss.ClientException)1 OSSException (com.aliyun.oss.OSSException)1 AbortMultipartUploadRequest (com.aliyun.oss.model.AbortMultipartUploadRequest)1 CompleteMultipartUploadRequest (com.aliyun.oss.model.CompleteMultipartUploadRequest)1 CompleteMultipartUploadResult (com.aliyun.oss.model.CompleteMultipartUploadResult)1 InitiateMultipartUploadRequest (com.aliyun.oss.model.InitiateMultipartUploadRequest)1 InitiateMultipartUploadResult (com.aliyun.oss.model.InitiateMultipartUploadResult)1 ObjectMetadata (com.aliyun.oss.model.ObjectMetadata)1 PartETag (com.aliyun.oss.model.PartETag)1 UploadPartRequest (com.aliyun.oss.model.UploadPartRequest)1 UploadPartResult (com.aliyun.oss.model.UploadPartResult)1 File (java.io.File)1 FileInputStream (java.io.FileInputStream)1 ArrayList (java.util.ArrayList)1