Search in sources :

Example 1 with Download

use of com.qcloud.cos.transfer.Download in project cos-java-sdk-v5 by tencentyun.

the class TransferManagerTest method testTransferManagerUploadDownBigFile.

@Test
public void testTransferManagerUploadDownBigFile() throws IOException, CosServiceException, CosClientException, InterruptedException {
    if (!judgeUserInfoValid()) {
        return;
    }
    TransferManager transferManager = new TransferManager(cosclient);
    File localFile = buildTestFile(1024 * 1024 * 10L);
    File downFile = new File(localFile.getAbsolutePath() + ".down");
    String key = "ut/" + localFile.getName();
    String destKey = key + ".copy";
    try {
        PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, localFile);
        ObjectMetadata objectMetadata = new ObjectMetadata();
        objectMetadata.setServerSideEncryption("AES256");
        putObjectRequest.setMetadata(objectMetadata);
        Upload upload = transferManager.upload(putObjectRequest);
        UploadResult uploadResult = upload.waitForUploadResult();
        assertTrue(uploadResult.getETag().contains("-"));
        assertTrue(upload.isResumeableMultipartUploadAfterFailed());
        assertNotNull(upload.getResumeableMultipartUploadId());
        assertNotNull(uploadResult.getRequestId());
        assertNotNull(uploadResult.getDateStr());
        GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
        Download download = transferManager.download(getObjectRequest, downFile);
        download.waitForCompletion();
        // check file
        assertEquals(Md5Utils.md5Hex(localFile), Md5Utils.md5Hex(downFile));
        CopyObjectRequest copyObjectRequest = new CopyObjectRequest(bucket, key, bucket, destKey);
        Copy copy = transferManager.copy(copyObjectRequest, cosclient, null);
        copy.waitForCompletion();
    } finally {
        // delete file on cos
        clearObject(key);
        // delete file on cos
        clearObject(destKey);
        if (localFile.exists()) {
            assertTrue(localFile.delete());
        }
        if (downFile.exists()) {
            assertTrue(downFile.delete());
        }
    }
}
Also used : TransferManager(com.qcloud.cos.transfer.TransferManager) CopyObjectRequest(com.qcloud.cos.model.CopyObjectRequest) Copy(com.qcloud.cos.transfer.Copy) MultipleFileUpload(com.qcloud.cos.transfer.MultipleFileUpload) Upload(com.qcloud.cos.transfer.Upload) UploadResult(com.qcloud.cos.model.UploadResult) File(java.io.File) ObjectMetadata(com.qcloud.cos.model.ObjectMetadata) GetObjectRequest(com.qcloud.cos.model.GetObjectRequest) MultipleFileDownload(com.qcloud.cos.transfer.MultipleFileDownload) Download(com.qcloud.cos.transfer.Download) PutObjectRequest(com.qcloud.cos.model.PutObjectRequest) Test(org.junit.Test)

Example 2 with Download

use of com.qcloud.cos.transfer.Download in project cos-java-sdk-v5 by tencentyun.

the class TransferManagerTest method testTransferManagerUploadDownCopySmallFile.

@Test
public void testTransferManagerUploadDownCopySmallFile() throws IOException, CosServiceException, CosClientException, InterruptedException {
    if (!judgeUserInfoValid()) {
        return;
    }
    TransferManager transferManager = new TransferManager(cosclient);
    File localFile = buildTestFile(1024 * 1024 * 2L);
    File downFile = new File(localFile.getAbsolutePath() + ".down");
    String key = "ut/" + localFile.getName();
    String destKey = key + ".copy";
    try {
        PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, localFile);
        Upload upload = transferManager.upload(putObjectRequest);
        UploadResult uploadResult = upload.waitForUploadResult();
        // head object
        headSimpleObject(key, localFile.length(), Md5Utils.md5Hex(localFile));
        assertEquals(Md5Utils.md5Hex(localFile), uploadResult.getETag());
        assertFalse(upload.isResumeableMultipartUploadAfterFailed());
        assertNull(upload.getResumeableMultipartUploadId());
        assertNotNull(uploadResult.getRequestId());
        assertNotNull(uploadResult.getDateStr());
        GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
        Download download = transferManager.download(getObjectRequest, downFile);
        download.waitForCompletion();
        // check file
        assertEquals(Md5Utils.md5Hex(localFile), Md5Utils.md5Hex(downFile));
        CopyObjectRequest copyObjectRequest = new CopyObjectRequest(bucket, key, bucket, destKey);
        Copy copy = transferManager.copy(copyObjectRequest, cosclient, null);
        copy.waitForCompletion();
    } finally {
        // clear object
        clearObject(key);
        // clear dest object
        clearObject(destKey);
        // delete smallfile
        if (localFile.exists()) {
            assertTrue(localFile.delete());
        }
        if (downFile.exists()) {
            assertTrue(downFile.delete());
        }
    }
}
Also used : TransferManager(com.qcloud.cos.transfer.TransferManager) CopyObjectRequest(com.qcloud.cos.model.CopyObjectRequest) Copy(com.qcloud.cos.transfer.Copy) MultipleFileUpload(com.qcloud.cos.transfer.MultipleFileUpload) Upload(com.qcloud.cos.transfer.Upload) UploadResult(com.qcloud.cos.model.UploadResult) File(java.io.File) GetObjectRequest(com.qcloud.cos.model.GetObjectRequest) MultipleFileDownload(com.qcloud.cos.transfer.MultipleFileDownload) Download(com.qcloud.cos.transfer.Download) PutObjectRequest(com.qcloud.cos.model.PutObjectRequest) Test(org.junit.Test)

Example 3 with Download

use of com.qcloud.cos.transfer.Download in project cos-java-sdk-v5 by tencentyun.

the class TransferManagerDemo method resumableDownloadFile.

public static void resumableDownloadFile() {
    // 1 初始化用户身份信息(secretId, secretKey)
    COSCredentials cred = new BasicCOSCredentials("AKIDXXXXXXXX", "1A2Z3YYYYYYYYYY");
    // 2 设置bucket的区域, COS地域的简称请参照 https://www.qcloud.com/document/product/436/6224
    ClientConfig clientConfig = new ClientConfig(new Region("ap-shanghai"));
    // 3 生成cos客户端
    COSClient cosclient = new COSClient(cred, clientConfig);
    ExecutorService threadPool = Executors.newFixedThreadPool(5);
    // 传入一个threadpool, 若不传入线程池, 默认TransferManager中会生成一个单线程的线程池。
    TransferManager transferManager = new TransferManager(cosclient, threadPool);
    GetObjectRequest getObj = new GetObjectRequest("mybucket-1000000000", "/path/to/key");
    File dstFile = new File("dstFile");
    Download download = transferManager.download(getObj, dstFile, true);
    showTransferProgress(download);
    try {
        download.waitForCompletion();
    } catch (CosServiceException e) {
        e.printStackTrace();
    } catch (CosClientException e) {
        e.printStackTrace();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    transferManager.shutdownNow();
    cosclient.shutdown();
}
Also used : TransferManager(com.qcloud.cos.transfer.TransferManager) COSCredentials(com.qcloud.cos.auth.COSCredentials) BasicCOSCredentials(com.qcloud.cos.auth.BasicCOSCredentials) BasicCOSCredentials(com.qcloud.cos.auth.BasicCOSCredentials) CosClientException(com.qcloud.cos.exception.CosClientException) COSClient(com.qcloud.cos.COSClient) CosServiceException(com.qcloud.cos.exception.CosServiceException) ExecutorService(java.util.concurrent.ExecutorService) Region(com.qcloud.cos.region.Region) ClientConfig(com.qcloud.cos.ClientConfig) GetObjectRequest(com.qcloud.cos.model.GetObjectRequest) File(java.io.File) PersistableDownload(com.qcloud.cos.transfer.PersistableDownload) MultipleFileDownload(com.qcloud.cos.transfer.MultipleFileDownload) Download(com.qcloud.cos.transfer.Download)

Example 4 with Download

use of com.qcloud.cos.transfer.Download in project cos-java-sdk-v5 by tencentyun.

the class KMSEncryptionClientDemo method transferManagerDemo.

static void transferManagerDemo() {
    ExecutorService threadPool = Executors.newFixedThreadPool(32);
    // 传入一个threadpool, 若不传入线程池, 默认TransferManager中会生成一个单线程的线程池。
    TransferManager transferManager = new TransferManager(cosClient, threadPool);
    TransferManagerConfiguration transferManagerConfiguration = new TransferManagerConfiguration();
    transferManagerConfiguration.setMultipartUploadThreshold(1024 * 1024);
    PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName, key, bigLocalFile);
    try {
        // 返回一个异步结果Upload, 可同步的调用waitForUploadResult等待upload结束, 成功返回UploadResult, 失败抛出异常.
        Upload upload = transferManager.upload(putObjectRequest);
        UploadResult uploadResult = upload.waitForUploadResult();
        System.out.println(uploadResult.getRequestId());
    } catch (CosServiceException e) {
        e.printStackTrace();
    } catch (CosClientException e) {
        e.printStackTrace();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    GetObjectRequest getObjectRequest = new GetObjectRequest(bucketName, key);
    try {
        // 返回一个异步结果Upload, 可同步的调用waitForUploadResult等待download结束, 成功返回DownloadResult, 失败抛出异常.
        Download download = transferManager.download(getObjectRequest, new File("downLen10m.txt"));
        download.waitForCompletion();
        System.out.println(download.getObjectMetadata().getRequestId());
    } catch (CosServiceException e) {
        e.printStackTrace();
    } catch (CosClientException e) {
        e.printStackTrace();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
    transferManager.shutdownNow();
}
Also used : TransferManager(com.qcloud.cos.transfer.TransferManager) TransferManagerConfiguration(com.qcloud.cos.transfer.TransferManagerConfiguration) CosServiceException(com.qcloud.cos.exception.CosServiceException) CosClientException(com.qcloud.cos.exception.CosClientException) ExecutorService(java.util.concurrent.ExecutorService) Upload(com.qcloud.cos.transfer.Upload) UploadResult(com.qcloud.cos.model.UploadResult) GetObjectRequest(com.qcloud.cos.model.GetObjectRequest) Download(com.qcloud.cos.transfer.Download) File(java.io.File) PutObjectRequest(com.qcloud.cos.model.PutObjectRequest)

Example 5 with Download

use of com.qcloud.cos.transfer.Download in project cos-java-sdk-v5 by tencentyun.

the class AbstractCOSEncryptionClientTest method testTransferManagerUploadDownBigFile.

@Test
public void testTransferManagerUploadDownBigFile() throws IOException, CosServiceException, CosClientException, InterruptedException {
    if (!judgeUserInfoValid()) {
        return;
    }
    TransferManager transferManager = new TransferManager(cosclient);
    File localFile = buildTestFile(1024 * 1024 * 10L);
    File downFile = new File(localFile.getAbsolutePath() + ".down");
    String key = "ut/" + localFile.getName();
    String destKey = key + ".copy";
    try {
        PutObjectRequest putObjectRequest = new PutObjectRequest(bucket, key, localFile);
        ObjectMetadata objectMetadata = new ObjectMetadata();
        putObjectRequest.setMetadata(objectMetadata);
        Upload upload = transferManager.upload(putObjectRequest);
        UploadResult uploadResult = upload.waitForUploadResult();
        assertTrue(uploadResult.getETag().contains("-"));
        assertNotNull(uploadResult.getRequestId());
        assertNotNull(uploadResult.getDateStr());
        GetObjectRequest getObjectRequest = new GetObjectRequest(bucket, key);
        Download download = transferManager.download(getObjectRequest, downFile);
        download.waitForCompletion();
        // check file
        assertEquals(Md5Utils.md5Hex(localFile), Md5Utils.md5Hex(downFile));
        CopyObjectRequest copyObjectRequest = new CopyObjectRequest(bucket, key, bucket, destKey);
        Copy copy = transferManager.copy(copyObjectRequest, cosclient, null);
        copy.waitForCompletion();
    } finally {
        // delete file on cos
        clearObject(key);
        // delete file on cos
        clearObject(destKey);
        if (localFile.exists()) {
            assertTrue(localFile.delete());
        }
        if (downFile.exists()) {
            assertTrue(downFile.delete());
        }
    }
}
Also used : TransferManager(com.qcloud.cos.transfer.TransferManager) CopyObjectRequest(com.qcloud.cos.model.CopyObjectRequest) Copy(com.qcloud.cos.transfer.Copy) Upload(com.qcloud.cos.transfer.Upload) UploadResult(com.qcloud.cos.model.UploadResult) File(java.io.File) ObjectMetadata(com.qcloud.cos.model.ObjectMetadata) GetObjectRequest(com.qcloud.cos.model.GetObjectRequest) Download(com.qcloud.cos.transfer.Download) PutObjectRequest(com.qcloud.cos.model.PutObjectRequest) Test(org.junit.Test)

Aggregations

GetObjectRequest (com.qcloud.cos.model.GetObjectRequest)7 Download (com.qcloud.cos.transfer.Download)7 TransferManager (com.qcloud.cos.transfer.TransferManager)7 File (java.io.File)7 MultipleFileDownload (com.qcloud.cos.transfer.MultipleFileDownload)5 CosClientException (com.qcloud.cos.exception.CosClientException)4 CosServiceException (com.qcloud.cos.exception.CosServiceException)4 PutObjectRequest (com.qcloud.cos.model.PutObjectRequest)4 UploadResult (com.qcloud.cos.model.UploadResult)4 Upload (com.qcloud.cos.transfer.Upload)4 ExecutorService (java.util.concurrent.ExecutorService)4 COSClient (com.qcloud.cos.COSClient)3 ClientConfig (com.qcloud.cos.ClientConfig)3 BasicCOSCredentials (com.qcloud.cos.auth.BasicCOSCredentials)3 COSCredentials (com.qcloud.cos.auth.COSCredentials)3 CopyObjectRequest (com.qcloud.cos.model.CopyObjectRequest)3 Region (com.qcloud.cos.region.Region)3 Copy (com.qcloud.cos.transfer.Copy)3 PersistableDownload (com.qcloud.cos.transfer.PersistableDownload)3 Test (org.junit.Test)3