Search in sources :

Example 1 with ObjectMetadata

use of com.obs.services.model.ObjectMetadata in project alluxio by Alluxio.

the class OBSOutputStream method close.

/**
 * Closes this output stream. When an output stream is closed, the local temporary file is
 * uploaded to OSS Service. Once the file is uploaded, the temporary file is deleted.
 */
@Override
public void close() throws IOException {
    if (mClosed.getAndSet(true)) {
        LOG.warn("OBSOutputStream is already closed");
        return;
    }
    mLocalOutputStream.close();
    try {
        BufferedInputStream in = new BufferedInputStream(new FileInputStream(mFile));
        ObjectMetadata objMeta = new ObjectMetadata();
        objMeta.setContentLength(mFile.length());
        if (mHash != null) {
            byte[] hashBytes = mHash.digest();
            objMeta.setContentMd5(new String(Base64.encodeBase64(hashBytes)));
        }
        mObsClient.putObject(mBucketName, mKey, in, objMeta);
        mFile.delete();
    } catch (ObsException e) {
        LOG.error("Failed to upload {}. Temporary file @ {}", mKey, mFile.getPath());
        throw new IOException(e);
    }
}
Also used : ObsException(com.obs.services.exception.ObsException) BufferedInputStream(java.io.BufferedInputStream) IOException(java.io.IOException) ObjectMetadata(com.obs.services.model.ObjectMetadata) FileInputStream(java.io.FileInputStream)

Example 2 with ObjectMetadata

use of com.obs.services.model.ObjectMetadata in project alluxio by Alluxio.

the class OBSUnderFileSystem method getObjectStatus.

@Override
protected ObjectStatus getObjectStatus(String key) {
    try {
        ObjectMetadata meta = mClient.getObjectMetadata(mBucketName, key);
        if (meta == null) {
            return null;
        }
        if (isEnvironmentPFS()) {
            /**
             * When in PFS environment:
             * 1. Directory will be explicitly created and have object meta.
             * 2. File will have object meta even if there is `/` at
             *    the end of the file name (e.g. `/dir1/file1/`).
             * However we should return null meta here.
             */
            if (isDirectoryInPFS(meta)) {
                return null;
            }
            if (!isDirectoryInPFS(meta) && key.endsWith(PATH_SEPARATOR)) {
                return null;
            }
        }
        Date lastModifiedDate = meta.getLastModified();
        Long lastModifiedTime = lastModifiedDate == null ? null : lastModifiedDate.getTime();
        return new ObjectStatus(key, meta.getEtag(), meta.getContentLength(), lastModifiedTime);
    } catch (ObsException e) {
        LOG.warn("Failed to get Object {}, return null", key, e);
        return null;
    }
}
Also used : ObsException(com.obs.services.exception.ObsException) ObjectMetadata(com.obs.services.model.ObjectMetadata) Date(java.util.Date)

Example 3 with ObjectMetadata

use of com.obs.services.model.ObjectMetadata in project alluxio by Alluxio.

the class OBSUnderFileSystem method createEmptyObject.

@Override
public boolean createEmptyObject(String key) {
    try {
        ObjectMetadata objMeta = new ObjectMetadata();
        objMeta.setContentLength(0L);
        mClient.putObject(mBucketName, key, new ByteArrayInputStream(new byte[0]), objMeta);
        return true;
    } catch (ObsException e) {
        LOG.error("Failed to create object: {}", key, e);
        return false;
    }
}
Also used : ObsException(com.obs.services.exception.ObsException) ByteArrayInputStream(java.io.ByteArrayInputStream) ObjectMetadata(com.obs.services.model.ObjectMetadata)

Example 4 with ObjectMetadata

use of com.obs.services.model.ObjectMetadata in project alluxio by Alluxio.

the class OBSUnderFileSystemTest method nullObjectMetaTest.

@Test
public void nullObjectMetaTest() throws Exception {
    ObjectMetadata fileMeta = new ObjectMetadata();
    fileMeta.setContentLength(10L);
    ObjectMetadata dirMeta = new ObjectMetadata();
    dirMeta.setContentLength(0L);
    /**
     * /xx/file1/ ( File1 actually exists, which is a file) , there is / after file1 name.
     * When OBS, the path object meta is null.
     * When PFS, the path object meta is not null. The object meta is same as /xx/file1
     */
    Mockito.when(mClient.getObjectMetadata(BUCKET_NAME, "pfs_file1")).thenReturn(fileMeta);
    Mockito.when(mClient.getObjectMetadata(BUCKET_NAME, "dir1")).thenReturn(dirMeta);
    mOBSUnderFileSystem = new OBSUnderFileSystem(new AlluxioURI(""), mClient, BUCKET_NAME, "obs", UnderFileSystemConfiguration.defaults(ConfigurationTestUtils.defaults()));
    Assert.assertNotNull(mOBSUnderFileSystem.getObjectStatus("pfs_file1"));
    Assert.assertNotNull(mOBSUnderFileSystem.getObjectStatus("dir1"));
}
Also used : ObjectMetadata(com.obs.services.model.ObjectMetadata) AlluxioURI(alluxio.AlluxioURI) Test(org.junit.Test)

Example 5 with ObjectMetadata

use of com.obs.services.model.ObjectMetadata in project jeesuite-libs by vakinge.

the class HuaweicloudProvider method getObjectMetadata.

@Override
public CObjectMetadata getObjectMetadata(String bucketName, String fileKey) {
    ObjectMetadata objectMetadata = obsClient.getObjectMetadata(bucketName, fileKey);
    if (objectMetadata == null) {
        return null;
    }
    CObjectMetadata result = new CObjectMetadata();
    Map<String, Object> customMetadata = objectMetadata.getMetadata();
    if (customMetadata != null) {
        Map<String, String> metadata = Maps.newHashMap();
        for (Map.Entry<String, Object> entry : customMetadata.entrySet()) {
            metadata.put(entry.getKey(), entry.getValue().toString());
        }
        result.setCustomMetadatas(metadata);
    }
    result.setMimeType(objectMetadata.getContentType());
    result.setFilesize(objectMetadata.getContentLength());
    return result;
}
Also used : CObjectMetadata(com.mendmix.cos.CObjectMetadata) ObsObject(com.obs.services.model.ObsObject) CUploadObject(com.mendmix.cos.CUploadObject) CObjectMetadata(com.mendmix.cos.CObjectMetadata) ObjectMetadata(com.obs.services.model.ObjectMetadata) Map(java.util.Map)

Aggregations

ObjectMetadata (com.obs.services.model.ObjectMetadata)8 ObsException (com.obs.services.exception.ObsException)4 Test (org.junit.Test)3 AlluxioURI (alluxio.AlluxioURI)2 CObjectMetadata (com.mendmix.cos.CObjectMetadata)2 BufferedInputStream (java.io.BufferedInputStream)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 FileInputStream (java.io.FileInputStream)2 InputStream (java.io.InputStream)2 Date (java.util.Date)2 JeesuiteBaseException (com.mendmix.common.JeesuiteBaseException)1 CUploadObject (com.mendmix.cos.CUploadObject)1 CUploadResult (com.mendmix.cos.CUploadResult)1 AccessControlList (com.obs.services.model.AccessControlList)1 ObsObject (com.obs.services.model.ObsObject)1 PutObjectResult (com.obs.services.model.PutObjectResult)1 File (java.io.File)1 IOException (java.io.IOException)1 Map (java.util.Map)1 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)1