Search in sources :

Example 6 with ObjectListing

use of com.aliyun.oss.model.ObjectListing in project hadoop by apache.

the class AliyunOSSFileSystemStore method listObjects.

/**
   * list objects.
   *
   * @param prefix prefix.
   * @param maxListingLength max no. of entries
   * @param marker last key in any previous search.
   * @param recursive whether to list directory recursively.
   * @return a list of matches.
   */
public ObjectListing listObjects(String prefix, int maxListingLength, String marker, boolean recursive) {
    String delimiter = recursive ? null : "/";
    prefix = AliyunOSSUtils.maybeAddTrailingSlash(prefix);
    ListObjectsRequest listRequest = new ListObjectsRequest(bucketName);
    listRequest.setPrefix(prefix);
    listRequest.setDelimiter(delimiter);
    listRequest.setMaxKeys(maxListingLength);
    listRequest.setMarker(marker);
    ObjectListing listing = ossClient.listObjects(listRequest);
    statistics.incrementReadOps(1);
    return listing;
}
Also used : ListObjectsRequest(com.aliyun.oss.model.ListObjectsRequest) ObjectListing(com.aliyun.oss.model.ObjectListing)

Example 7 with ObjectListing

use of com.aliyun.oss.model.ObjectListing in project hadoop by apache.

the class AliyunOSSFileSystemStore method deleteDirs.

/**
   * Delete a directory from Aliyun OSS.
   *
   * @param key directory key to delete.
   * @throws IOException if failed to delete directory.
   */
public void deleteDirs(String key) throws IOException {
    key = AliyunOSSUtils.maybeAddTrailingSlash(key);
    ListObjectsRequest listRequest = new ListObjectsRequest(bucketName);
    listRequest.setPrefix(key);
    listRequest.setDelimiter(null);
    listRequest.setMaxKeys(maxKeys);
    while (true) {
        ObjectListing objects = ossClient.listObjects(listRequest);
        statistics.incrementReadOps(1);
        List<String> keysToDelete = new ArrayList<String>();
        for (OSSObjectSummary objectSummary : objects.getObjectSummaries()) {
            keysToDelete.add(objectSummary.getKey());
        }
        deleteObjects(keysToDelete);
        if (objects.isTruncated()) {
            listRequest.setMarker(objects.getNextMarker());
        } else {
            break;
        }
    }
}
Also used : ListObjectsRequest(com.aliyun.oss.model.ListObjectsRequest) OSSObjectSummary(com.aliyun.oss.model.OSSObjectSummary) ArrayList(java.util.ArrayList) ObjectListing(com.aliyun.oss.model.ObjectListing)

Aggregations

ObjectListing (com.aliyun.oss.model.ObjectListing)7 OSSObjectSummary (com.aliyun.oss.model.OSSObjectSummary)4 ListObjectsRequest (com.aliyun.oss.model.ListObjectsRequest)3 ArrayList (java.util.ArrayList)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 Path (org.apache.hadoop.fs.Path)2 ClientException (com.aliyun.oss.ClientException)1 OSSException (com.aliyun.oss.OSSException)1 ObjectMetadata (com.aliyun.oss.model.ObjectMetadata)1 FileNotFoundException (java.io.FileNotFoundException)1