Search in sources :

Example 6 with ServiceException

use of org.jets3t.service.ServiceException in project hadoop by apache.

the class Jets3tNativeFileSystemStore method copy.

@Override
public void copy(String srcKey, String dstKey) throws IOException {
    try {
        if (LOG.isDebugEnabled()) {
            LOG.debug("Copying srcKey: " + srcKey + "to dstKey: " + dstKey + "in bucket: " + bucket.getName());
        }
        if (multipartEnabled) {
            S3Object object = s3Service.getObjectDetails(bucket, srcKey, null, null, null, null);
            if (multipartCopyBlockSize > 0 && object.getContentLength() > multipartCopyBlockSize) {
                copyLargeFile(object, dstKey);
                return;
            }
        }
        S3Object dstObject = new S3Object(dstKey);
        dstObject.setServerSideEncryptionAlgorithm(serverSideEncryptionAlgorithm);
        s3Service.copyObject(bucket.getName(), srcKey, bucket.getName(), dstObject, false);
    } catch (ServiceException e) {
        handleException(e, srcKey);
    }
}
Also used : ServiceException(org.jets3t.service.ServiceException) S3ServiceException(org.jets3t.service.S3ServiceException) S3Object(org.jets3t.service.model.S3Object)

Example 7 with ServiceException

use of org.jets3t.service.ServiceException in project hadoop by apache.

the class Jets3tNativeFileSystemStore method retrieveMetadata.

@Override
public FileMetadata retrieveMetadata(String key) throws IOException {
    StorageObject object = null;
    try {
        LOG.debug("Getting metadata for key: {} from bucket: {}", key, bucket.getName());
        object = s3Service.getObjectDetails(bucket.getName(), key);
        return new FileMetadata(key, object.getContentLength(), object.getLastModifiedDate().getTime());
    } catch (ServiceException e) {
        try {
            // process
            handleException(e, key);
            return null;
        } catch (FileNotFoundException fnfe) {
            // and downgrade missing files
            return null;
        }
    } finally {
        if (object != null) {
            object.closeDataInputStream();
        }
    }
}
Also used : StorageObject(org.jets3t.service.model.StorageObject) ServiceException(org.jets3t.service.ServiceException) S3ServiceException(org.jets3t.service.S3ServiceException) FileNotFoundException(java.io.FileNotFoundException)

Example 8 with ServiceException

use of org.jets3t.service.ServiceException in project hadoop by apache.

the class Jets3tNativeFileSystemStore method processException.

/**
   * Handle any service exception by translating it into an IOException
   * @param thrown exception
   * @param original original exception -thrown if no other translation could
   * be made
   * @param key key sought from object store or "" for undefined
   * @return an exception to throw. If isProcessingCause==true this may be null.
   */
private IOException processException(Throwable thrown, Throwable original, String key) {
    IOException result;
    if (thrown.getCause() != null) {
        // recurse down
        result = processException(thrown.getCause(), original, key);
    } else if (thrown instanceof HttpException) {
        // nested HttpException - examine error code and react
        HttpException httpException = (HttpException) thrown;
        String responseMessage = httpException.getResponseMessage();
        int responseCode = httpException.getResponseCode();
        String bucketName = "s3n://" + bucket.getName();
        String text = String.format("%s : %03d : %s", bucketName, responseCode, responseMessage);
        String filename = !key.isEmpty() ? (bucketName + "/" + key) : text;
        IOException ioe;
        switch(responseCode) {
            case 404:
                result = new FileNotFoundException(filename);
                break;
            case // invalid range
            416:
                result = new EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF + ": " + filename);
                break;
            case //forbidden
            403:
                result = new AccessControlException("Permission denied" + ": " + filename);
                break;
            default:
                result = new IOException(text);
        }
        result.initCause(thrown);
    } else if (thrown instanceof S3ServiceException) {
        S3ServiceException se = (S3ServiceException) thrown;
        LOG.debug("S3ServiceException: {}: {} : {}", se.getS3ErrorCode(), se.getS3ErrorMessage(), se, se);
        if ("InvalidRange".equals(se.getS3ErrorCode())) {
            result = new EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF);
        } else {
            result = new S3Exception(se);
        }
    } else if (thrown instanceof ServiceException) {
        ServiceException se = (ServiceException) thrown;
        LOG.debug("S3ServiceException: {}: {} : {}", se.getErrorCode(), se.toString(), se, se);
        result = new S3Exception(se);
    } else if (thrown instanceof IOException) {
        result = (IOException) thrown;
    } else {
        // here there is no exception derived yet.
        // this means no inner cause, and no translation made yet.
        // convert the original to an IOException -rather than just the
        // exception at the base of the tree
        result = new S3Exception(original);
    }
    return result;
}
Also used : ServiceException(org.jets3t.service.ServiceException) S3ServiceException(org.jets3t.service.S3ServiceException) FileNotFoundException(java.io.FileNotFoundException) EOFException(java.io.EOFException) AccessControlException(org.apache.hadoop.security.AccessControlException) HttpException(org.jets3t.service.impl.rest.HttpException) IOException(java.io.IOException) S3ServiceException(org.jets3t.service.S3ServiceException)

Example 9 with ServiceException

use of org.jets3t.service.ServiceException in project hadoop by apache.

the class Jets3tNativeFileSystemStore method retrieve.

/**
   * @param key
   * The key is the object name that is being retrieved from the S3 bucket
   * @return
   * This method returns null if the key is not found
   * @throws IOException
   */
@Override
public InputStream retrieve(String key) throws IOException {
    try {
        LOG.debug("Getting key: {} from bucket: {}", key, bucket.getName());
        S3Object object = s3Service.getObject(bucket.getName(), key);
        return object.getDataInputStream();
    } catch (ServiceException e) {
        handleException(e, key);
        //return null if key not found
        return null;
    }
}
Also used : ServiceException(org.jets3t.service.ServiceException) S3ServiceException(org.jets3t.service.S3ServiceException) S3Object(org.jets3t.service.model.S3Object)

Example 10 with ServiceException

use of org.jets3t.service.ServiceException in project hadoop by apache.

the class Jets3tNativeFileSystemStore method retrieve.

/**
   *
   * @param key
   * The key is the object name that is being retrieved from the S3 bucket
   * @return
   * This method returns null if the key is not found
   * @throws IOException
   */
@Override
public InputStream retrieve(String key, long byteRangeStart) throws IOException {
    try {
        LOG.debug("Getting key: {} from bucket: {} with byteRangeStart: {}", key, bucket.getName(), byteRangeStart);
        S3Object object = s3Service.getObject(bucket, key, null, null, null, null, byteRangeStart, null);
        return object.getDataInputStream();
    } catch (ServiceException e) {
        handleException(e, key);
        return null;
    }
}
Also used : ServiceException(org.jets3t.service.ServiceException) S3ServiceException(org.jets3t.service.S3ServiceException) S3Object(org.jets3t.service.model.S3Object)

Aggregations

ServiceException (org.jets3t.service.ServiceException)24 S3ServiceException (org.jets3t.service.S3ServiceException)11 IOException (java.io.IOException)10 S3Object (org.jets3t.service.model.S3Object)10 SegmentLoadingException (io.druid.segment.loading.SegmentLoadingException)4 StorageObject (org.jets3t.service.model.StorageObject)4 BufferedInputStream (java.io.BufferedInputStream)3 ByteArrayInputStream (java.io.ByteArrayInputStream)3 GSObject (org.jets3t.service.model.GSObject)3 AlluxioURI (alluxio.AlluxioURI)2 DataSegment (io.druid.timeline.DataSegment)2 File (java.io.File)2 FileNotFoundException (java.io.FileNotFoundException)2 InputStream (java.io.InputStream)2 Predicate (com.google.common.base.Predicate)1 ByteSource (com.google.common.io.ByteSource)1 FileUtils (io.druid.java.util.common.FileUtils)1 IAE (io.druid.java.util.common.IAE)1 UOE (io.druid.java.util.common.UOE)1 EOFException (java.io.EOFException)1