Search in sources :

Example 6 with WrappedKeyNotFoundException

use of org.apache.storm.utils.WrappedKeyNotFoundException in project storm by apache.

the class AsyncLocalizer method getBlobs.

/**
 * This function either returns the blobs in the existing cache or if they don't exist in the cache, it downloads them in parallel (up
 * to SUPERVISOR_BLOBSTORE_DOWNLOAD_THREAD_COUNT) and will block until all of them have been downloaded.
 */
List<LocalizedResource> getBlobs(List<LocalResource> localResources, PortAndAssignment pna, BlobChangingCallback cb) throws AuthorizationException, KeyNotFoundException, IOException {
    if ((boolean) conf.getOrDefault(Config.DISABLE_SYMLINKS, false)) {
        throw new WrappedKeyNotFoundException("symlinks are disabled so blobs cannot be downloaded.");
    }
    String user = pna.getOwner();
    ArrayList<LocalizedResource> results = new ArrayList<>();
    List<CompletableFuture<?>> futures = new ArrayList<>();
    try {
        for (LocalResource localResource : localResources) {
            String key = localResource.getBlobName();
            boolean uncompress = localResource.shouldUncompress();
            LocalizedResource lrsrc = uncompress ? getUserArchive(user, key) : getUserFile(user, key);
            // go off to blobstore and get it
            // assume dir passed in exists and has correct permission
            LOG.debug("fetching blob: {}", key);
            lrsrc.addReference(pna, localResource.needsCallback() ? cb : null);
            futures.add(downloadOrUpdate(lrsrc));
            results.add(lrsrc);
        }
        for (CompletableFuture<?> futureRsrc : futures) {
            futureRsrc.get();
        }
    } catch (ExecutionException e) {
        Utils.unwrapAndThrow(AuthorizationException.class, e);
        Utils.unwrapAndThrow(KeyNotFoundException.class, e);
        throw new IOException("Error getting blobs", e);
    } catch (RejectedExecutionException re) {
        throw new IOException("RejectedExecutionException: ", re);
    } catch (InterruptedException ie) {
        throw new IOException("Interrupted Exception", ie);
    }
    return results;
}
Also used : WrappedKeyNotFoundException(org.apache.storm.utils.WrappedKeyNotFoundException) AuthorizationException(org.apache.storm.generated.AuthorizationException) ArrayList(java.util.ArrayList) IOException(java.io.IOException) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) CompletableFuture(java.util.concurrent.CompletableFuture) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) ExecutionException(java.util.concurrent.ExecutionException) WrappedKeyNotFoundException(org.apache.storm.utils.WrappedKeyNotFoundException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException)

Aggregations

WrappedKeyNotFoundException (org.apache.storm.utils.WrappedKeyNotFoundException)6 IOException (java.io.IOException)3 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 FileNotFoundException (java.io.FileNotFoundException)2 InputStream (java.io.InputStream)2 KeyNotFoundException (org.apache.storm.generated.KeyNotFoundException)2 SettableBlobMeta (org.apache.storm.generated.SettableBlobMeta)2 KeeperException (org.apache.storm.shade.org.apache.zookeeper.KeeperException)2 ArrayList (java.util.ArrayList)1 HashSet (java.util.HashSet)1 TreeSet (java.util.TreeSet)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 ExecutionException (java.util.concurrent.ExecutionException)1 RejectedExecutionException (java.util.concurrent.RejectedExecutionException)1 BlobStoreFile (org.apache.storm.blobstore.BlobStoreFile)1 AuthorizationException (org.apache.storm.generated.AuthorizationException)1 NimbusInfo (org.apache.storm.nimbus.NimbusInfo)1