Search in sources :

Example 1 with LocalCacheFileSystem

use of alluxio.hadoop.LocalCacheFileSystem in project presto by prestodb.

the class AlluxioCachingFileSystem method initialize.

@Override
public synchronized void initialize(URI uri, Configuration configuration) throws IOException {
    this.localCacheFileSystem = new LocalCacheFileSystem(dataTier, uriStatus -> {
        // CacheContext is the mechanism to pass the hiveFileContext to the source filesystem
        // hiveFileContext is critical to use to open file.
        CacheContext cacheContext = uriStatus.getCacheContext();
        checkState(cacheContext instanceof PrestoCacheContext);
        HiveFileContext hiveFileContext = ((PrestoCacheContext) cacheContext).getHiveFileContext();
        try {
            return dataTier.openFile(new Path(uriStatus.getPath()), hiveFileContext);
        } catch (Exception e) {
            throw new IOException("Failed to open file", e);
        }
    });
    this.cacheQuotaEnabled = configuration.getBoolean(USER_CLIENT_CACHE_QUOTA_ENABLED.getName(), false);
    localCacheFileSystem.initialize(uri, configuration);
}
Also used : UTF_8(java.nio.charset.StandardCharsets.UTF_8) ExtendedFileSystem(com.facebook.presto.hive.filesystem.ExtendedFileSystem) USER_CLIENT_CACHE_QUOTA_ENABLED(alluxio.conf.PropertyKey.USER_CLIENT_CACHE_QUOTA_ENABLED) CacheContext(alluxio.client.file.CacheContext) LocalCacheFileSystem(alluxio.hadoop.LocalCacheFileSystem) IOException(java.io.IOException) HiveFileContext(com.facebook.presto.hive.HiveFileContext) CachingFileSystem(com.facebook.presto.cache.CachingFileSystem) Preconditions.checkState(com.google.common.base.Preconditions.checkState) Hashing.md5(com.google.common.hash.Hashing.md5) URIStatus(alluxio.client.file.URIStatus) FileInfo(alluxio.wire.FileInfo) Configuration(org.apache.hadoop.conf.Configuration) Path(org.apache.hadoop.fs.Path) URI(java.net.URI) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream) Path(org.apache.hadoop.fs.Path) LocalCacheFileSystem(alluxio.hadoop.LocalCacheFileSystem) HiveFileContext(com.facebook.presto.hive.HiveFileContext) IOException(java.io.IOException) CacheContext(alluxio.client.file.CacheContext) IOException(java.io.IOException)

Aggregations

CacheContext (alluxio.client.file.CacheContext)1 URIStatus (alluxio.client.file.URIStatus)1 USER_CLIENT_CACHE_QUOTA_ENABLED (alluxio.conf.PropertyKey.USER_CLIENT_CACHE_QUOTA_ENABLED)1 LocalCacheFileSystem (alluxio.hadoop.LocalCacheFileSystem)1 FileInfo (alluxio.wire.FileInfo)1 CachingFileSystem (com.facebook.presto.cache.CachingFileSystem)1 HiveFileContext (com.facebook.presto.hive.HiveFileContext)1 ExtendedFileSystem (com.facebook.presto.hive.filesystem.ExtendedFileSystem)1 Preconditions.checkState (com.google.common.base.Preconditions.checkState)1 Hashing.md5 (com.google.common.hash.Hashing.md5)1 IOException (java.io.IOException)1 URI (java.net.URI)1 UTF_8 (java.nio.charset.StandardCharsets.UTF_8)1 Configuration (org.apache.hadoop.conf.Configuration)1 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)1 Path (org.apache.hadoop.fs.Path)1