Search in sources :

Example 11 with CacheContext

use of alluxio.client.file.CacheContext in project alluxio by Alluxio.

the class LocalCacheManagerWithMemPageStoreTest method putWithQuotaMoreThanCacheCapacity.

@Test
public void putWithQuotaMoreThanCacheCapacity() throws Exception {
    mConf.set(PropertyKey.USER_CLIENT_CACHE_QUOTA_ENABLED, true);
    CacheScope partitionCacheScope = CacheScope.create("schema.table.partition");
    CacheScope tableCacheScope = CacheScope.create("schema.table");
    CacheScope schemaCacheScope = CacheScope.create("schema");
    CacheScope[] quotaCacheScopes = { partitionCacheScope, tableCacheScope, schemaCacheScope, CacheScope.GLOBAL };
    for (CacheScope cacheScope : quotaCacheScopes) {
        mMetaStore = new QuotaMetaStore(mConf);
        mPageStore = PageStore.create(PageStoreOptions.create(mConf));
        mCacheManager = createLocalCacheManager(mConf, mMetaStore, mPageStore);
        CacheQuota quota = new CacheQuota(ImmutableMap.of(cacheScope.level(), (long) CACHE_SIZE_BYTES + 1));
        int cacheSize = CACHE_SIZE_BYTES / PAGE_SIZE_BYTES;
        for (int i = 0; i < 2 * cacheSize; i++) {
            PageId pageId = new PageId("3", i);
            CacheContext context = CacheContext.defaults().setCacheScope(partitionCacheScope).setCacheQuota(quota);
            assertTrue(mCacheManager.put(pageId, page(i, PAGE_SIZE_BYTES), context));
            if (i >= cacheSize) {
                assertEquals(0, mCacheManager.get(new PageId("3", i - cacheSize), PAGE_SIZE_BYTES, mBuf, 0));
                // check the subsequent page is still in cache
                assertEquals(true, mMetaStore.hasPage(new PageId("3", i - cacheSize + 1)));
            }
        }
    }
}
Also used : CacheContext(alluxio.client.file.CacheContext) CacheScope(alluxio.client.quota.CacheScope) CacheQuota(alluxio.client.quota.CacheQuota) Test(org.junit.Test)

Example 12 with CacheContext

use of alluxio.client.file.CacheContext in project alluxio by Alluxio.

the class LocalCacheManagerWithMemPageStoreTest method putWithInsufficientParentQuota.

@Test
public void putWithInsufficientParentQuota() throws Exception {
    mConf.set(PropertyKey.USER_CLIENT_CACHE_QUOTA_ENABLED, true);
    CacheScope partitionCacheScope1 = CacheScope.create("schema.table.partition1");
    CacheScope partitionCacheScope2 = CacheScope.create("schema.table.partition2");
    CacheScope tableCacheScope = CacheScope.create("schema.table");
    CacheScope schemaCacheScope = CacheScope.create("schema");
    CacheScope[] quotaCacheScopes = { tableCacheScope, schemaCacheScope, CacheScope.GLOBAL };
    for (CacheScope cacheScope : quotaCacheScopes) {
        mMetaStore = new QuotaMetaStore(mConf);
        mPageStore = PageStore.create(PageStoreOptions.create(mConf));
        mCacheManager = createLocalCacheManager(mConf, mMetaStore, mPageStore);
        CacheQuota quota = new CacheQuota(ImmutableMap.of(partitionCacheScope1.level(), (long) PAGE1.length + PAGE2.length, cacheScope.level(), (long) PAGE1.length + PAGE2.length - 1));
        CacheContext context1 = CacheContext.defaults().setCacheScope(partitionCacheScope1).setCacheQuota(quota);
        assertTrue(mCacheManager.put(PAGE_ID1, PAGE1, context1));
        assertEquals(PAGE1.length, mCacheManager.get(PAGE_ID1, PAGE1.length, mBuf, 0));
        CacheContext context2 = CacheContext.defaults().setCacheScope(partitionCacheScope2).setCacheQuota(quota);
        assertTrue(mCacheManager.put(PAGE_ID2, PAGE2, context2));
        assertEquals(0, mCacheManager.get(PAGE_ID1, PAGE1.length, mBuf, 0));
        assertEquals(PAGE2.length, mCacheManager.get(PAGE_ID2, PAGE2.length, mBuf, 0));
    }
}
Also used : CacheContext(alluxio.client.file.CacheContext) CacheScope(alluxio.client.quota.CacheScope) CacheQuota(alluxio.client.quota.CacheQuota) Test(org.junit.Test)

Aggregations

CacheContext (alluxio.client.file.CacheContext)12 CacheQuota (alluxio.client.quota.CacheQuota)8 CacheScope (alluxio.client.quota.CacheScope)8 Test (org.junit.Test)8 URIStatus (alluxio.client.file.URIStatus)3 FileInfo (alluxio.wire.FileInfo)3 IOException (java.io.IOException)2 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)2 USER_CLIENT_CACHE_QUOTA_ENABLED (alluxio.conf.PropertyKey.USER_CLIENT_CACHE_QUOTA_ENABLED)1 LocalCacheFileSystem (alluxio.hadoop.LocalCacheFileSystem)1 CachingFileSystem (com.facebook.presto.cache.CachingFileSystem)1 HiveFileContext (com.facebook.presto.hive.HiveFileContext)1 ExtendedFileSystem (com.facebook.presto.hive.filesystem.ExtendedFileSystem)1 Preconditions.checkState (com.google.common.base.Preconditions.checkState)1 Stopwatch (com.google.common.base.Stopwatch)1 Hashing.md5 (com.google.common.hash.Hashing.md5)1 URI (java.net.URI)1 UTF_8 (java.nio.charset.StandardCharsets.UTF_8)1 Configuration (org.apache.hadoop.conf.Configuration)1 FileStatus (org.apache.hadoop.fs.FileStatus)1