Examples with MutableInodeDirectory alluxio.master.file.meta.MutableInodeDirectory used on opensource projects

Example 1 with MutableInodeDirectory

use of alluxio.master.file.meta.MutableInodeDirectory in project alluxio by Alluxio.

the class CachingInodeStoreMockedBackingStoreTest method edgeIndexTest.

@Test
public void edgeIndexTest() throws Exception {
    // Run many concurrent operations, then check that the edge cache's indices are accurate.
    long endTimeMs = System.currentTimeMillis() + 200;
    ThreadLocalRandom random = ThreadLocalRandom.current();
    List<MutableInodeDirectory> dirs = new ArrayList<>();
    for (int i = 1; i < 5; i++) {
        MutableInodeDirectory dir = createInodeDir(i, 0);
        dirs.add(dir);
        mStore.addChild(TEST_INODE_ID, dir);
    }
    AtomicInteger operations = new AtomicInteger(0);
    ExecutorService executor = Executors.newFixedThreadPool(10);
    int numThreads = 10;
    executor.invokeAll(Collections.nCopies(numThreads, () -> {
        while (operations.get() < 10_000 || System.currentTimeMillis() < endTimeMs) {
            // Sometimes add, sometimes delete.
            if (random.nextBoolean()) {
                MutableInodeDirectory dir = createInodeDir(random.nextLong(10, 15), random.nextLong(1, 5));
                mStore.addChild(dir.getParentId(), dir);
            } else {
                mStore.removeChild(dirs.get(random.nextInt(dirs.size())).getId(), Long.toString(random.nextLong(10, 15)));
            }
            operations.incrementAndGet();
            assertTrue(mStore.mEdgeCache.mMap.size() <= CACHE_SIZE + numThreads);
        }
        return null;
    }));
    alluxio.util.CommonUtils.waitFor("eviction thread to finish", () -> mStore.mEdgeCache.mEvictionThread.mIsSleeping);
    mStore.mEdgeCache.verifyIndices();
}

Example 2 with MutableInodeDirectory

use of alluxio.master.file.meta.MutableInodeDirectory in project alluxio by Alluxio.

the class CachingInodeStoreMockedBackingStoreTest method flushToBackingStore.

@Test
public void flushToBackingStore() throws Exception {
    for (long inodeId = 10; inodeId < 10 + CACHE_SIZE / 2; inodeId++) {
        MutableInodeDirectory dir = createInodeDir(inodeId, 0);
        mStore.addChild(0, dir);
    }
    assertEquals(0, Iterables.size(mBackingStore.getChildren(0L)));
    mStore.mEdgeCache.flush();
    mStore.mInodeCache.flush();
    assertEquals(CACHE_SIZE / 2, Iterables.size(mBackingStore.getChildren(0L)));
}

Example 3 with MutableInodeDirectory

use of alluxio.master.file.meta.MutableInodeDirectory in project alluxio by Alluxio.

the class CachingInodeStoreMockedBackingStoreTest method listingCacheAddRemoveEdges.

@Test(timeout = 10000)
public void listingCacheAddRemoveEdges() throws Exception {
    // Perform operations including adding and removing many files within a directory. This test has
    // rooted out some bugs related to cache weight tracking.
    MutableInodeDirectory bigDir = createInodeDir(1, 0);
    mStore.writeNewInode(bigDir);
    for (int i = 1000; i < 1000 + CACHE_SIZE; i++) {
        MutableInodeDirectory subDir = createInodeDir(i, bigDir.getId());
        mStore.addChild(bigDir.getId(), subDir);
        mStore.removeChild(bigDir.getId(), subDir.getName());
    }
    List<MutableInodeDirectory> inodes = new ArrayList<>();
    for (int i = 10; i < 10 + (CACHE_SIZE / 2); i++) {
        MutableInodeDirectory otherDir = createInodeDir(i, 0);
        inodes.add(otherDir);
        mStore.writeNewInode(otherDir);
    }
    for (MutableInodeDirectory inode : inodes) {
        for (int i = 0; i < 10; i++) {
            assertEquals(0, Iterables.size(mStore.getChildIds(inode.getId())));
        }
        verify(mBackingStore, times(0)).getChildIds(inode.getId());
    }
}

Example 4 with MutableInodeDirectory

use of alluxio.master.file.meta.MutableInodeDirectory in project alluxio by Alluxio.

the class CachingInodeStoreMockedBackingStoreTest method listChildrenOverCacheSize.

@Test
public void listChildrenOverCacheSize() {
    for (long inodeId = 10; inodeId < 10 + CACHE_SIZE * 2; inodeId++) {
        MutableInodeDirectory dir = createInodeDir(inodeId, 0);
        mStore.addChild(0, dir);
    }
    assertEquals(CACHE_SIZE * 2, Iterables.size(mStore.getChildren(0L)));
}

Example 5 with MutableInodeDirectory

use of alluxio.master.file.meta.MutableInodeDirectory in project alluxio by Alluxio.

the class CachingInodeStoreMockedBackingStoreTest method listingCacheBigDirEviction.

@Test
public void listingCacheBigDirEviction() throws Exception {
    MutableInodeDirectory bigDir = createInodeDir(1, 0);
    long dirSize = CACHE_SIZE;
    for (int i = 10; i < 10 + dirSize; i++) {
        mStore.addChild(bigDir.getId(), createInodeDir(i, bigDir.getId()));
    }
    // Cache the large directory
    assertEquals(dirSize, Iterables.size(mStore.getChildIds(bigDir.getId())));
    // Perform another operation to trigger eviction
    mStore.addChild(bigDir.getId(), createInodeDir(10000, bigDir.getId()));
    assertFalse(mStore.mListingCache.getCachedChildIds(TEST_INODE_ID).isPresent());
}