Search in sources :

Example 1 with DelegatingStoreChannel

use of org.neo4j.graphdb.mockfs.DelegatingStoreChannel in project neo4j by neo4j.

the class MuninnPageCacheTest method mustUnblockPageFaultersWhenEvictionGetsException.

@Test(timeout = SEMI_LONG_TIMEOUT_MILLIS)
public void mustUnblockPageFaultersWhenEvictionGetsException() throws Exception {
    writeInitialDataTo(file("a"));
    FileSystemAbstraction fs = new DelegatingFileSystemAbstraction(this.fs) {

        @Override
        public StoreChannel open(File fileName, String mode) throws IOException {
            return new DelegatingStoreChannel(super.open(fileName, mode)) {

                @Override
                public void writeAll(ByteBuffer src, long position) throws IOException {
                    throw new IOException("uh-oh...");
                }
            };
        }
    };
    MuninnPageCache pageCache = createPageCache(fs, 2, 8, PageCacheTracer.NULL, DefaultPageCursorTracerSupplier.INSTANCE);
    final PagedFile pagedFile = pageCache.map(file("a"), 8);
    // all writes.
    try (PageCursor cursor = pagedFile.io(0, PF_SHARED_WRITE_LOCK)) {
        for (int i = 0; i < 1000; i++) {
            assertTrue(cursor.next());
        }
        fail("Expected an exception at this point");
    } catch (IOException ignore) {
    // Good.
    }
}
Also used : DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) PagedFile(org.neo4j.io.pagecache.PagedFile) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) IOException(java.io.IOException) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) PageCursor(org.neo4j.io.pagecache.PageCursor) PageCacheTest(org.neo4j.io.pagecache.PageCacheTest) Test(org.junit.Test)

Example 2 with DelegatingStoreChannel

use of org.neo4j.graphdb.mockfs.DelegatingStoreChannel in project neo4j by neo4j.

the class PageCacheTest method pageFaultForReadMustThrowIfOutOfStorageSpace.

@Test(timeout = SEMI_LONG_TIMEOUT_MILLIS)
public void pageFaultForReadMustThrowIfOutOfStorageSpace() throws IOException {
    generateFileWithRecords(file("a"), recordCount, recordSize);
    final AtomicInteger writeCounter = new AtomicInteger();
    FileSystemAbstraction fs = new DelegatingFileSystemAbstraction(this.fs) {

        @Override
        public StoreChannel open(File fileName, String mode) throws IOException {
            return new DelegatingStoreChannel(super.open(fileName, mode)) {

                @Override
                public void writeAll(ByteBuffer src, long position) throws IOException {
                    if (writeCounter.incrementAndGet() >= 1) {
                        throw new IOException("No space left on device");
                    }
                    super.writeAll(src, position);
                }
            };
        }
    };
    getPageCache(fs, maxPages, pageCachePageSize, PageCacheTracer.NULL, PageCursorTracerSupplier.NULL);
    PagedFile pagedFile = pageCache.map(file("a"), filePageSize);
    // Create 1 dirty page
    try (PageCursor cursor = pagedFile.io(0, PF_SHARED_WRITE_LOCK)) {
        assertTrue(cursor.next());
    }
    // Read pages until the dirty page gets flushed
    try (PageCursor cursor = pagedFile.io(0, PF_SHARED_READ_LOCK)) {
        //noinspection InfiniteLoopStatement
        for (; ; ) {
            expectedException.expect(IOException.class);
            //noinspection StatementWithEmptyBody
            while (cursor.next()) {
            // Profound and interesting I/O.
            }
            // Use rewind if we get to the end, because it is non-
            // deterministic which pages get evicted and when.
            cursor.rewind();
        }
    } finally {
        // Unmapping and closing the PageCache will want to flush,
        // but we can't do that with a full drive.
        pageCache = null;
    }
}
Also used : DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) Long.toHexString(java.lang.Long.toHexString) Matchers.containsString(org.hamcrest.Matchers.containsString) IOException(java.io.IOException) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 3 with DelegatingStoreChannel

use of org.neo4j.graphdb.mockfs.DelegatingStoreChannel in project neo4j by neo4j.

the class PageCacheTest method pageFaultForWriteMustThrowIfOutOfStorageSpace.

@Test(timeout = SHORT_TIMEOUT_MILLIS)
public void pageFaultForWriteMustThrowIfOutOfStorageSpace() throws IOException {
    final AtomicInteger writeCounter = new AtomicInteger();
    FileSystemAbstraction fs = new DelegatingFileSystemAbstraction(this.fs) {

        @Override
        public StoreChannel open(File fileName, String mode) throws IOException {
            return new DelegatingStoreChannel(super.open(fileName, mode)) {

                @Override
                public void writeAll(ByteBuffer src, long position) throws IOException {
                    if (writeCounter.incrementAndGet() > 10) {
                        throw new IOException("No space left on device");
                    }
                    super.writeAll(src, position);
                }
            };
        }
    };
    fs.create(file("a")).close();
    getPageCache(fs, maxPages, pageCachePageSize, PageCacheTracer.NULL, PageCursorTracerSupplier.NULL);
    PagedFile pagedFile = pageCache.map(file("a"), filePageSize);
    try (PageCursor cursor = pagedFile.io(0, PF_SHARED_WRITE_LOCK)) {
        expectedException.expect(IOException.class);
        //noinspection StatementWithEmptyBody
        while (cursor.next()) {
        // Profound and interesting I/O.
        }
    } finally {
        // Unmapping and closing the PageCache will want to flush,
        // but we can't do that with a full drive.
        pageCache = null;
    }
}
Also used : DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) Long.toHexString(java.lang.Long.toHexString) Matchers.containsString(org.hamcrest.Matchers.containsString) IOException(java.io.IOException) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 4 with DelegatingStoreChannel

use of org.neo4j.graphdb.mockfs.DelegatingStoreChannel in project neo4j by neo4j.

the class PageCacheTest method mustNotFlushCleanPagesWhenEvicting.

@Test(timeout = SHORT_TIMEOUT_MILLIS)
public void mustNotFlushCleanPagesWhenEvicting() throws Exception {
    generateFileWithRecords(file("a"), recordCount, recordSize);
    final AtomicBoolean observedWrite = new AtomicBoolean();
    FileSystemAbstraction fs = new DelegatingFileSystemAbstraction(this.fs) {

        @Override
        public StoreChannel open(File fileName, String mode) throws IOException {
            StoreChannel channel = super.open(fileName, mode);
            return new DelegatingStoreChannel(channel) {

                @Override
                public int write(ByteBuffer src, long position) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }

                @Override
                public long write(ByteBuffer[] srcs, int offset, int length) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }

                @Override
                public void writeAll(ByteBuffer src, long position) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }

                @Override
                public void writeAll(ByteBuffer src) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }

                @Override
                public int write(ByteBuffer src) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }

                @Override
                public long write(ByteBuffer[] srcs) throws IOException {
                    observedWrite.set(true);
                    throw new IOException("not allowed");
                }
            };
        }
    };
    getPageCache(fs, maxPages, pageCachePageSize, PageCacheTracer.NULL, PageCursorTracerSupplier.NULL);
    try (PagedFile pagedFile = pageCache.map(file("a"), filePageSize);
        PageCursor cursor = pagedFile.io(0, PF_SHARED_READ_LOCK)) {
        while (cursor.next()) {
            verifyRecordsMatchExpected(cursor);
        }
    }
    assertFalse(observedWrite.get());
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) StoreChannel(org.neo4j.io.fs.StoreChannel) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) Long.toHexString(java.lang.Long.toHexString) Matchers.containsString(org.hamcrest.Matchers.containsString) IOException(java.io.IOException) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 5 with DelegatingStoreChannel

use of org.neo4j.graphdb.mockfs.DelegatingStoreChannel in project neo4j by neo4j.

the class PageCacheTest method mustRecoverFromFullDriveWhenMoreStorageBecomesAvailable.

@Test(timeout = SHORT_TIMEOUT_MILLIS)
public void mustRecoverFromFullDriveWhenMoreStorageBecomesAvailable() throws IOException {
    final AtomicBoolean hasSpace = new AtomicBoolean();
    FileSystemAbstraction fs = new DelegatingFileSystemAbstraction(this.fs) {

        @Override
        public StoreChannel open(File fileName, String mode) throws IOException {
            return new DelegatingStoreChannel(super.open(fileName, mode)) {

                @Override
                public void writeAll(ByteBuffer src, long position) throws IOException {
                    if (!hasSpace.get()) {
                        throw new IOException("No space left on device");
                    }
                    super.writeAll(src, position);
                }
            };
        }
    };
    fs.create(file("a")).close();
    getPageCache(fs, maxPages, pageCachePageSize, PageCacheTracer.NULL, PageCursorTracerSupplier.NULL);
    PagedFile pagedFile = pageCache.map(file("a"), filePageSize);
    try (PageCursor cursor = pagedFile.io(0, PF_SHARED_WRITE_LOCK)) {
        //noinspection InfiniteLoopStatement
        for (; ; ) // Keep writing until we get an exception! (when the cache starts evicting stuff)
        {
            assertTrue(cursor.next());
            writeRecords(cursor);
        }
    } catch (IOException ignore) {
    // We're not out of space! Salty tears...
    }
    // Fix the situation:
    hasSpace.set(true);
    // Closing the last reference of a paged file implies a flush, and it mustn't throw:
    pagedFile.close();
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) DelegatingStoreChannel(org.neo4j.graphdb.mockfs.DelegatingStoreChannel) Long.toHexString(java.lang.Long.toHexString) Matchers.containsString(org.hamcrest.Matchers.containsString) IOException(java.io.IOException) DelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Aggregations

File (java.io.File)9 Test (org.junit.Test)9 DelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction)9 DelegatingStoreChannel (org.neo4j.graphdb.mockfs.DelegatingStoreChannel)9 IOException (java.io.IOException)8 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)8 ByteBuffer (java.nio.ByteBuffer)7 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)6 Long.toHexString (java.lang.Long.toHexString)5 Matchers.containsString (org.hamcrest.Matchers.containsString)5 AdversarialPagedFile (org.neo4j.adversaries.pagecache.AdversarialPagedFile)5 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)3 StoreChannel (org.neo4j.io.fs.StoreChannel)3 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 PrintStream (java.io.PrintStream)1 FileLock (java.nio.channels.FileLock)1 OverlappingFileLockException (java.nio.channels.OverlappingFileLockException)1 ArrayList (java.util.ArrayList)1 Arrays.asList (java.util.Arrays.asList)1