Search in sources :

Example 11 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class RecoveryRequiredCheckerTest method shouldWantToRecoverBrokenStore.

@Test
public void shouldWantToRecoverBrokenStore() throws Exception {
    try (FileSystemAbstraction fileSystemAbstraction = createSomeDataAndCrash(storeDir, fileSystem)) {
        PageCache pageCache = pageCacheRule.getPageCache(fileSystemAbstraction);
        RecoveryRequiredChecker recoverer = new RecoveryRequiredChecker(fileSystemAbstraction, pageCache);
        assertThat(recoverer.isRecoveryRequiredAt(storeDir), is(true));
    }
}
Also used : EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 12 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class RecordStorageEngineTest method mustFlushStoresWithGivenIOLimiter.

@Test
public void mustFlushStoresWithGivenIOLimiter() throws Exception {
    IOLimiter limiter = (stamp, completedIOs, swapper) -> 0;
    FileSystemAbstraction fs = fsRule.get();
    AtomicReference<IOLimiter> observedLimiter = new AtomicReference<>();
    PageCache pageCache = new DelegatingPageCache(pageCacheRule.getPageCache(fs)) {

        @Override
        public void flushAndForce(IOLimiter limiter) throws IOException {
            super.flushAndForce(limiter);
            observedLimiter.set(limiter);
        }
    };
    RecordStorageEngine engine = storageEngineRule.getWith(fs, pageCache).build();
    engine.flushAndForce(limiter);
    assertThat(observedLimiter.get(), sameInstance(limiter));
}
Also used : RecordFormat(org.neo4j.kernel.impl.store.format.RecordFormat) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) StoreFileMetadata(org.neo4j.storageengine.api.StoreFileMetadata) EphemeralFileSystemRule(org.neo4j.test.rule.fs.EphemeralFileSystemRule) Exceptions(org.neo4j.helpers.Exceptions) BatchTransactionApplierFacade(org.neo4j.kernel.impl.api.BatchTransactionApplierFacade) IOLimiter(org.neo4j.io.pagecache.IOLimiter) AtomicReference(java.util.concurrent.atomic.AtomicReference) TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) DelegatingPageCache(org.neo4j.io.pagecache.DelegatingPageCache) RecordStorageEngineRule(org.neo4j.test.rule.RecordStorageEngineRule) Assert.assertSame(org.junit.Assert.assertSame) Assert.assertThat(org.junit.Assert.assertThat) Mockito.doThrow(org.mockito.Mockito.doThrow) PageCacheRule(org.neo4j.test.rule.PageCacheRule) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) TransactionApplicationMode(org.neo4j.storageengine.api.TransactionApplicationMode) CountsTracker(org.neo4j.kernel.impl.store.counts.CountsTracker) Assert.fail(org.junit.Assert.fail) StoreFile(org.neo4j.kernel.impl.storemigration.StoreFile) PageCache(org.neo4j.io.pagecache.PageCache) UnderlyingStorageException(org.neo4j.kernel.impl.store.UnderlyingStorageException) BatchTransactionApplier(org.neo4j.kernel.impl.api.BatchTransactionApplier) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) Assert.assertNotNull(org.junit.Assert.assertNotNull) Collection(java.util.Collection) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) File(java.io.File) Mockito.verify(org.mockito.Mockito.verify) RuleChain(org.junit.rules.RuleChain) Matchers.any(org.mockito.Matchers.any) Rule(org.junit.Rule) CountsAccessor(org.neo4j.kernel.impl.api.CountsAccessor) StoreFileType(org.neo4j.kernel.impl.storemigration.StoreFileType) DatabaseHealth(org.neo4j.kernel.internal.DatabaseHealth) Matchers.sameInstance(org.hamcrest.Matchers.sameInstance) StoreType(org.neo4j.kernel.impl.store.StoreType) Matchers.equalTo(org.hamcrest.Matchers.equalTo) CommandsToApply(org.neo4j.storageengine.api.CommandsToApply) Optional(java.util.Optional) FakeCommitment(org.neo4j.kernel.impl.transaction.log.FakeCommitment) Matchers.anyOf(org.hamcrest.Matchers.anyOf) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Mockito.mock(org.mockito.Mockito.mock) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DelegatingPageCache(org.neo4j.io.pagecache.DelegatingPageCache) IOLimiter(org.neo4j.io.pagecache.IOLimiter) AtomicReference(java.util.concurrent.atomic.AtomicReference) DelegatingPageCache(org.neo4j.io.pagecache.DelegatingPageCache) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 13 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class ConfigurableStandalonePageCacheFactoryTest method mustAutomaticallyStartEvictionThread.

@Test(timeout = 10000)
public void mustAutomaticallyStartEvictionThread() throws IOException {
    try (FileSystemAbstraction fs = new DelegateFileSystemAbstraction(Jimfs.newFileSystem(jimConfig()))) {
        File file = new File("/a").getCanonicalFile();
        fs.create(file).close();
        try (PageCache cache = ConfigurableStandalonePageCacheFactory.createPageCache(fs);
            PagedFile pf = cache.map(file, 4096);
            PageCursor cursor = pf.io(0, PagedFile.PF_SHARED_WRITE_LOCK)) {
            // If the eviction thread has not been started, then this test will block forever.
            for (int i = 0; i < 10_000; i++) {
                assertTrue(cursor.next());
                cursor.putInt(42);
            }
        }
    }
}
Also used : DelegateFileSystemAbstraction(org.neo4j.io.fs.DelegateFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) PagedFile(org.neo4j.io.pagecache.PagedFile) DelegateFileSystemAbstraction(org.neo4j.io.fs.DelegateFileSystemAbstraction) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) PageCursor(org.neo4j.io.pagecache.PageCursor) Test(org.junit.Test)

Example 14 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class IdGeneratorImplTest method shouldForceStickyMark.

@Test
public void shouldForceStickyMark() throws Exception {
    // GIVEN
    try (FileSystemAbstraction fs = new DefaultFileSystemAbstraction()) {
        File dir = new File("target/test-data/" + getClass().getName());
        fs.mkdirs(dir);
        File file = new File(dir, "ids");
        fs.deleteFile(file);
        IdGeneratorImpl.createGenerator(fs, file, 0, false);
        // WHEN opening the id generator, where the jvm crashes right after
        executeSubProcess(getClass(), 1, MINUTES, file.getAbsolutePath());
        // THEN
        try {
            IdGeneratorImpl.readHighId(fs, file);
            fail("Should have thrown, saying something with sticky generator");
        } catch (InvalidIdGeneratorException e) {
        // THEN Good
        }
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) InvalidIdGeneratorException(org.neo4j.kernel.impl.store.InvalidIdGeneratorException) File(java.io.File) Test(org.junit.Test)

Example 15 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class TestGrowingFileMemoryMapping method shouldGrowAFileWhileContinuingToMemoryMapNewRegions.

@Test
public void shouldGrowAFileWhileContinuingToMemoryMapNewRegions() throws Exception {
    // don't run on windows because memory mapping doesn't work properly there
    assumeTrue(!SystemUtils.IS_OS_WINDOWS);
    // given
    int NUMBER_OF_RECORDS = 1000000;
    File storeDir = testDirectory.graphDbDir();
    Config config = Config.embeddedDefaults(stringMap(pagecache_memory.name(), mmapSize(NUMBER_OF_RECORDS, NodeRecordFormat.RECORD_SIZE)));
    FileSystemAbstraction fileSystemAbstraction = fileSystemRule.get();
    DefaultIdGeneratorFactory idGeneratorFactory = new DefaultIdGeneratorFactory(fileSystemAbstraction);
    PageCache pageCache = pageCacheRule.getPageCache(fileSystemAbstraction, config);
    StoreFactory storeFactory = new StoreFactory(storeDir, config, idGeneratorFactory, pageCache, fileSystemAbstraction, NullLogProvider.getInstance());
    NeoStores neoStores = storeFactory.openAllNeoStores(true);
    NodeStore nodeStore = neoStores.getNodeStore();
    // when
    int iterations = 2 * NUMBER_OF_RECORDS;
    long startingId = nodeStore.nextId();
    long nodeId = startingId;
    for (int i = 0; i < iterations; i++) {
        NodeRecord record = new NodeRecord(nodeId, false, i, 0);
        record.setInUse(true);
        nodeStore.updateRecord(record);
        nodeId = nodeStore.nextId();
    }
    // then
    NodeRecord record = new NodeRecord(0, false, 0, 0);
    for (int i = 0; i < iterations; i++) {
        record.setId(startingId + i);
        nodeStore.getRecord(i, record, NORMAL);
        assertTrue("record[" + i + "] should be in use", record.inUse());
        assertThat("record[" + i + "] should have nextRelId of " + i, record.getNextRel(), is((long) i));
    }
    neoStores.close();
}
Also used : NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) DefaultIdGeneratorFactory(org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Aggregations

FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)125 File (java.io.File)88 Test (org.junit.Test)82 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)34 IOException (java.io.IOException)28 Config (org.neo4j.kernel.configuration.Config)23 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)22 PageCache (org.neo4j.io.pagecache.PageCache)22 DelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction)20 ByteBuffer (java.nio.ByteBuffer)13 StoreChannel (org.neo4j.io.fs.StoreChannel)11 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)10 UncloseableDelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.UncloseableDelegatingFileSystemAbstraction)9 DefaultIdGeneratorFactory (org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory)9 OutputStream (java.io.OutputStream)8 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)8 DelegatingStoreChannel (org.neo4j.graphdb.mockfs.DelegatingStoreChannel)8 Map (java.util.Map)7 Matchers.containsString (org.hamcrest.Matchers.containsString)7 AdversarialPagedFile (org.neo4j.adversaries.pagecache.AdversarialPagedFile)7