Search in sources :

Example 56 with DefaultFileSystemAbstraction

use of org.neo4j.io.fs.DefaultFileSystemAbstraction in project neo4j by neo4j.

the class StoreMigratorTest method shouldNotDoActualStoreMigrationBetween3_0_5_and_next.

@Test
public void shouldNotDoActualStoreMigrationBetween3_0_5_and_next() throws Exception {
    // GIVEN a store in vE.H.0 format
    File storeDir = directory.directory();
    new TestGraphDatabaseFactory().newEmbeddedDatabaseBuilder(storeDir).setConfig(GraphDatabaseSettings.record_format, HighLimitV3_0_0.NAME).newGraphDatabase().shutdown();
    Config config = Config.embeddedDefaults(stringMap(pagecache_memory.name(), "8m"));
    try (FileSystemAbstraction fs = new DefaultFileSystemAbstraction();
        PageCache pageCache = new ConfiguringPageCacheFactory(fs, config, NULL, PageCursorTracerSupplier.NULL, NullLog.getInstance()).getOrCreatePageCache()) {
        // For test code sanity
        String fromStoreVersion = StoreVersion.HIGH_LIMIT_V3_0_0.versionString();
        Result hasVersionResult = new StoreVersionCheck(pageCache).hasVersion(new File(storeDir, NEO_STORE.fileName(STORE)), fromStoreVersion);
        assertTrue(hasVersionResult.actualVersion, hasVersionResult.outcome.isSuccessful());
        // WHEN
        StoreMigrator migrator = new StoreMigrator(fs, pageCache, config, NullLogService.getInstance(), NO_INDEX_PROVIDER);
        MigrationProgressMonitor.Section monitor = mock(MigrationProgressMonitor.Section.class);
        File migrationDir = new File(storeDir, "migration");
        fs.mkdirs(migrationDir);
        migrator.migrate(storeDir, migrationDir, monitor, fromStoreVersion, StoreVersion.HIGH_LIMIT_V3_0_6.versionString());
        // THEN
        verifyNoMoreInteractions(monitor);
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) ConfiguringPageCacheFactory(org.neo4j.kernel.impl.pagecache.ConfiguringPageCacheFactory) Result(org.neo4j.kernel.impl.storemigration.StoreVersionCheck.Result) StoreVersionCheck(org.neo4j.kernel.impl.storemigration.StoreVersionCheck) MigrationProgressMonitor(org.neo4j.kernel.impl.storemigration.monitoring.MigrationProgressMonitor) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 57 with DefaultFileSystemAbstraction

use of org.neo4j.io.fs.DefaultFileSystemAbstraction in project neo4j by neo4j.

the class BatchInserterImplTest method testFailsOnExistingStoreLockFile.

@Test
public void testFailsOnExistingStoreLockFile() throws IOException {
    // Given
    File parent = testDirectory.graphDbDir();
    try (FileSystemAbstraction fileSystemAbstraction = new DefaultFileSystemAbstraction();
        StoreLocker lock = new StoreLocker(fileSystemAbstraction)) {
        lock.checkLock(parent);
        // Then
        expected.expect(StoreLockException.class);
        expected.expectMessage("Unable to obtain lock on store lock file");
        // When
        BatchInserters.inserter(parent.getAbsoluteFile(), fileSystemAbstraction);
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) StoreLocker(org.neo4j.kernel.internal.StoreLocker) File(java.io.File) Test(org.junit.Test)

Example 58 with DefaultFileSystemAbstraction

use of org.neo4j.io.fs.DefaultFileSystemAbstraction in project neo4j by neo4j.

the class RotatingFileOutputStreamSupplierTest method rotationShouldNotDeadlockOnListener.

@Test(timeout = TEST_TIMEOUT_MILLIS)
public void rotationShouldNotDeadlockOnListener() throws Exception {
    String logContent = "Output file created";
    final AtomicReference<Exception> listenerException = new AtomicReference<>(null);
    CountDownLatch latch = new CountDownLatch(1);
    RotationListener listener = new RotationListener() {

        @Override
        public void outputFileCreated(OutputStream out) {
            try {
                Thread thread = new Thread(() -> {
                    try {
                        out.write(logContent.getBytes());
                        out.flush();
                    } catch (IOException e) {
                        listenerException.set(e);
                    }
                });
                thread.start();
                thread.join();
            } catch (Exception e) {
                listenerException.set(e);
            }
            super.outputFileCreated(out);
        }

        @Override
        public void rotationCompleted(OutputStream out) {
            latch.countDown();
        }
    };
    ExecutorService executor = Executors.newSingleThreadExecutor();
    DefaultFileSystemAbstraction defaultFileSystemAbstraction = new DefaultFileSystemAbstraction();
    RotatingFileOutputStreamSupplier supplier = new RotatingFileOutputStreamSupplier(defaultFileSystemAbstraction, logFile, 0, 0, 10, executor, listener);
    OutputStream outputStream = supplier.get();
    LockingPrintWriter lockingPrintWriter = new LockingPrintWriter(outputStream);
    lockingPrintWriter.withLock(() -> {
        supplier.rotate();
        latch.await();
        return Void.TYPE;
    });
    executor.shutdown();
    boolean terminated = executor.awaitTermination(TEST_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS);
    if (!terminated) {
        throw new IllegalStateException("Rotation execution failed to complete within reasonable time.");
    }
    List<String> strings = Files.readAllLines(logFile.toPath());
    String actual = String.join("", strings);
    assertEquals(logContent, actual);
    assertNull(listenerException.get());
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) AdversarialOutputStream(org.neo4j.adversaries.fs.AdversarialOutputStream) OutputStream(java.io.OutputStream) AtomicReference(java.util.concurrent.atomic.AtomicReference) IOException(java.io.IOException) CountDownLatch(java.util.concurrent.CountDownLatch) FileNotFoundException(java.io.FileNotFoundException) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) ClosedChannelException(java.nio.channels.ClosedChannelException) IOException(java.io.IOException) RotationListener(org.neo4j.logging.RotatingFileOutputStreamSupplier.RotationListener) ExecutorService(java.util.concurrent.ExecutorService) Test(org.junit.Test)

Example 59 with DefaultFileSystemAbstraction

use of org.neo4j.io.fs.DefaultFileSystemAbstraction in project neo4j by neo4j.

the class AbstractInProcessServerBuilder method newServer.

@Override
public ServerControls newServer() {
    try (FileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction()) {
        final OutputStream logOutputStream;
        try {
            logOutputStream = createOrOpenAsOuputStream(fileSystem, new File(serverFolder, "neo4j.log"), true);
        } catch (IOException e) {
            throw new RuntimeException("Unable to create log file", e);
        }
        config.put(ServerSettings.third_party_packages.name(), toStringForThirdPartyPackageProperty(extensions.toList()));
        final FormattedLogProvider userLogProvider = FormattedLogProvider.toOutputStream(logOutputStream);
        GraphDatabaseDependencies dependencies = GraphDatabaseDependencies.newDependencies();
        Iterable<KernelExtensionFactory<?>> kernelExtensions = append(new Neo4jHarnessExtensions(procedures), dependencies.kernelExtensions());
        dependencies = dependencies.kernelExtensions(kernelExtensions).userLogProvider(userLogProvider);
        AbstractNeoServer neoServer = createNeoServer(config, dependencies, userLogProvider);
        InProcessServerControls controls = new InProcessServerControls(serverFolder, neoServer, logOutputStream);
        controls.start();
        try {
            fixtures.applyTo(controls);
        } catch (Exception e) {
            controls.close();
            throw Exceptions.launderedException(e);
        }
        return controls;
    } catch (IOException e) {
        throw Exceptions.launderedException(e);
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) OutputStream(java.io.OutputStream) IOException(java.io.IOException) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) AbstractNeoServer(org.neo4j.server.AbstractNeoServer) IOException(java.io.IOException) FormattedLogProvider(org.neo4j.logging.FormattedLogProvider) GraphDatabaseDependencies(org.neo4j.kernel.GraphDatabaseDependencies) File(java.io.File)

Example 60 with DefaultFileSystemAbstraction

use of org.neo4j.io.fs.DefaultFileSystemAbstraction in project neo4j by neo4j.

the class DatabaseStartupTest method startTheDatabaseWithWrongVersionShouldFailAlsoWhenUpgradeIsAllowed.

@Test
public void startTheDatabaseWithWrongVersionShouldFailAlsoWhenUpgradeIsAllowed() throws Throwable {
    // given
    // create a store
    File storeDir = testDirectory.graphDbDir();
    GraphDatabaseService db = new TestGraphDatabaseFactory().newEmbeddedDatabase(storeDir);
    try (Transaction tx = db.beginTx()) {
        db.createNode();
        tx.success();
    }
    db.shutdown();
    // mess up the version in the metadatastore
    String badStoreVersion = "bad";
    try (FileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction();
        PageCache pageCache = StandalonePageCacheFactory.createPageCache(fileSystem)) {
        MetaDataStore.setRecord(pageCache, new File(storeDir, MetaDataStore.DEFAULT_NAME), MetaDataStore.Position.STORE_VERSION, MetaDataStore.versionStringToLong(badStoreVersion));
    }
    // when
    try {
        new TestGraphDatabaseFactory().newEmbeddedDatabaseBuilder(storeDir).setConfig(GraphDatabaseSettings.allow_store_upgrade, "true").newGraphDatabase();
        fail("It should have failed.");
    } catch (RuntimeException ex) {
        // then
        assertTrue(ex.getCause() instanceof LifecycleException);
        assertTrue(ex.getCause().getCause() instanceof StoreUpgrader.UnexpectedUpgradingStoreVersionException);
    }
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) LifecycleException(org.neo4j.kernel.lifecycle.LifecycleException) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Transaction(org.neo4j.graphdb.Transaction) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Aggregations

DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)82 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)43 File (java.io.File)24 Path (java.nio.file.Path)21 PageCache (org.neo4j.io.pagecache.PageCache)21 Test (org.junit.Test)14 Test (org.junit.jupiter.api.Test)13 IOException (java.io.IOException)12 DatabaseLayout (org.neo4j.io.layout.DatabaseLayout)11 Config (org.neo4j.kernel.configuration.Config)11 Config (org.neo4j.configuration.Config)9 ThreadPoolJobScheduler (org.neo4j.test.scheduler.ThreadPoolJobScheduler)8 GraphDatabaseService (org.neo4j.graphdb.GraphDatabaseService)7 Transaction (org.neo4j.graphdb.Transaction)7 PrintStream (java.io.PrintStream)6 Before (org.junit.Before)6 CommandFailed (org.neo4j.commandline.admin.CommandFailed)6 Args (org.neo4j.helpers.Args)6 StandalonePageCacheFactory.createPageCache (org.neo4j.io.pagecache.impl.muninn.StandalonePageCacheFactory.createPageCache)6 JobScheduler (org.neo4j.scheduler.JobScheduler)6