Search in sources :

Example 46 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class LuceneSchemaIndexCorruptionTest method newFaultySchemaIndexProvider.

private LuceneSchemaIndexProvider newFaultySchemaIndexProvider(long faultyIndexId, Exception error) {
    DirectoryFactory directoryFactory = mock(DirectoryFactory.class);
    File indexRootFolder = testDirectory.graphDbDir();
    FaultyIndexStorageFactory storageFactory = new FaultyIndexStorageFactory(faultyIndexId, error, directoryFactory, indexRootFolder);
    return new LuceneSchemaIndexProvider(fs.get(), directoryFactory, indexRootFolder, logProvider, Config.defaults(), OperationalMode.single) {

        @Override
        protected IndexStorageFactory buildIndexStorageFactory(FileSystemAbstraction fileSystem, DirectoryFactory directoryFactory, File schemaIndexStoreFolder) {
            return storageFactory;
        }
    };
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DirectoryFactory(org.neo4j.kernel.api.impl.index.storage.DirectoryFactory) File(java.io.File)

Example 47 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class RebuildCountsTest method shouldRebuildMissingCountsStoreOnStart.

// Indexing counts are recovered/rebuild in IndexingService.start() and are not tested here
@Test
public void shouldRebuildMissingCountsStoreOnStart() throws IOException {
    // given
    createAliensAndHumans();
    // when
    FileSystemAbstraction fs = shutdown();
    deleteCounts(fs);
    restart(fs);
    // then
    CountsTracker tracker = counts();
    assertEquals(ALIENS + HUMANS, tracker.nodeCount(-1, newDoubleLongRegister()).readSecond());
    assertEquals(ALIENS, tracker.nodeCount(labelId(ALIEN), newDoubleLongRegister()).readSecond());
    assertEquals(HUMANS, tracker.nodeCount(labelId(HUMAN), newDoubleLongRegister()).readSecond());
    // and also
    internalLogProvider.assertAtLeastOnce(inLog(MetaDataStore.class).warn("Missing counts store, rebuilding it."));
}
Also used : UncloseableDelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.UncloseableDelegatingFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) CountsTracker(org.neo4j.kernel.impl.store.counts.CountsTracker) Test(org.junit.Test)

Example 48 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class RebuildCountsTest method shouldRebuildMissingCountsStoreAfterRecovery.

@Test
public void shouldRebuildMissingCountsStoreAfterRecovery() throws IOException {
    // given
    createAliensAndHumans();
    // when
    rotateLog();
    deleteHumans();
    FileSystemAbstraction fs = crash();
    deleteCounts(fs);
    restart(fs);
    // then
    CountsTracker tracker = counts();
    assertEquals(ALIENS, tracker.nodeCount(-1, newDoubleLongRegister()).readSecond());
    assertEquals(ALIENS, tracker.nodeCount(labelId(ALIEN), newDoubleLongRegister()).readSecond());
    assertEquals(0, tracker.nodeCount(labelId(HUMAN), newDoubleLongRegister()).readSecond());
    // and also
    internalLogProvider.assertAtLeastOnce(inLog(MetaDataStore.class).warn("Missing counts store, rebuilding it."));
}
Also used : UncloseableDelegatingFileSystemAbstraction(org.neo4j.graphdb.mockfs.UncloseableDelegatingFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) CountsTracker(org.neo4j.kernel.impl.store.counts.CountsTracker) Test(org.junit.Test)

Example 49 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class DatabaseStartupTest method startTheDatabaseWithWrongVersionShouldFailWithUpgradeNotAllowed.

@Test
public void startTheDatabaseWithWrongVersionShouldFailWithUpgradeNotAllowed() throws Throwable {
    // given
    // create a store
    File storeDir = testDirectory.graphDbDir();
    GraphDatabaseService db = new TestGraphDatabaseFactory().newEmbeddedDatabase(storeDir);
    try (Transaction tx = db.beginTx()) {
        db.createNode();
        tx.success();
    }
    db.shutdown();
    // mess up the version in the metadatastore
    try (FileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction();
        PageCache pageCache = StandalonePageCacheFactory.createPageCache(fileSystem)) {
        MetaDataStore.setRecord(pageCache, new File(storeDir, MetaDataStore.DEFAULT_NAME), MetaDataStore.Position.STORE_VERSION, MetaDataStore.versionStringToLong("bad"));
    }
    // when
    try {
        new TestGraphDatabaseFactory().newEmbeddedDatabase(storeDir);
        fail("It should have failed.");
    } catch (RuntimeException ex) {
        // then
        assertTrue(ex.getCause() instanceof LifecycleException);
        assertTrue(ex.getCause().getCause() instanceof UpgradeNotAllowedByConfigurationException);
        assertEquals("Failed to start Neo4j with an older data store version. To enable automatic upgrade, " + "please set configuration parameter \"dbms.allow_format_migration=true\"", ex.getCause().getCause().getMessage());
    }
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) UpgradeNotAllowedByConfigurationException(org.neo4j.kernel.impl.storemigration.UpgradeNotAllowedByConfigurationException) LifecycleException(org.neo4j.kernel.lifecycle.LifecycleException) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Transaction(org.neo4j.graphdb.Transaction) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 50 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class CsvImporter method doImport.

@Override
public void doImport() throws IOException {
    FileSystemAbstraction fs = outsideWorld.fileSystem();
    File storeDir = config.get(DatabaseManagementSystemSettings.database_path);
    File logsDir = config.get(GraphDatabaseSettings.logs_directory);
    File reportFile = new File(reportFileName);
    OutputStream badOutput = new BufferedOutputStream(fs.openAsOutputStream(reportFile, false));
    Collector badCollector = badCollector(badOutput, isIgnoringSomething() ? BadCollector.UNLIMITED_TOLERANCE : 0, collect(ignoreBadRelationships, ignoreDuplicateNodes, ignoreExtraColumns));
    Configuration configuration = importConfiguration(null, false, config);
    CsvInput input = new CsvInput(nodeData(inputEncoding, nodesFiles), defaultFormatNodeFileHeader(), relationshipData(inputEncoding, relationshipsFiles), defaultFormatRelationshipFileHeader(), idType, csvConfiguration(args, false), badCollector, configuration.maxNumberOfProcessors());
    ImportTool.doImport(outsideWorld.errorStream(), outsideWorld.errorStream(), storeDir, logsDir, reportFile, fs, nodesFiles, relationshipsFiles, false, input, config, badOutput, configuration);
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) ImportTool.importConfiguration(org.neo4j.tooling.ImportTool.importConfiguration) Configuration(org.neo4j.unsafe.impl.batchimport.Configuration) ImportTool.csvConfiguration(org.neo4j.tooling.ImportTool.csvConfiguration) BufferedOutputStream(java.io.BufferedOutputStream) OutputStream(java.io.OutputStream) BadCollector(org.neo4j.unsafe.impl.batchimport.input.BadCollector) Collector(org.neo4j.unsafe.impl.batchimport.input.Collector) Collectors.badCollector(org.neo4j.unsafe.impl.batchimport.input.Collectors.badCollector) CsvInput(org.neo4j.unsafe.impl.batchimport.input.csv.CsvInput) File(java.io.File) BufferedOutputStream(java.io.BufferedOutputStream)

Aggregations

FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)125 File (java.io.File)88 Test (org.junit.Test)82 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)34 IOException (java.io.IOException)28 Config (org.neo4j.kernel.configuration.Config)23 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)22 PageCache (org.neo4j.io.pagecache.PageCache)22 DelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction)20 ByteBuffer (java.nio.ByteBuffer)13 StoreChannel (org.neo4j.io.fs.StoreChannel)11 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)10 UncloseableDelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.UncloseableDelegatingFileSystemAbstraction)9 DefaultIdGeneratorFactory (org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory)9 OutputStream (java.io.OutputStream)8 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)8 DelegatingStoreChannel (org.neo4j.graphdb.mockfs.DelegatingStoreChannel)8 Map (java.util.Map)7 Matchers.containsString (org.hamcrest.Matchers.containsString)7 AdversarialPagedFile (org.neo4j.adversaries.pagecache.AdversarialPagedFile)7