Search in sources :

Example 1 with LegacyStoreVersionCheck

use of org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck in project neo4j by neo4j.

the class DatabaseMigrator method migrate.

/**
     * Performs construction of {@link StoreUpgrader} and all of the necessary participants and performs store
     * migration if that is required.
     * @param storeDir store to migrate
     */
public void migrate(File storeDir) {
    LogProvider logProvider = logService.getInternalLogProvider();
    UpgradableDatabase upgradableDatabase = new UpgradableDatabase(fs, new StoreVersionCheck(pageCache), new LegacyStoreVersionCheck(fs), format);
    StoreUpgrader storeUpgrader = new StoreUpgrader(upgradableDatabase, progressMonitor, config, fs, pageCache, logProvider);
    StoreMigrationParticipant schemaMigrator = schemaIndexProvider.storeMigrationParticipant(fs, pageCache, labelScanStoreProvider);
    LegacyIndexMigrator legacyIndexMigrator = new LegacyIndexMigrator(fs, indexProviders, logProvider);
    StoreMigrator storeMigrator = new StoreMigrator(fs, pageCache, config, logService, schemaIndexProvider);
    storeUpgrader.addParticipant(schemaMigrator);
    storeUpgrader.addParticipant(legacyIndexMigrator);
    storeUpgrader.addParticipant(storeMigrator);
    storeUpgrader.migrateIfNeeded(storeDir);
}
Also used : LogProvider(org.neo4j.logging.LogProvider) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) LegacyIndexMigrator(org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator) StoreMigrator(org.neo4j.kernel.impl.storemigration.participant.StoreMigrator) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck)

Example 2 with LegacyStoreVersionCheck

use of org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck in project neo4j by neo4j.

the class MigrationTestUtils method allStoreFilesHaveNoTrailer.

public static boolean allStoreFilesHaveNoTrailer(FileSystemAbstraction fs, File dir) {
    final Iterable<StoreFile> storeFilesWithGivenVersions = Iterables.filter(ALL_EXCEPT_COUNTS_STORE, StoreFile.legacyStoreFilesForVersion(StandardV3_0.STORE_VERSION));
    LegacyStoreVersionCheck legacyStoreVersionCheck = new LegacyStoreVersionCheck(fs);
    boolean success = true;
    for (StoreFile storeFile : storeFilesWithGivenVersions) {
        File file = new File(dir, storeFile.storeFileName());
        StoreVersionCheck.Result result = legacyStoreVersionCheck.hasVersion(file, StandardV3_0.STORE_VERSION, storeFile.isOptional());
        success &= result.outcome == Outcome.unexpectedStoreVersion || result.outcome == Outcome.storeVersionNotFound;
    }
    return success;
}
Also used : LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) File(java.io.File)

Example 3 with LegacyStoreVersionCheck

use of org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck in project neo4j by neo4j.

the class StoreMigratorIT method shouldComputeTheLastTxInfoCorrectly.

@Test
public void shouldComputeTheLastTxInfoCorrectly() throws Exception {
    // given
    File storeDirectory = directory.graphDbDir();
    File prepare = directory.directory("prepare");
    MigrationTestUtils.prepareSampleLegacyDatabase(version, fs, storeDirectory, prepare);
    // and a state of the migration saying that it has done the actual migration
    LogService logService = NullLogService.getInstance();
    PageCache pageCache = pageCacheRule.getPageCache(fs);
    UpgradableDatabase upgradableDatabase = new UpgradableDatabase(fs, new StoreVersionCheck(pageCache), new LegacyStoreVersionCheck(fs), selectFormat());
    String versionToMigrateFrom = upgradableDatabase.checkUpgradeable(storeDirectory).storeVersion();
    SilentMigrationProgressMonitor progressMonitor = new SilentMigrationProgressMonitor();
    StoreMigrator migrator = new StoreMigrator(fs, pageCache, CONFIG, logService, schemaIndexProvider);
    File migrationDir = new File(storeDirectory, StoreUpgrader.MIGRATION_DIRECTORY);
    fs.mkdir(migrationDir);
    // when
    migrator.migrate(storeDirectory, migrationDir, progressMonitor.startSection("section"), versionToMigrateFrom, upgradableDatabase.currentVersion());
    // then
    assertTrue(txIdComparator.apply(migrator.readLastTxInformation(migrationDir)));
}
Also used : StoreVersionCheck(org.neo4j.kernel.impl.storemigration.StoreVersionCheck) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) UpgradableDatabase(org.neo4j.kernel.impl.storemigration.UpgradableDatabase) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) File(java.io.File) SilentMigrationProgressMonitor(org.neo4j.kernel.impl.storemigration.monitoring.SilentMigrationProgressMonitor) LogService(org.neo4j.kernel.impl.logging.LogService) NullLogService(org.neo4j.kernel.impl.logging.NullLogService) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 4 with LegacyStoreVersionCheck

use of org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck in project neo4j by neo4j.

the class StoreMigrationIT method data.

@Parameterized.Parameters(name = "Migrate: {0}->{1}")
public static Iterable<Object[]> data() throws IOException {
    FileSystemAbstraction fs = fileSystemRule.get();
    PageCache pageCache = pageCacheRule.getPageCache(fs);
    File dir = TestDirectory.testDirectory(StoreMigrationIT.class).prepareDirectoryForTest("migration");
    StoreVersionCheck storeVersionCheck = new StoreVersionCheck(pageCache);
    LegacyStoreVersionCheck legacyStoreVersionCheck = new LegacyStoreVersionCheck(fs);
    List<Object[]> data = new ArrayList<>();
    ArrayList<RecordFormats> recordFormatses = new ArrayList<>();
    RecordFormatSelector.allFormats().forEach((f) -> addIfNotThere(f, recordFormatses));
    for (RecordFormats toFormat : recordFormatses) {
        UpgradableDatabase upgradableDatabase = new UpgradableDatabase(fs, storeVersionCheck, legacyStoreVersionCheck, toFormat);
        for (RecordFormats fromFormat : recordFormatses) {
            File db = new File(dir, baseDirName(toFormat, fromFormat));
            try {
                createDb(fromFormat, db);
                if (!upgradableDatabase.hasCurrentVersion(db)) {
                    upgradableDatabase.checkUpgradeable(db);
                    data.add(new Object[] { fromFormat, toFormat });
                }
            } catch (Exception e) {
            //This means that the combination is not migratable.
            }
            fs.deleteRecursively(db);
        }
    }
    return data;
}
Also used : LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) RecordFormats(org.neo4j.kernel.impl.store.format.RecordFormats) ArrayList(java.util.ArrayList) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) IOException(java.io.IOException) ConsistencyCheckIncompleteException(org.neo4j.consistency.checking.full.ConsistencyCheckIncompleteException)

Example 5 with LegacyStoreVersionCheck

use of org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck in project neo4j by neo4j.

the class StoreUpgraderInterruptionTestIT method shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMovingFiles.

@Test
public void shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMovingFiles() throws IOException, ConsistencyCheckIncompleteException {
    File workingDirectory = directory.directory("working");
    File prepareDirectory = directory.directory("prepare");
    MigrationTestUtils.prepareSampleLegacyDatabase(version, fs, workingDirectory, prepareDirectory);
    PageCache pageCache = pageCacheRule.getPageCache(fs);
    StoreVersionCheck check = new StoreVersionCheck(pageCache);
    UpgradableDatabase upgradableDatabase = new UpgradableDatabase(fs, check, new LegacyStoreVersionCheck(fs), Standard.LATEST_RECORD_FORMATS);
    SilentMigrationProgressMonitor progressMonitor = new SilentMigrationProgressMonitor();
    LogService logService = NullLogService.getInstance();
    StoreMigrator failingStoreMigrator = new StoreMigrator(fs, pageCache, CONFIG, logService, schemaIndexProvider) {

        @Override
        public void moveMigratedFiles(File migrationDir, File storeDir, String versionToUpgradeFrom, String versionToMigrateTo) throws IOException {
            super.moveMigratedFiles(migrationDir, storeDir, versionToUpgradeFrom, versionToMigrateTo);
            throw new RuntimeException("This upgrade is failing");
        }
    };
    assertEquals(!StandardV2_3.STORE_VERSION.equals(version), allLegacyStoreFilesHaveVersion(fs, workingDirectory, version));
    try {
        newUpgrader(upgradableDatabase, pageCache, progressMonitor, createIndexMigrator(), failingStoreMigrator).migrateIfNeeded(workingDirectory);
        fail("Should throw exception");
    } catch (RuntimeException e) {
        assertEquals("This upgrade is failing", e.getMessage());
    }
    assertTrue(checkNeoStoreHasDefaultFormatVersion(check, workingDirectory));
    assertTrue(allStoreFilesHaveNoTrailer(fs, workingDirectory));
    progressMonitor = new SilentMigrationProgressMonitor();
    StoreMigrator migrator = new StoreMigrator(fs, pageCache, CONFIG, logService, schemaIndexProvider);
    newUpgrader(upgradableDatabase, pageCache, progressMonitor, createIndexMigrator(), migrator).migrateIfNeeded(workingDirectory);
    assertTrue(checkNeoStoreHasDefaultFormatVersion(check, workingDirectory));
    assertTrue(allStoreFilesHaveNoTrailer(fs, workingDirectory));
    pageCache.close();
    // Since consistency checker is in read only mode we need to start/stop db to generate label scan store.
    startStopDatabase(workingDirectory);
    assertConsistentStore(workingDirectory);
}
Also used : StoreVersionCheck(org.neo4j.kernel.impl.storemigration.StoreVersionCheck) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) UpgradableDatabase(org.neo4j.kernel.impl.storemigration.UpgradableDatabase) StoreMigrator(org.neo4j.kernel.impl.storemigration.participant.StoreMigrator) LegacyStoreVersionCheck(org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck) File(java.io.File) SilentMigrationProgressMonitor(org.neo4j.kernel.impl.storemigration.monitoring.SilentMigrationProgressMonitor) PageCache(org.neo4j.io.pagecache.PageCache) LogService(org.neo4j.kernel.impl.logging.LogService) NullLogService(org.neo4j.kernel.impl.logging.NullLogService) Test(org.junit.Test)

Aggregations

LegacyStoreVersionCheck (org.neo4j.kernel.impl.storemigration.legacystore.LegacyStoreVersionCheck)21 File (java.io.File)17 PageCache (org.neo4j.io.pagecache.PageCache)17 StoreVersionCheck (org.neo4j.kernel.impl.storemigration.StoreVersionCheck)17 UpgradableDatabase (org.neo4j.kernel.impl.storemigration.UpgradableDatabase)17 Test (org.junit.Test)16 SilentMigrationProgressMonitor (org.neo4j.kernel.impl.storemigration.monitoring.SilentMigrationProgressMonitor)8 MigrationTestUtils.truncateFile (org.neo4j.kernel.impl.storemigration.MigrationTestUtils.truncateFile)7 LogService (org.neo4j.kernel.impl.logging.LogService)6 NullLogService (org.neo4j.kernel.impl.logging.NullLogService)6 StoreUpgrader (org.neo4j.kernel.impl.storemigration.StoreUpgrader)6 StoreMigrator (org.neo4j.kernel.impl.storemigration.participant.StoreMigrator)4 IOException (java.io.IOException)3 Matchers.containsString (org.hamcrest.Matchers.containsString)3 StoreFactory (org.neo4j.kernel.impl.store.StoreFactory)3 UnableToUpgradeException (org.neo4j.kernel.impl.storemigration.StoreUpgrader.UnableToUpgradeException)3 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)2 FilenameFilter (java.io.FilenameFilter)1 ArrayList (java.util.ArrayList)1 Before (org.junit.Before)1