Search in sources :

Example 1 with IdGeneratorMigrator

use of org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator in project neo4j by neo4j.

the class StoreUpgraderInterruptionTestIT method shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMigration.

@Test
public void shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMigration() throws IOException, ConsistencyCheckIncompleteException {
    MigrationTestUtils.prepareSampleLegacyDatabase(version, fs, workingDatabaseLayout.databaseDirectory(), prepareDirectory);
    RecordStoreVersionCheck versionCheck = new RecordStoreVersionCheck(fs, pageCache, workingDatabaseLayout, NullLogProvider.getInstance(), Config.defaults(), NULL);
    MigrationProgressMonitor progressMonitor = MigrationProgressMonitor.SILENT;
    LogService logService = NullLogService.getInstance();
    RecordStorageMigrator failingStoreMigrator = new RecordStorageMigrator(fs, pageCache, CONFIG, logService, jobScheduler, NULL, batchImporterFactory, INSTANCE) {

        @Override
        public void migrate(DatabaseLayout directoryLayout, DatabaseLayout migrationLayout, ProgressReporter progressReporter, String versionToMigrateFrom, String versionToMigrateTo, IndexImporterFactory indexImporterFactory) throws IOException, KernelException {
            super.migrate(directoryLayout, migrationLayout, progressReporter, versionToMigrateFrom, versionToMigrateTo, indexImporterFactory);
            throw new RuntimeException("This upgrade is failing");
        }
    };
    try {
        newUpgrader(versionCheck, progressMonitor, createIndexMigrator(), failingStoreMigrator).migrateIfNeeded(workingDatabaseLayout, false);
        fail("Should throw exception");
    } catch (RuntimeException e) {
        assertEquals("This upgrade is failing", e.getMessage());
    }
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, baselineFormat));
    RecordStorageMigrator migrator = new RecordStorageMigrator(fs, pageCache, CONFIG, logService, jobScheduler, NULL, batchImporterFactory, INSTANCE);
    IdGeneratorMigrator idMigrator = new IdGeneratorMigrator(fs, pageCache, CONFIG, NULL);
    SchemaIndexMigrator indexMigrator = createIndexMigrator();
    newUpgrader(versionCheck, progressMonitor, indexMigrator, migrator, idMigrator).migrateIfNeeded(workingDatabaseLayout, false);
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, successorFormat));
    // Since consistency checker is in read only mode we need to start/stop db to generate label scan store.
    startStopDatabase(neo4jLayout.homeDirectory());
    assertConsistentStore(workingDatabaseLayout);
}
Also used : MigrationProgressMonitor(org.neo4j.storageengine.migration.MigrationProgressMonitor) RecordStoreVersionCheck(org.neo4j.kernel.impl.storemigration.RecordStoreVersionCheck) RecordStorageMigrator(org.neo4j.kernel.impl.storemigration.RecordStorageMigrator) DatabaseLayout(org.neo4j.io.layout.DatabaseLayout) IndexImporterFactory(org.neo4j.internal.batchimport.IndexImporterFactory) ProgressReporter(org.neo4j.common.ProgressReporter) IdGeneratorMigrator(org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator) SchemaIndexMigrator(org.neo4j.storageengine.migration.SchemaIndexMigrator) NullLogService(org.neo4j.logging.internal.NullLogService) LogService(org.neo4j.logging.internal.LogService) Test(org.junit.Test)

Example 2 with IdGeneratorMigrator

use of org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator in project neo4j by neo4j.

the class RecordStorageEngineFactory method migrationParticipants.

@Override
public List<StoreMigrationParticipant> migrationParticipants(FileSystemAbstraction fs, Config config, PageCache pageCache, JobScheduler jobScheduler, LogService logService, PageCacheTracer cacheTracer, MemoryTracker memoryTracker) {
    BatchImporterFactory batchImporterFactory = BatchImporterFactory.withHighestPriority();
    RecordStorageMigrator recordStorageMigrator = new RecordStorageMigrator(fs, pageCache, config, logService, jobScheduler, cacheTracer, batchImporterFactory, memoryTracker);
    IdGeneratorMigrator idGeneratorMigrator = new IdGeneratorMigrator(fs, pageCache, config, cacheTracer);
    return List.of(recordStorageMigrator, idGeneratorMigrator);
}
Also used : RecordStorageMigrator(org.neo4j.kernel.impl.storemigration.RecordStorageMigrator) BatchImporterFactory(org.neo4j.internal.batchimport.BatchImporterFactory) IdGeneratorMigrator(org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator)

Example 3 with IdGeneratorMigrator

use of org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator in project neo4j by neo4j.

the class StoreUpgraderInterruptionTestIT method shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMovingFiles.

@Test
public void shouldSucceedWithUpgradeAfterPreviousAttemptDiedDuringMovingFiles() throws IOException, ConsistencyCheckIncompleteException {
    MigrationTestUtils.prepareSampleLegacyDatabase(version, fs, workingDatabaseLayout.databaseDirectory(), prepareDirectory);
    RecordStoreVersionCheck versionCheck = new RecordStoreVersionCheck(fs, pageCache, workingDatabaseLayout, NullLogProvider.getInstance(), Config.defaults(), NULL);
    MigrationProgressMonitor progressMonitor = MigrationProgressMonitor.SILENT;
    LogService logService = NullLogService.getInstance();
    RecordStorageMigrator failingStoreMigrator = new RecordStorageMigrator(fs, pageCache, CONFIG, logService, jobScheduler, NULL, batchImporterFactory, INSTANCE) {

        @Override
        public void moveMigratedFiles(DatabaseLayout migrationLayout, DatabaseLayout directoryLayout, String versionToUpgradeFrom, String versionToMigrateTo) throws IOException {
            super.moveMigratedFiles(migrationLayout, directoryLayout, versionToUpgradeFrom, versionToMigrateTo);
            throw new RuntimeException("This upgrade is failing");
        }
    };
    IdGeneratorMigrator idMigrator = new IdGeneratorMigrator(fs, pageCache, CONFIG, NULL);
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, baselineFormat));
    try {
        newUpgrader(versionCheck, progressMonitor, createIndexMigrator(), failingStoreMigrator, idMigrator).migrateIfNeeded(workingDatabaseLayout, false);
        fail("Should throw exception");
    } catch (RuntimeException e) {
        assertEquals("This upgrade is failing", e.getMessage());
    }
    RecordStorageMigrator migrator = new RecordStorageMigrator(fs, pageCache, CONFIG, logService, jobScheduler, NULL, batchImporterFactory, INSTANCE);
    newUpgrader(versionCheck, progressMonitor, createIndexMigrator(), migrator, idMigrator).migrateIfNeeded(workingDatabaseLayout, false);
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, successorFormat));
    pageCache.close();
    // Since consistency checker is in read only mode we need to start/stop db to generate label scan store.
    startStopDatabase(neo4jLayout.homeDirectory());
    assertConsistentStore(workingDatabaseLayout);
}
Also used : MigrationProgressMonitor(org.neo4j.storageengine.migration.MigrationProgressMonitor) RecordStoreVersionCheck(org.neo4j.kernel.impl.storemigration.RecordStoreVersionCheck) RecordStorageMigrator(org.neo4j.kernel.impl.storemigration.RecordStorageMigrator) DatabaseLayout(org.neo4j.io.layout.DatabaseLayout) IdGeneratorMigrator(org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator) NullLogService(org.neo4j.logging.internal.NullLogService) LogService(org.neo4j.logging.internal.LogService) Test(org.junit.Test)

Example 4 with IdGeneratorMigrator

use of org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator in project neo4j by neo4j.

the class StoreUpgraderInterruptionTestIT method tracePageCacheAccessOnIdStoreUpgrade.

@Test
public void tracePageCacheAccessOnIdStoreUpgrade() throws IOException, ConsistencyCheckIncompleteException {
    MigrationTestUtils.prepareSampleLegacyDatabase(version, fs, workingDatabaseLayout.databaseDirectory(), prepareDirectory);
    RecordStoreVersionCheck versionCheck = new RecordStoreVersionCheck(fs, pageCache, workingDatabaseLayout, NullLogProvider.getInstance(), Config.defaults(), NULL);
    MigrationProgressMonitor progressMonitor = MigrationProgressMonitor.SILENT;
    LogService logService = NullLogService.getInstance();
    var idMigratorTracer = new DefaultPageCacheTracer();
    var recordMigratorTracer = new DefaultPageCacheTracer();
    IdGeneratorMigrator idMigrator = new IdGeneratorMigrator(fs, pageCache, CONFIG, idMigratorTracer);
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, baselineFormat));
    var migrator = new RecordStorageMigrator(fs, pageCache, CONFIG, logService, jobScheduler, recordMigratorTracer, batchImporterFactory, INSTANCE);
    newUpgrader(versionCheck, progressMonitor, createIndexMigrator(), migrator, idMigrator).migrateIfNeeded(workingDatabaseLayout, false);
    assertTrue(checkNeoStoreHasFormatVersion(versionCheck, successorFormat));
    startStopDatabase(neo4jLayout.homeDirectory());
    assertConsistentStore(workingDatabaseLayout);
    assertEquals(43, idMigratorTracer.faults());
    assertEquals(83, idMigratorTracer.hits());
    assertEquals(126, idMigratorTracer.pins());
    assertEquals(126, idMigratorTracer.unpins());
    assertEquals(61, recordMigratorTracer.faults());
    assertEquals(228, recordMigratorTracer.hits());
    assertEquals(289, recordMigratorTracer.pins());
    assertEquals(289, recordMigratorTracer.unpins());
}
Also used : MigrationProgressMonitor(org.neo4j.storageengine.migration.MigrationProgressMonitor) RecordStoreVersionCheck(org.neo4j.kernel.impl.storemigration.RecordStoreVersionCheck) RecordStorageMigrator(org.neo4j.kernel.impl.storemigration.RecordStorageMigrator) IdGeneratorMigrator(org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator) NullLogService(org.neo4j.logging.internal.NullLogService) LogService(org.neo4j.logging.internal.LogService) DefaultPageCacheTracer(org.neo4j.io.pagecache.tracing.DefaultPageCacheTracer) Test(org.junit.Test)

Aggregations

IdGeneratorMigrator (org.neo4j.kernel.impl.storemigration.IdGeneratorMigrator)4 RecordStorageMigrator (org.neo4j.kernel.impl.storemigration.RecordStorageMigrator)4 Test (org.junit.Test)3 RecordStoreVersionCheck (org.neo4j.kernel.impl.storemigration.RecordStoreVersionCheck)3 LogService (org.neo4j.logging.internal.LogService)3 NullLogService (org.neo4j.logging.internal.NullLogService)3 MigrationProgressMonitor (org.neo4j.storageengine.migration.MigrationProgressMonitor)3 DatabaseLayout (org.neo4j.io.layout.DatabaseLayout)2 ProgressReporter (org.neo4j.common.ProgressReporter)1 BatchImporterFactory (org.neo4j.internal.batchimport.BatchImporterFactory)1 IndexImporterFactory (org.neo4j.internal.batchimport.IndexImporterFactory)1 DefaultPageCacheTracer (org.neo4j.io.pagecache.tracing.DefaultPageCacheTracer)1 SchemaIndexMigrator (org.neo4j.storageengine.migration.SchemaIndexMigrator)1