Search in sources :

Example 21 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class CheckTxLogsTest method shouldDetectAnInconsistentCheckPointPointingToALogFileGreaterThanMaxLogVersion.

@Test
public void shouldDetectAnInconsistentCheckPointPointingToALogFileGreaterThanMaxLogVersion() throws Exception {
    // given
    File log = logFile(1);
    writeCheckPoint(log, 2, LOG_HEADER_SIZE);
    CapturingInconsistenciesHandler handler = new CapturingInconsistenciesHandler();
    CheckTxLogs checker = new CheckTxLogs(System.out, fsRule.get());
    // when
    checker.validateCheckPoints(new PhysicalLogFiles(storeDirectory, fsRule.get()), handler);
    // then
    assertEquals(1, handler.checkPointInconsistencies.size());
    assertEquals(1, handler.checkPointInconsistencies.get(0).logVersion);
    assertEquals(new LogPosition(2, LOG_HEADER_SIZE), handler.checkPointInconsistencies.get(0).logPosition);
    assertThat(handler.checkPointInconsistencies.get(0).size, lessThan(0L));
}
Also used : File(java.io.File) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.Test)

Example 22 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class RaftLogMetadataCacheTest method shouldClearTheCache.

@Test
public void shouldClearTheCache() {
    // given
    final RaftLogMetadataCache cache = new RaftLogMetadataCache(2);
    final long index = 12L;
    final long term = 12L;
    final LogPosition position = new LogPosition(3, 4);
    // when
    cache.cacheMetadata(index, term, position);
    cache.clear();
    RaftLogMetadataCache.RaftLogEntryMetadata metadata = cache.getMetadata(index);
    // then
    assertNull(metadata);
}
Also used : LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.Test)

Example 23 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class RaftLogMetadataCacheTest method shouldReturnTheTxValueTIfInTheCached.

@Test
public void shouldReturnTheTxValueTIfInTheCached() {
    // given
    final RaftLogMetadataCache cache = new RaftLogMetadataCache(2);
    final long index = 12L;
    final long term = 12L;
    final LogPosition position = new LogPosition(3, 4);
    // when
    cache.cacheMetadata(index, term, position);
    final RaftLogMetadataCache.RaftLogEntryMetadata metadata = cache.getMetadata(index);
    // then
    assertEquals(new RaftLogMetadataCache.RaftLogEntryMetadata(term, position), metadata);
}
Also used : LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.Test)

Example 24 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class StoreMigrator method updateOrAddNeoStoreFieldsAsPartOfMigration.

private void updateOrAddNeoStoreFieldsAsPartOfMigration(File migrationDir, File storeDir, String versionToMigrateTo) throws IOException {
    final File storeDirNeoStore = new File(storeDir, MetaDataStore.DEFAULT_NAME);
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.UPGRADE_TRANSACTION_ID, MetaDataStore.getRecord(pageCache, storeDirNeoStore, Position.LAST_TRANSACTION_ID));
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.UPGRADE_TIME, System.currentTimeMillis());
    // Store the checksum of the transaction id the upgrade is at right now. Store it both as
    // LAST_TRANSACTION_CHECKSUM and UPGRADE_TRANSACTION_CHECKSUM. Initially the last transaction and the
    // upgrade transaction will be the same, but imagine this scenario:
    //  - legacy store is migrated on instance A at transaction T
    //  - upgraded store is copied, via backup or HA or whatever to instance B
    //  - instance A performs a transaction
    //  - instance B would like to communicate with A where B's last transaction checksum
    //    is verified on A. A, at this point not having logs from pre-migration era, will need to
    //    know the checksum of transaction T to accommodate for this request from B. A will be able
    //    to look up checksums for transactions succeeding T by looking at its transaction logs,
    //    but T needs to be stored in neostore to be accessible. Obvioously this scenario is only
    //    problematic as long as we don't migrate and translate old logs.
    TransactionId lastTxInfo = readLastTxInformation(migrationDir);
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.LAST_TRANSACTION_CHECKSUM, lastTxInfo.checksum());
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.UPGRADE_TRANSACTION_CHECKSUM, lastTxInfo.checksum());
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.LAST_TRANSACTION_COMMIT_TIMESTAMP, lastTxInfo.commitTimestamp());
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.UPGRADE_TRANSACTION_COMMIT_TIMESTAMP, lastTxInfo.commitTimestamp());
    // add LAST_CLOSED_TRANSACTION_LOG_VERSION and LAST_CLOSED_TRANSACTION_LOG_BYTE_OFFSET to the migrated
    // NeoStore
    LogPosition logPosition = readLastTxLogPosition(migrationDir);
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.LAST_CLOSED_TRANSACTION_LOG_VERSION, logPosition.getLogVersion());
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.LAST_CLOSED_TRANSACTION_LOG_BYTE_OFFSET, logPosition.getByteOffset());
    // Upgrade version in NeoStore
    MetaDataStore.setRecord(pageCache, storeDirNeoStore, Position.STORE_VERSION, MetaDataStore.versionStringToLong(versionToMigrateTo));
}
Also used : PagedFile(org.neo4j.io.pagecache.PagedFile) StoreFile(org.neo4j.kernel.impl.storemigration.StoreFile) File(java.io.File) TransactionId(org.neo4j.kernel.impl.store.TransactionId) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 25 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class StoreMigrator method extractTransactionLogPosition.

private LogPosition extractTransactionLogPosition(File neoStore, File storeDir, long lastTxId) throws IOException {
    long lastClosedTxLogVersion = MetaDataStore.getRecord(pageCache, neoStore, Position.LAST_CLOSED_TRANSACTION_LOG_VERSION);
    long lastClosedTxLogByteOffset = MetaDataStore.getRecord(pageCache, neoStore, Position.LAST_CLOSED_TRANSACTION_LOG_BYTE_OFFSET);
    if (lastClosedTxLogVersion != MetaDataRecordFormat.FIELD_NOT_PRESENT && lastClosedTxLogByteOffset != MetaDataRecordFormat.FIELD_NOT_PRESENT) {
        return new LogPosition(lastClosedTxLogVersion, lastClosedTxLogByteOffset);
    }
    // The legacy store we're migrating doesn't have this record in neostore so try to extract it from tx log
    if (lastTxId == TransactionIdStore.BASE_TX_ID) {
        return new LogPosition(BASE_TX_LOG_VERSION, BASE_TX_LOG_BYTE_OFFSET);
    }
    PhysicalLogFiles logFiles = new PhysicalLogFiles(storeDir, fileSystem);
    long logVersion = logFiles.getHighestLogVersion();
    if (logVersion == -1) {
        return new LogPosition(BASE_TX_LOG_VERSION, BASE_TX_LOG_BYTE_OFFSET);
    }
    long offset = fileSystem.getFileSize(logFiles.getLogFileForVersion(logVersion));
    return new LogPosition(logVersion, offset);
}
Also used : LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)

Aggregations

LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)42 Test (org.junit.Test)26 File (java.io.File)11 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)9 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)9 IOException (java.io.IOException)6 LogPositionMarker (org.neo4j.kernel.impl.transaction.log.LogPositionMarker)6 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)5 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)5 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)5 InMemoryClosableChannel (org.neo4j.kernel.impl.transaction.log.InMemoryClosableChannel)4 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)4 CheckPoint (org.neo4j.kernel.impl.transaction.log.entry.CheckPoint)4 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)4 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)4 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)4 LatestCheckPoint (org.neo4j.kernel.recovery.LatestCheckPointFinder.LatestCheckPoint)4 LogFile (org.neo4j.kernel.impl.transaction.log.LogFile)3 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)3 HashMap (java.util.HashMap)2