Search in sources :

Example 11 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TransactionLogCatchUpWriterTest method verifyTransactionsInLog.

private void verifyTransactionsInLog(long fromTxId, long endTxId) throws IOException {
    long expectedTxId = fromTxId;
    PhysicalLogFiles logFiles = new PhysicalLogFiles(storeDir, fs);
    LogVersionedStoreChannel versionedStoreChannel = PhysicalLogFile.openForVersion(logFiles, fs, 0, false);
    try (ReadableLogChannel channel = new ReadAheadLogChannel(versionedStoreChannel, LogVersionBridge.NO_MORE_CHANNELS, 1024)) {
        try (PhysicalTransactionCursor<ReadableLogChannel> txCursor = new PhysicalTransactionCursor<>(channel, new VersionAwareLogEntryReader<>())) {
            while (txCursor.next()) {
                CommittedTransactionRepresentation tx = txCursor.get();
                long txId = tx.getCommitEntry().getTxId();
                assertThat(expectedTxId, lessThanOrEqualTo(endTxId));
                assertEquals(expectedTxId, txId);
                expectedTxId++;
            }
        }
    }
}
Also used : PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)

Example 12 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class BatchingTxApplierTest method createTxWithId.

private CommittedTransactionRepresentation createTxWithId(long txId) {
    CommittedTransactionRepresentation tx = mock(CommittedTransactionRepresentation.class);
    LogEntryCommit commitEntry = mock(LogEntryCommit.class);
    when(commitEntry.getTxId()).thenReturn(txId);
    TransactionRepresentation txRep = mock(TransactionRepresentation.class);
    when(tx.getTransactionRepresentation()).thenReturn(txRep);
    when(tx.getCommitEntry()).thenReturn(commitEntry);
    return tx;
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)

Example 13 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class RebuildFromLogs method findLastTransactionId.

private long findLastTransactionId(PhysicalLogFiles logFiles, long highestVersion) throws IOException {
    ReadableLogChannel logChannel = new ReadAheadLogChannel(PhysicalLogFile.openForVersion(logFiles, fs, highestVersion, false), NO_MORE_CHANNELS);
    long lastTransactionId = -1;
    LogEntryReader<ReadableClosablePositionAwareChannel> entryReader = new VersionAwareLogEntryReader<>();
    try (IOCursor<CommittedTransactionRepresentation> cursor = new PhysicalTransactionCursor<>(logChannel, entryReader)) {
        while (cursor.next()) {
            lastTransactionId = cursor.get().getCommitEntry().getTxId();
        }
    }
    return lastTransactionId;
}
Also used : PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)

Example 14 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class RemoteStore method getPullIndex.

/**
     * Later stages of the startup process require at least one transaction to
     * figure out the mapping between the transaction log and the consensus log.
     *
     * If there are no transaction logs then we can pull from and including
     * the index which the metadata store points to. This would be the case
     * for example with a backup taken during an idle period of the system.
     *
     * However, if there are transaction logs then we want to find out where
     * they end and pull from there, excluding the last one so that we do not
     * get duplicate entries.
     */
private long getPullIndex(File storeDir) throws IOException {
    /* this is the metadata store */
    ReadOnlyTransactionIdStore txIdStore = new ReadOnlyTransactionIdStore(pageCache, storeDir);
    /* Clean as in clean shutdown. Without transaction logs this should be the truth,
        * but otherwise it can be used as a starting point for scanning the logs. */
    long lastCleanTxId = txIdStore.getLastCommittedTransactionId();
    /* these are the transaction logs */
    ReadOnlyTransactionStore txStore = new ReadOnlyTransactionStore(pageCache, fs, storeDir, new Monitors());
    long lastTxId = BASE_TX_ID;
    try (Lifespan ignored = new Lifespan(txStore)) {
        TransactionCursor cursor;
        try {
            cursor = txStore.getTransactions(lastCleanTxId);
        } catch (NoSuchTransactionException e) {
            log.info("No transaction logs found. Will use metadata store as base for pull request.");
            return lastCleanTxId;
        }
        while (cursor.next()) {
            CommittedTransactionRepresentation tx = cursor.get();
            lastTxId = tx.getCommitEntry().getTxId();
        }
        if (lastTxId < lastCleanTxId) {
            throw new IllegalStateException("Metadata index was higher than transaction log index.");
        }
        // we don't want to pull a transaction we already have in the log, hence +1
        return lastTxId + 1;
    }
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) ReadOnlyTransactionIdStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionIdStore) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) Monitors(org.neo4j.kernel.monitoring.Monitors) NoSuchTransactionException(org.neo4j.kernel.impl.transaction.log.NoSuchTransactionException) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) ReadOnlyTransactionStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionStore)

Example 15 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TransactionLogCatchUpWriter method onTxReceived.

@Override
public synchronized void onTxReceived(TxPullResponse txPullResponse) {
    CommittedTransactionRepresentation tx = txPullResponse.tx();
    long receivedTxId = tx.getCommitEntry().getTxId();
    if (receivedTxId != expectedTxId) {
        throw new RuntimeException(format("Expected txId: %d but got: %d", expectedTxId, receivedTxId));
    }
    lastTxId = receivedTxId;
    expectedTxId++;
    try {
        writer.append(tx.getTransactionRepresentation(), lastTxId);
    } catch (IOException e) {
        log.error("Failed when appending to transaction log", e);
    }
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) IOException(java.io.IOException)

Aggregations

CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)25 IOException (java.io.IOException)8 Test (org.junit.Test)7 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)7 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)6 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)6 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)6 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)6 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 Visitor (org.neo4j.helpers.collection.Visitor)4 TransactionToApply (org.neo4j.kernel.impl.api.TransactionToApply)4 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)4 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)4 TransactionCursor (org.neo4j.kernel.impl.transaction.log.TransactionCursor)4 OnePhaseCommit (org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit)4 Recovery (org.neo4j.kernel.recovery.Recovery)4 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)3 File (java.io.File)2