Search in sources :

Example 6 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class ApplyTransactionsCommand method applyTransactions.

private long applyTransactions(File fromPath, GraphDatabaseAPI toDb, long fromTxExclusive, long toTxInclusive, PrintStream out) throws IOException, TransactionFailureException {
    DependencyResolver resolver = toDb.getDependencyResolver();
    TransactionRepresentationCommitProcess commitProcess = new TransactionRepresentationCommitProcess(resolver.resolveDependency(TransactionAppender.class), resolver.resolveDependency(StorageEngine.class));
    LifeSupport life = new LifeSupport();
    try (DefaultFileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction();
        PageCache pageCache = StandalonePageCacheFactory.createPageCache(fileSystem)) {
        LogicalTransactionStore source = life.add(new ReadOnlyTransactionStore(pageCache, fileSystem, fromPath, new Monitors()));
        life.start();
        long lastAppliedTx = fromTxExclusive;
        // Some progress if there are more than a couple of transactions to apply
        ProgressListener progress = toTxInclusive - fromTxExclusive >= 100 ? textual(out).singlePart("Application progress", toTxInclusive - fromTxExclusive) : ProgressListener.NONE;
        try (IOCursor<CommittedTransactionRepresentation> cursor = source.getTransactions(fromTxExclusive + 1)) {
            while (cursor.next()) {
                CommittedTransactionRepresentation transaction = cursor.get();
                TransactionRepresentation transactionRepresentation = transaction.getTransactionRepresentation();
                try {
                    commitProcess.commit(new TransactionToApply(transactionRepresentation), NULL, EXTERNAL);
                    progress.add(1);
                } catch (final Throwable e) {
                    System.err.println("ERROR applying transaction " + transaction.getCommitEntry().getTxId());
                    throw e;
                }
                lastAppliedTx = transaction.getCommitEntry().getTxId();
                if (lastAppliedTx == toTxInclusive) {
                    break;
                }
            }
        }
        return lastAppliedTx;
    } finally {
        life.shutdown();
    }
}
Also used : TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) TransactionAppender(org.neo4j.kernel.impl.transaction.log.TransactionAppender) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) ReadOnlyTransactionStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionStore) DependencyResolver(org.neo4j.graphdb.DependencyResolver) ProgressListener(org.neo4j.helpers.progress.ProgressListener) TransactionRepresentationCommitProcess(org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess) Monitors(org.neo4j.kernel.monitoring.Monitors) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) PageCache(org.neo4j.io.pagecache.PageCache)

Example 7 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TxPullResponseDecoder method decode.

@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
    NetworkReadableClosableChannelNetty4 logChannel = new NetworkReadableClosableChannelNetty4(msg);
    StoreId storeId = StoreIdMarshal.INSTANCE.unmarshal(logChannel);
    LogEntryReader<NetworkReadableClosableChannelNetty4> reader = new VersionAwareLogEntryReader<>(new RecordStorageCommandReaderFactory());
    PhysicalTransactionCursor<NetworkReadableClosableChannelNetty4> transactionCursor = new PhysicalTransactionCursor<>(logChannel, reader);
    transactionCursor.next();
    CommittedTransactionRepresentation tx = transactionCursor.get();
    if (tx != null) {
        out.add(new TxPullResponse(storeId, tx));
    }
}
Also used : PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) RecordStorageCommandReaderFactory(org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageCommandReaderFactory) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) StoreId(org.neo4j.causalclustering.identity.StoreId) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) NetworkReadableClosableChannelNetty4(org.neo4j.causalclustering.messaging.NetworkReadableClosableChannelNetty4)

Example 8 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class LastCommittedIndexFinder method getLastCommittedIndex.

public long getLastCommittedIndex() {
    long lastCommittedIndex;
    long lastTxId = transactionIdStore.getLastCommittedTransactionId();
    byte[] lastHeaderFound = null;
    try (IOCursor<CommittedTransactionRepresentation> transactions = transactionStore.getTransactions(lastTxId)) {
        while (transactions.next()) {
            CommittedTransactionRepresentation committedTransactionRepresentation = transactions.get();
            lastHeaderFound = committedTransactionRepresentation.getStartEntry().getAdditionalHeader();
        }
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    if (lastHeaderFound == null) {
        throw new RuntimeException("We must have at least one transaction telling us where we are at in the consensus log.");
    }
    lastCommittedIndex = decodeLogIndexFromTxHeader(lastHeaderFound);
    log.info("Last committed index %d", lastCommittedIndex);
    return lastCommittedIndex;
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)

Example 9 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class BackupServiceIT method checkLastCommittedTxIdInLogAndNeoStore.

private void checkLastCommittedTxIdInLogAndNeoStore(long txId, long txIdFromOrigin) throws Exception {
    // Assert last committed transaction can be found in tx log and is the last tx in the log
    LifeSupport life = new LifeSupport();
    PageCache pageCache = pageCacheRule.getPageCache(fileSystem);
    LogicalTransactionStore transactionStore = life.add(new ReadOnlyTransactionStore(pageCache, fileSystem, backupDir, monitors));
    life.start();
    try (IOCursor<CommittedTransactionRepresentation> cursor = transactionStore.getTransactions(txId)) {
        assertTrue(cursor.next());
        assertEquals(txId, cursor.get().getCommitEntry().getTxId());
        assertFalse(cursor.next());
    } finally {
        life.shutdown();
    }
    // Assert last committed transaction is correct in neostore
    assertEquals(txId, txIdFromOrigin);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) PageCache(org.neo4j.io.pagecache.PageCache) ReadOnlyTransactionStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionStore)

Example 10 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TxPullResponseEncodeDecodeTest method newCommittedTransactionRepresentation.

private CommittedTransactionRepresentation newCommittedTransactionRepresentation() {
    final long arbitraryRecordId = 27L;
    Command.NodeCommand command = new Command.NodeCommand(new NodeRecord(arbitraryRecordId), new NodeRecord(arbitraryRecordId));
    PhysicalTransactionRepresentation physicalTransactionRepresentation = new PhysicalTransactionRepresentation(asList(new LogEntryCommand(command).getXaCommand()));
    physicalTransactionRepresentation.setHeader(new byte[] {}, 0, 0, 0, 0, 0, 0);
    LogEntryStart startEntry = new LogEntryStart(0, 0, 0L, 0L, new byte[] {}, LogPosition.UNSPECIFIED);
    OnePhaseCommit commitEntry = new OnePhaseCommit(42, 0);
    return new CommittedTransactionRepresentation(startEntry, physicalTransactionRepresentation, commitEntry);
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) Command(org.neo4j.kernel.impl.transaction.command.Command) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation)

Aggregations

CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)25 IOException (java.io.IOException)8 Test (org.junit.Test)7 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)7 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)6 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)6 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)6 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)6 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 Visitor (org.neo4j.helpers.collection.Visitor)4 TransactionToApply (org.neo4j.kernel.impl.api.TransactionToApply)4 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)4 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)4 TransactionCursor (org.neo4j.kernel.impl.transaction.log.TransactionCursor)4 OnePhaseCommit (org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit)4 Recovery (org.neo4j.kernel.recovery.Recovery)4 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)3 File (java.io.File)2