Search in sources :

Example 16 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TxPullRequestHandler method channelRead0.

@Override
protected void channelRead0(ChannelHandlerContext ctx, final TxPullRequest msg) throws Exception {
    long firstTxId = Math.max(msg.previousTxId(), BASE_TX_ID) + 1;
    long lastTxId = firstTxId;
    CatchupResult status = SUCCESS_END_OF_STREAM;
    StoreId localStoreId = storeIdSupplier.get();
    long lastCommittedTransactionId = transactionIdStore.getLastCommittedTransactionId();
    if (localStoreId == null || !localStoreId.equals(msg.expectedStoreId())) {
        status = E_STORE_ID_MISMATCH;
        log.info("Failed to serve TxPullRequest for tx %d and storeId %s because that storeId is different " + "from this machine with %s", lastTxId, msg.expectedStoreId(), localStoreId);
    } else if (!databaseAvailable.getAsBoolean()) {
        // database is not available for pulling transactions...
        status = E_STORE_UNAVAILABLE;
        log.info("Failed to serve TxPullRequest for tx %d because the local database is unavailable.", lastTxId);
    } else if (lastCommittedTransactionId >= firstTxId) {
        try (IOCursor<CommittedTransactionRepresentation> cursor = logicalTransactionStore.getTransactions(firstTxId)) {
            status = SUCCESS_END_OF_BATCH;
            for (int i = 0; i < batchSize; i++) {
                if (cursor.next()) {
                    ctx.write(ResponseMessageType.TX);
                    CommittedTransactionRepresentation tx = cursor.get();
                    lastTxId = tx.getCommitEntry().getTxId();
                    ctx.write(new TxPullResponse(localStoreId, tx));
                } else {
                    status = SUCCESS_END_OF_STREAM;
                    break;
                }
            }
            ctx.flush();
        } catch (NoSuchTransactionException e) {
            status = E_TRANSACTION_PRUNED;
            log.info("Failed to serve TxPullRequest for tx %d because the transaction does not exist.", lastTxId);
        }
    }
    ctx.write(ResponseMessageType.TX_STREAM_FINISHED);
    TxStreamFinishedResponse response = new TxStreamFinishedResponse(status, lastCommittedTransactionId);
    ctx.write(response);
    ctx.flush();
    monitor.increment();
    protocol.expect(State.MESSAGE_TYPE);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) StoreId(org.neo4j.causalclustering.identity.StoreId) CatchupResult(org.neo4j.causalclustering.catchup.CatchupResult) NoSuchTransactionException(org.neo4j.kernel.impl.transaction.log.NoSuchTransactionException)

Example 17 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class ResponsePackerTest method shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving.

@Test
public void shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving() throws Exception {
    // GIVEN
    LogicalTransactionStore transactionStore = mock(LogicalTransactionStore.class);
    long lastAppliedTransactionId = 5L;
    TransactionCursor endlessCursor = new EndlessCursor(lastAppliedTransactionId + 1);
    when(transactionStore.getTransactions(anyLong())).thenReturn(endlessCursor);
    final long targetTransactionId = 8L;
    final TransactionIdStore transactionIdStore = new DeadSimpleTransactionIdStore(targetTransactionId, 0, BASE_TX_COMMIT_TIMESTAMP, 0, 0);
    ResponsePacker packer = new ResponsePacker(transactionStore, transactionIdStore, Suppliers.singleton(StoreIdTestFactory.newStoreIdForCurrentVersion()));
    // WHEN
    Response<Object> response = packer.packTransactionStreamResponse(requestContextStartingAt(5L), null);
    final AtomicLong nextExpectedVisit = new AtomicLong(lastAppliedTransactionId);
    response.accept(new Response.Handler() {

        @Override
        public void obligation(long txId) throws IOException {
            fail("Should not be called");
        }

        @Override
        public Visitor<CommittedTransactionRepresentation, Exception> transactions() {
            return new Visitor<CommittedTransactionRepresentation, Exception>() {

                @Override
                public boolean visit(CommittedTransactionRepresentation element) {
                    // THEN
                    long txId = element.getCommitEntry().getTxId();
                    assertThat(txId, lessThanOrEqualTo(targetTransactionId));
                    assertEquals(nextExpectedVisit.incrementAndGet(), txId);
                    // Move the target transaction id forward one step, effectively always keeping it out of reach
                    transactionIdStore.setLastCommittedAndClosedTransactionId(transactionIdStore.getLastCommittedTransactionId() + 1, 0, BASE_TX_COMMIT_TIMESTAMP, 3, 4);
                    return true;
                }
            };
        }
    });
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) Visitor(org.neo4j.helpers.collection.Visitor) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) IOException(java.io.IOException) IOException(java.io.IOException) Response(org.neo4j.com.Response) AtomicLong(java.util.concurrent.atomic.AtomicLong) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) Test(org.junit.Test)

Example 18 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class MadeUpServerImplementation method streamBackTransactions.

@Override
public Response<Integer> streamBackTransactions(int responseToSendBack, final int txCount) {
    TransactionStream transactions = new TransactionStream() {

        @Override
        public void accept(Visitor<CommittedTransactionRepresentation, Exception> visitor) throws Exception {
            for (int i = 1; i <= txCount; i++) {
                CommittedTransactionRepresentation transaction = createTransaction(TransactionIdStore.BASE_TX_ID + i);
                visitor.visit(transaction);
            }
        }
    };
    return new TransactionStreamResponse<>(responseToSendBack, storeIdToRespondWith, transactions, ResourceReleaser.NO_OP);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) Visitor(org.neo4j.helpers.collection.Visitor)

Example 19 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class DefaultRecoverySPI method startRecovery.

@Override
public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
    // Calling this method means that recovery is required, tell storage engine about it
    // This method will be called before recovery actually starts and so will ensure that
    // each store is aware that recovery will be performed. At this point all the stores have
    // already started btw.
    // Go and read more at {@link CommonAbstractStore#deleteIdGenerator()}
    storageEngine.prepareForRecoveryRequired();
    transactionsToApply = new TransactionQueue(10_000, (first, last) -> storageEngine.apply(first, RECOVERY));
    recoveryVisitor = new RecoveryVisitor(transactionsToApply);
    return recoveryVisitor;
}
Also used : TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) IOException(java.io.IOException) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) IOLimiter(org.neo4j.io.pagecache.IOLimiter) TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) RECOVERY(org.neo4j.storageengine.api.TransactionApplicationMode.RECOVERY) Visitor(org.neo4j.helpers.collection.Visitor) NO_COMMITMENT(org.neo4j.kernel.impl.transaction.log.Commitment.NO_COMMITMENT) TransactionQueue(org.neo4j.kernel.impl.api.TransactionQueue) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) TransactionQueue(org.neo4j.kernel.impl.api.TransactionQueue)

Example 20 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class Recovery method init.

@Override
public void init() throws Throwable {
    LogPosition recoveryFromPosition = spi.getPositionToRecoverFrom();
    if (LogPosition.UNSPECIFIED.equals(recoveryFromPosition)) {
        return;
    }
    monitor.recoveryRequired(recoveryFromPosition);
    LogPosition recoveryToPosition;
    CommittedTransactionRepresentation lastTransaction = null;
    Visitor<CommittedTransactionRepresentation, Exception> recoveryVisitor = spi.startRecovery();
    try (TransactionCursor transactionsToRecover = spi.getTransactions(recoveryFromPosition)) {
        while (transactionsToRecover.next()) {
            lastTransaction = transactionsToRecover.get();
            long txId = lastTransaction.getCommitEntry().getTxId();
            recoveryVisitor.visit(lastTransaction);
            monitor.transactionRecovered(txId);
            numberOfRecoveredTransactions++;
        }
        recoveryToPosition = transactionsToRecover.position();
    }
    if (recoveryToPosition.equals(LogPosition.UNSPECIFIED)) {
        recoveryToPosition = recoveryFromPosition;
    }
    spi.allTransactionsRecovered(lastTransaction, recoveryToPosition);
    recoveredLog = true;
    spi.forceEverything();
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) IOException(java.io.IOException) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Aggregations

CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)25 IOException (java.io.IOException)8 Test (org.junit.Test)7 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)7 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)6 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)6 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)6 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)6 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 Visitor (org.neo4j.helpers.collection.Visitor)4 TransactionToApply (org.neo4j.kernel.impl.api.TransactionToApply)4 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)4 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)4 TransactionCursor (org.neo4j.kernel.impl.transaction.log.TransactionCursor)4 OnePhaseCommit (org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit)4 Recovery (org.neo4j.kernel.recovery.Recovery)4 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)3 File (java.io.File)2