Search in sources :

Example 31 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TransactionLogCatchUpWriter method onTxReceived.

@Override
public synchronized void onTxReceived(TxPullResponse txPullResponse) {
    CommittedTransactionRepresentation tx = txPullResponse.tx();
    long receivedTxId = tx.getCommitEntry().getTxId();
    if (receivedTxId != expectedTxId) {
        throw new RuntimeException(format("Expected txId: %d but got: %d", expectedTxId, receivedTxId));
    }
    lastTxId = receivedTxId;
    expectedTxId++;
    try {
        writer.append(tx.getTransactionRepresentation(), lastTxId);
    } catch (IOException e) {
        log.error("Failed when appending to transaction log", e);
    }
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) IOException(java.io.IOException)

Example 32 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TxPullRequestHandler method channelRead0.

@Override
protected void channelRead0(ChannelHandlerContext ctx, final TxPullRequest msg) throws Exception {
    long firstTxId = Math.max(msg.previousTxId(), BASE_TX_ID) + 1;
    long lastTxId = firstTxId;
    CatchupResult status = SUCCESS_END_OF_STREAM;
    StoreId localStoreId = storeIdSupplier.get();
    long lastCommittedTransactionId = transactionIdStore.getLastCommittedTransactionId();
    if (localStoreId == null || !localStoreId.equals(msg.expectedStoreId())) {
        status = E_STORE_ID_MISMATCH;
        log.info("Failed to serve TxPullRequest for tx %d and storeId %s because that storeId is different " + "from this machine with %s", lastTxId, msg.expectedStoreId(), localStoreId);
    } else if (!databaseAvailable.getAsBoolean()) {
        // database is not available for pulling transactions...
        status = E_STORE_UNAVAILABLE;
        log.info("Failed to serve TxPullRequest for tx %d because the local database is unavailable.", lastTxId);
    } else if (lastCommittedTransactionId >= firstTxId) {
        try (IOCursor<CommittedTransactionRepresentation> cursor = logicalTransactionStore.getTransactions(firstTxId)) {
            status = SUCCESS_END_OF_BATCH;
            for (int i = 0; i < batchSize; i++) {
                if (cursor.next()) {
                    ctx.write(ResponseMessageType.TX);
                    CommittedTransactionRepresentation tx = cursor.get();
                    lastTxId = tx.getCommitEntry().getTxId();
                    ctx.write(new TxPullResponse(localStoreId, tx));
                } else {
                    status = SUCCESS_END_OF_STREAM;
                    break;
                }
            }
            ctx.flush();
        } catch (NoSuchTransactionException e) {
            status = E_TRANSACTION_PRUNED;
            log.info("Failed to serve TxPullRequest for tx %d because the transaction does not exist.", lastTxId);
        }
    }
    ctx.write(ResponseMessageType.TX_STREAM_FINISHED);
    TxStreamFinishedResponse response = new TxStreamFinishedResponse(status, lastCommittedTransactionId);
    ctx.write(response);
    ctx.flush();
    monitor.increment();
    protocol.expect(State.MESSAGE_TYPE);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) StoreId(org.neo4j.causalclustering.identity.StoreId) CatchupResult(org.neo4j.causalclustering.catchup.CatchupResult) NoSuchTransactionException(org.neo4j.kernel.impl.transaction.log.NoSuchTransactionException)

Example 33 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class TransactionRecordStateTest method shouldDeleteDynamicLabelsForDeletedNodeForRecoveredTransaction.

@Test
public void shouldDeleteDynamicLabelsForDeletedNodeForRecoveredTransaction() throws Throwable {
    // GIVEN a store that has got a node with a dynamic label record
    NeoStores store = neoStoresRule.open();
    BatchTransactionApplier applier = new NeoStoreBatchTransactionApplier(store, mock(CacheAccessBackDoor.class), LockService.NO_LOCK_SERVICE);
    AtomicLong nodeId = new AtomicLong();
    AtomicLong dynamicLabelRecordId = new AtomicLong();
    apply(applier, transaction(nodeWithDynamicLabelRecord(store, nodeId, dynamicLabelRecordId)));
    assertDynamicLabelRecordInUse(store, dynamicLabelRecordId.get(), true);
    // WHEN applying a transaction, which has first round-tripped through a log (written then read)
    TransactionRepresentation transaction = transaction(deleteNode(store, nodeId.get()));
    InMemoryVersionableReadableClosablePositionAwareChannel channel = new InMemoryVersionableReadableClosablePositionAwareChannel();
    writeToChannel(transaction, channel);
    CommittedTransactionRepresentation recoveredTransaction = readFromChannel(channel);
    // and applying that recovered transaction
    apply(applier, recoveredTransaction.getTransactionRepresentation());
    // THEN should have the dynamic label record should be deleted as well
    assertDynamicLabelRecordInUse(store, dynamicLabelRecordId.get(), false);
}
Also used : AtomicLong(java.util.concurrent.atomic.AtomicLong) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) NeoStores(org.neo4j.kernel.impl.store.NeoStores) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) NeoStoreBatchTransactionApplier(org.neo4j.kernel.impl.transaction.command.NeoStoreBatchTransactionApplier) InMemoryVersionableReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.InMemoryVersionableReadableClosablePositionAwareChannel) NeoStoreBatchTransactionApplier(org.neo4j.kernel.impl.transaction.command.NeoStoreBatchTransactionApplier) BatchTransactionApplier(org.neo4j.kernel.impl.api.BatchTransactionApplier) CacheAccessBackDoor(org.neo4j.kernel.impl.core.CacheAccessBackDoor) Test(org.junit.Test)

Example 34 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class PhysicalLogicalTransactionStoreTest method verifyTransaction.

private void verifyTransaction(TransactionIdStore transactionIdStore, TransactionMetadataCache positionCache, byte[] additionalHeader, int masterId, int authorId, long timeStarted, long latestCommittedTxWhenStarted, long timeCommitted, LogicalTransactionStore store) throws IOException {
    TransactionMetadata expectedMetadata;
    try (TransactionCursor cursor = store.getTransactions(TransactionIdStore.BASE_TX_ID + 1)) {
        boolean hasNext = cursor.next();
        assertTrue(hasNext);
        CommittedTransactionRepresentation tx = cursor.get();
        TransactionRepresentation transaction = tx.getTransactionRepresentation();
        assertArrayEquals(additionalHeader, transaction.additionalHeader());
        assertEquals(masterId, transaction.getMasterId());
        assertEquals(authorId, transaction.getAuthorId());
        assertEquals(timeStarted, transaction.getTimeStarted());
        assertEquals(timeCommitted, transaction.getTimeCommitted());
        assertEquals(latestCommittedTxWhenStarted, transaction.getLatestCommittedTxWhenStarted());
        expectedMetadata = new TransactionMetadata(masterId, authorId, tx.getStartEntry().getStartPosition(), tx.getStartEntry().checksum(), timeCommitted);
    }
    positionCache.clear();
    TransactionMetadata actualMetadata = store.getMetadataFor(transactionIdStore.getLastCommittedTransactionId());
    assertEquals(expectedMetadata, actualMetadata);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadata(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache.TransactionMetadata)

Example 35 with CommittedTransactionRepresentation

use of org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation in project neo4j by neo4j.

the class RecoverIndexDropIT method shouldDropIndexOnRecovery.

@Test
void shouldDropIndexOnRecovery() throws IOException {
    // given a transaction stream ending in an INDEX DROP command.
    CommittedTransactionRepresentation dropTransaction = prepareDropTransaction();
    DatabaseManagementService managementService = configure(new TestDatabaseManagementServiceBuilder(databaseLayout)).build();
    GraphDatabaseService db = managementService.database(DEFAULT_DATABASE_NAME);
    long initialIndexCount = currentIndexCount(db);
    createIndex(db);
    StorageEngineFactory storageEngineFactory = ((GraphDatabaseAPI) db).getDependencyResolver().resolveDependency(StorageEngineFactory.class);
    managementService.shutdown();
    appendDropTransactionToTransactionLog(databaseLayout.getTransactionLogsDirectory(), dropTransaction, storageEngineFactory);
    // when recovering this (the drop transaction with the index file intact)
    Monitors monitors = new Monitors();
    AssertRecoveryIsPerformed recoveryMonitor = new AssertRecoveryIsPerformed();
    monitors.addMonitorListener(recoveryMonitor);
    managementService = configure(new TestDatabaseManagementServiceBuilder(databaseLayout).setMonitors(monitors)).build();
    db = managementService.database(DEFAULT_DATABASE_NAME);
    try {
        assertTrue(recoveryMonitor.recoveryWasRequired);
        // then
        assertEquals(initialIndexCount, currentIndexCount(db));
    } finally {
        // and the ability to shut down w/o failing on still open files
        managementService.shutdown();
    }
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) StorageEngineFactory(org.neo4j.storageengine.api.StorageEngineFactory) Monitors(org.neo4j.monitoring.Monitors) DatabaseManagementService(org.neo4j.dbms.api.DatabaseManagementService) Test(org.junit.jupiter.api.Test)

Aggregations

CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)45 TransactionCursor (org.neo4j.kernel.impl.transaction.log.TransactionCursor)14 Test (org.junit.jupiter.api.Test)12 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)11 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)10 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)10 IOException (java.io.IOException)8 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)8 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)8 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)6 StorageEngine (org.neo4j.storageengine.api.StorageEngine)6 TransactionToApply (org.neo4j.kernel.impl.api.TransactionToApply)5 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)5 Test (org.junit.Test)4 Visitor (org.neo4j.helpers.collection.Visitor)4 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)4 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)4 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)4 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)4 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3