Search in sources :

Example 6 with LogEntryCommand

use of org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand in project neo4j by neo4j.

the class IndexCreationTest method verifyThatIndexCreationTransactionIsTheFirstOne.

private void verifyThatIndexCreationTransactionIsTheFirstOne() throws Exception {
    PhysicalLogFile pLogFile = db.getDependencyResolver().resolveDependency(PhysicalLogFile.class);
    long version = db.getDependencyResolver().resolveDependency(LogVersionRepository.class).getCurrentLogVersion();
    db.getDependencyResolver().resolveDependency(LogRotation.class).rotateLogFile();
    db.getDependencyResolver().resolveDependency(CheckPointer.class).forceCheckPoint(new SimpleTriggerInfo("test"));
    ReadableLogChannel logChannel = pLogFile.getReader(LogPosition.start(version));
    final AtomicBoolean success = new AtomicBoolean(false);
    try (IOCursor<LogEntry> cursor = new LogEntryCursor(new VersionAwareLogEntryReader<>(), logChannel)) {
        List<StorageCommand> commandsInFirstEntry = new ArrayList<>();
        boolean startFound = false;
        while (cursor.next()) {
            LogEntry entry = cursor.get();
            if (entry instanceof LogEntryStart) {
                if (startFound) {
                    throw new IllegalArgumentException("More than one start entry");
                }
                startFound = true;
            }
            if (startFound && entry instanceof LogEntryCommand) {
                commandsInFirstEntry.add(entry.<LogEntryCommand>as().getXaCommand());
            }
            if (entry instanceof LogEntryCommit) {
                // The first COMMIT
                assertTrue(startFound);
                assertFalse("Index creation transaction wasn't the first one", commandsInFirstEntry.isEmpty());
                List<StorageCommand> createCommands = Iterators.asList(new FilteringIterator<>(commandsInFirstEntry.iterator(), item -> item instanceof IndexDefineCommand));
                assertEquals(1, createCommands.size());
                success.set(true);
                break;
            }
        }
    }
    assertTrue("Didn't find any commit record in log " + version, success.get());
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) Iterators(org.neo4j.helpers.collection.Iterators) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) IOCursor(org.neo4j.cursor.IOCursor) Node(org.neo4j.graphdb.Node) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) ArrayList(java.util.ArrayList) StorageCommand(org.neo4j.storageengine.api.StorageCommand) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) After(org.junit.After) Transaction(org.neo4j.graphdb.Transaction) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) ExecutorService(java.util.concurrent.ExecutorService) Before(org.junit.Before) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) IndexDefineCommand(org.neo4j.kernel.impl.index.IndexDefineCommand) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) TestDirectory(org.neo4j.test.rule.TestDirectory) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) TimeUnit(java.util.concurrent.TimeUnit) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) Rule(org.junit.Rule) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) Executors.newCachedThreadPool(java.util.concurrent.Executors.newCachedThreadPool) Assert.assertFalse(org.junit.Assert.assertFalse) FilteringIterator(org.neo4j.helpers.collection.FilteringIterator) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) Assert.assertEquals(org.junit.Assert.assertEquals) Index(org.neo4j.graphdb.index.Index) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) IndexDefineCommand(org.neo4j.kernel.impl.index.IndexDefineCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) ArrayList(java.util.ArrayList) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 7 with LogEntryCommand

use of org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand in project neo4j by neo4j.

the class LogTruncationTest method assertHandlesLogTruncation.

private void assertHandlesLogTruncation(Command cmd) throws IOException {
    inMemoryChannel.reset();
    writer.serialize(new PhysicalTransactionRepresentation(Arrays.asList(cmd)));
    int bytesSuccessfullyWritten = inMemoryChannel.writerPosition();
    try {
        LogEntry logEntry = logEntryReader.readLogEntry(inMemoryChannel);
        StorageCommand command = ((LogEntryCommand) logEntry).getXaCommand();
        assertEquals(cmd, command);
    } catch (Exception e) {
        throw new AssertionError("Failed to deserialize " + cmd.toString() + ", because: ", e);
    }
    bytesSuccessfullyWritten--;
    while (bytesSuccessfullyWritten-- > 0) {
        inMemoryChannel.reset();
        writer.serialize(new PhysicalTransactionRepresentation(Arrays.asList(cmd)));
        inMemoryChannel.truncateTo(bytesSuccessfullyWritten);
        LogEntry deserialized = logEntryReader.readLogEntry(inMemoryChannel);
        assertNull("Deserialization did not detect log truncation!" + "Record: " + cmd + ", deserialized: " + deserialized, deserialized);
    }
}
Also used : LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) IOException(java.io.IOException)

Example 8 with LogEntryCommand

use of org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand in project neo4j by neo4j.

the class TxPullResponseEncodeDecodeTest method newCommittedTransactionRepresentation.

private CommittedTransactionRepresentation newCommittedTransactionRepresentation() {
    final long arbitraryRecordId = 27L;
    Command.NodeCommand command = new Command.NodeCommand(new NodeRecord(arbitraryRecordId), new NodeRecord(arbitraryRecordId));
    PhysicalTransactionRepresentation physicalTransactionRepresentation = new PhysicalTransactionRepresentation(asList(new LogEntryCommand(command).getXaCommand()));
    physicalTransactionRepresentation.setHeader(new byte[] {}, 0, 0, 0, 0, 0, 0);
    LogEntryStart startEntry = new LogEntryStart(0, 0, 0L, 0L, new byte[] {}, LogPosition.UNSPECIFIED);
    OnePhaseCommit commitEntry = new OnePhaseCommit(42, 0);
    return new CommittedTransactionRepresentation(startEntry, physicalTransactionRepresentation, commitEntry);
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) NodeRecord(org.neo4j.kernel.impl.store.record.NodeRecord) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) Command(org.neo4j.kernel.impl.transaction.command.Command) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation)

Example 9 with LogEntryCommand

use of org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand in project neo4j by neo4j.

the class PhysicalTransactionCursor method next.

@Override
public boolean next() throws IOException {
    // Clear the previous deserialized transaction so that it won't have to be kept in heap while deserializing
    // the next one. Could be problematic if both are really big.
    current = null;
    while (true) {
        if (!logEntryCursor.next()) {
            return false;
        }
        LogEntry entry = logEntryCursor.get();
        if (entry instanceof LogEntryInlinedCheckPoint) {
            // this is a good position anyhow
            channel.getCurrentPosition(lastGoodPositionMarker);
            continue;
        }
        assert entry instanceof LogEntryStart : "Expected Start entry, read " + entry + " instead";
        LogEntryStart startEntry = (LogEntryStart) entry;
        LogEntryCommit commitEntry;
        List<StorageCommand> entries = new ArrayList<>();
        while (true) {
            if (!logEntryCursor.next()) {
                return false;
            }
            entry = logEntryCursor.get();
            if (entry instanceof LogEntryCommit) {
                commitEntry = (LogEntryCommit) entry;
                break;
            }
            LogEntryCommand command = (LogEntryCommand) entry;
            entries.add(command.getCommand());
        }
        PhysicalTransactionRepresentation transaction = new PhysicalTransactionRepresentation(entries);
        transaction.setHeader(startEntry.getAdditionalHeader(), startEntry.getTimeWritten(), startEntry.getLastCommittedTxWhenTransactionStarted(), commitEntry.getTimeWritten(), -1, ANONYMOUS);
        current = new CommittedTransactionRepresentation(startEntry, transaction, commitEntry);
        channel.getCurrentPosition(lastGoodPositionMarker);
        return true;
    }
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) StorageCommand(org.neo4j.storageengine.api.StorageCommand) ArrayList(java.util.ArrayList) LogEntryInlinedCheckPoint(org.neo4j.kernel.impl.transaction.log.entry.LogEntryInlinedCheckPoint) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 10 with LogEntryCommand

use of org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand in project neo4j by neo4j.

the class CheckTxLogs method scan.

private <C extends Command, R extends AbstractBaseRecord> boolean scan(PhysicalLogFiles logFiles, InconsistenciesHandler handler, CheckType<C, R> check, boolean checkTxIds) throws IOException {
    out.println("Checking logs for " + check.name() + " inconsistencies");
    CommittedRecords<R> state = new CommittedRecords<>(check);
    List<CommandAndLogVersion> txCommands = new ArrayList<>();
    boolean validLogs = true;
    long commandsRead = 0;
    long lastSeenTxId = BASE_TX_ID;
    try (LogEntryCursor logEntryCursor = LogTestUtils.openLogs(fs, logFiles)) {
        while (logEntryCursor.next()) {
            LogEntry entry = logEntryCursor.get();
            if (entry instanceof LogEntryCommand) {
                StorageCommand command = ((LogEntryCommand) entry).getXaCommand();
                if (check.commandClass().isInstance(command)) {
                    long logVersion = logEntryCursor.getCurrentLogVersion();
                    txCommands.add(new CommandAndLogVersion(command, logVersion));
                }
            } else if (entry instanceof LogEntryCommit) {
                long txId = ((LogEntryCommit) entry).getTxId();
                if (checkTxIds) {
                    validLogs &= checkNoDuplicatedTxsInTheLog(lastSeenTxId, txId, handler);
                    lastSeenTxId = txId;
                }
                for (CommandAndLogVersion txCommand : txCommands) {
                    validLogs &= checkAndHandleInconsistencies(txCommand, check, state, txId, handler);
                }
                txCommands.clear();
            }
            commandsRead++;
        }
    }
    out.println("Processed " + commandsRead + " commands");
    out.println(state);
    if (!txCommands.isEmpty()) {
        out.println("Found " + txCommands.size() + " uncommitted commands at the end.");
        for (CommandAndLogVersion txCommand : txCommands) {
            validLogs &= checkAndHandleInconsistencies(txCommand, check, state, -1, handler);
        }
        txCommands.clear();
    }
    return validLogs;
}
Also used : LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) ArrayList(java.util.ArrayList) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Aggregations

LogEntryCommand (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand)14 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)9 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)8 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)7 StorageCommand (org.neo4j.storageengine.api.StorageCommand)7 Command (org.neo4j.kernel.impl.transaction.command.Command)6 PhysicalTransactionRepresentation (org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation)6 ArrayList (java.util.ArrayList)4 Test (org.junit.Test)4 NodeRecord (org.neo4j.kernel.impl.store.record.NodeRecord)4 OnePhaseCommit (org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit)4 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)3 IOException (java.io.IOException)2 LinkedList (java.util.LinkedList)2 RecordStorageCommandReaderFactory (org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageCommandReaderFactory)2 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)2 TransactionRepresentation (org.neo4j.kernel.impl.transaction.TransactionRepresentation)2 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)2 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)2 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)2