Search in sources :

Example 6 with VersionAwareLogEntryReader

use of org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader in project neo4j by neo4j.

the class TxPullResponseDecoder method decode.

@Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
    NetworkReadableClosableChannelNetty4 logChannel = new NetworkReadableClosableChannelNetty4(msg);
    StoreId storeId = StoreIdMarshal.INSTANCE.unmarshal(logChannel);
    LogEntryReader<NetworkReadableClosableChannelNetty4> reader = new VersionAwareLogEntryReader<>(new RecordStorageCommandReaderFactory());
    PhysicalTransactionCursor<NetworkReadableClosableChannelNetty4> transactionCursor = new PhysicalTransactionCursor<>(logChannel, reader);
    transactionCursor.next();
    CommittedTransactionRepresentation tx = transactionCursor.get();
    if (tx != null) {
        out.add(new TxPullResponse(storeId, tx));
    }
}
Also used : PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) RecordStorageCommandReaderFactory(org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageCommandReaderFactory) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) StoreId(org.neo4j.causalclustering.identity.StoreId) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) NetworkReadableClosableChannelNetty4(org.neo4j.causalclustering.messaging.NetworkReadableClosableChannelNetty4)

Example 7 with VersionAwareLogEntryReader

use of org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader in project neo4j by neo4j.

the class ReplicatedTransactionFactory method read.

public static TransactionRepresentation read(NetworkReadableClosableChannelNetty4 channel, byte[] extraHeader) throws IOException {
    LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>(new RecordStorageCommandReaderFactory());
    int authorId = channel.getInt();
    int masterId = channel.getInt();
    long latestCommittedTxWhenStarted = channel.getLong();
    long timeStarted = channel.getLong();
    long timeCommitted = channel.getLong();
    int lockSessionId = channel.getInt();
    int headerLength = channel.getInt();
    byte[] header;
    if (headerLength == 0) {
        header = extraHeader;
    } else {
        header = new byte[headerLength];
    }
    channel.get(header, headerLength);
    LogEntryCommand entryRead;
    List<StorageCommand> commands = new LinkedList<>();
    while ((entryRead = (LogEntryCommand) reader.readLogEntry(channel)) != null) {
        commands.add(entryRead.getXaCommand());
    }
    PhysicalTransactionRepresentation tx = new PhysicalTransactionRepresentation(commands);
    tx.setHeader(header, masterId, authorId, timeStarted, latestCommittedTxWhenStarted, timeCommitted, lockSessionId);
    return tx;
}
Also used : RecordStorageCommandReaderFactory(org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageCommandReaderFactory) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LinkedList(java.util.LinkedList) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)

Example 8 with VersionAwareLogEntryReader

use of org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader in project neo4j by neo4j.

the class ReplicatedTokenRequestSerializer method extractCommands.

static Collection<StorageCommand> extractCommands(byte[] commandBytes) {
    ByteBuf txBuffer = Unpooled.wrappedBuffer(commandBytes);
    NetworkReadableClosableChannelNetty4 channel = new NetworkReadableClosableChannelNetty4(txBuffer);
    LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>(new RecordStorageCommandReaderFactory());
    LogEntryCommand entryRead;
    List<StorageCommand> commands = new LinkedList<>();
    try {
        while ((entryRead = (LogEntryCommand) reader.readLogEntry(channel)) != null) {
            commands.add(entryRead.getXaCommand());
        }
    } catch (IOException e) {
        // TODO: Handle or throw.
        e.printStackTrace();
    }
    return commands;
}
Also used : RecordStorageCommandReaderFactory(org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageCommandReaderFactory) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) IOException(java.io.IOException) ByteBuf(io.netty.buffer.ByteBuf) NetworkReadableClosableChannelNetty4(org.neo4j.causalclustering.messaging.NetworkReadableClosableChannelNetty4) LinkedList(java.util.LinkedList) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)

Example 9 with VersionAwareLogEntryReader

use of org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader in project neo4j by neo4j.

the class IndexCreationTest method verifyThatIndexCreationTransactionIsTheFirstOne.

private void verifyThatIndexCreationTransactionIsTheFirstOne() throws Exception {
    PhysicalLogFile pLogFile = db.getDependencyResolver().resolveDependency(PhysicalLogFile.class);
    long version = db.getDependencyResolver().resolveDependency(LogVersionRepository.class).getCurrentLogVersion();
    db.getDependencyResolver().resolveDependency(LogRotation.class).rotateLogFile();
    db.getDependencyResolver().resolveDependency(CheckPointer.class).forceCheckPoint(new SimpleTriggerInfo("test"));
    ReadableLogChannel logChannel = pLogFile.getReader(LogPosition.start(version));
    final AtomicBoolean success = new AtomicBoolean(false);
    try (IOCursor<LogEntry> cursor = new LogEntryCursor(new VersionAwareLogEntryReader<>(), logChannel)) {
        List<StorageCommand> commandsInFirstEntry = new ArrayList<>();
        boolean startFound = false;
        while (cursor.next()) {
            LogEntry entry = cursor.get();
            if (entry instanceof LogEntryStart) {
                if (startFound) {
                    throw new IllegalArgumentException("More than one start entry");
                }
                startFound = true;
            }
            if (startFound && entry instanceof LogEntryCommand) {
                commandsInFirstEntry.add(entry.<LogEntryCommand>as().getXaCommand());
            }
            if (entry instanceof LogEntryCommit) {
                // The first COMMIT
                assertTrue(startFound);
                assertFalse("Index creation transaction wasn't the first one", commandsInFirstEntry.isEmpty());
                List<StorageCommand> createCommands = Iterators.asList(new FilteringIterator<>(commandsInFirstEntry.iterator(), item -> item instanceof IndexDefineCommand));
                assertEquals(1, createCommands.size());
                success.set(true);
                break;
            }
        }
    }
    assertTrue("Didn't find any commit record in log " + version, success.get());
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) Iterators(org.neo4j.helpers.collection.Iterators) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) IOCursor(org.neo4j.cursor.IOCursor) Node(org.neo4j.graphdb.Node) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) ArrayList(java.util.ArrayList) StorageCommand(org.neo4j.storageengine.api.StorageCommand) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) After(org.junit.After) Transaction(org.neo4j.graphdb.Transaction) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) ExecutorService(java.util.concurrent.ExecutorService) Before(org.junit.Before) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) IndexDefineCommand(org.neo4j.kernel.impl.index.IndexDefineCommand) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) TestDirectory(org.neo4j.test.rule.TestDirectory) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) TimeUnit(java.util.concurrent.TimeUnit) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) Rule(org.junit.Rule) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) Executors.newCachedThreadPool(java.util.concurrent.Executors.newCachedThreadPool) Assert.assertFalse(org.junit.Assert.assertFalse) FilteringIterator(org.neo4j.helpers.collection.FilteringIterator) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) Assert.assertEquals(org.junit.Assert.assertEquals) Index(org.neo4j.graphdb.index.Index) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) IndexDefineCommand(org.neo4j.kernel.impl.index.IndexDefineCommand) StorageCommand(org.neo4j.storageengine.api.StorageCommand) ArrayList(java.util.ArrayList) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 10 with VersionAwareLogEntryReader

use of org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader in project neo4j by neo4j.

the class BackupProtocolTest method shouldGatherForensicsInFullBackupRequest.

private void shouldGatherForensicsInFullBackupRequest(boolean forensics) throws Exception {
    // GIVEN
    Response<Void> response = Response.EMPTY;
    StoreId storeId = response.getStoreId();
    String host = "localhost";
    int port = BackupServer.DEFAULT_PORT;
    LifeSupport life = new LifeSupport();
    LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
    BackupClient client = life.add(new BackupClient(host, port, null, NullLogProvider.getInstance(), storeId, 10_000, mock(ResponseUnpacker.class), mock(ByteCounterMonitor.class), mock(RequestMonitor.class), reader));
    ControlledBackupInterface backup = new ControlledBackupInterface();
    life.add(new BackupServer(backup, new HostnamePort(host, port), NullLogProvider.getInstance(), mock(ByteCounterMonitor.class), mock(RequestMonitor.class)));
    life.start();
    try {
        // WHEN
        StoreWriter writer = mock(StoreWriter.class);
        client.fullBackup(writer, forensics);
        // THEN
        assertEquals(forensics, backup.receivedForensics);
    } finally {
        life.shutdown();
    }
}
Also used : HostnamePort(org.neo4j.helpers.HostnamePort) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) StoreId(org.neo4j.kernel.impl.store.StoreId) StoreWriter(org.neo4j.com.storecopy.StoreWriter) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)

Aggregations

VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)39 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)18 ReadableLogChannel (org.neo4j.kernel.impl.transaction.log.ReadableLogChannel)9 LogFiles (org.neo4j.kernel.impl.transaction.log.files.LogFiles)9 IOException (java.io.IOException)8 ReadAheadLogChannel (org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel)8 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)8 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)8 File (java.io.File)7 Test (org.junit.Test)7 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)7 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)7 LatestCheckPointFinder (org.neo4j.kernel.recovery.LatestCheckPointFinder)7 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)6 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)5 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)5 LogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader)5 LogFile (org.neo4j.kernel.impl.transaction.log.files.LogFile)5 CountDownLatch (java.util.concurrent.CountDownLatch)4 StoreChannel (org.neo4j.io.fs.StoreChannel)4