Search in sources :

Example 16 with PhysicalLogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel in project neo4j by neo4j.

the class LogTestUtils method filterTransactionLogFile.

private static void filterTransactionLogFile(FileSystemAbstraction fileSystem, Path file, final LogHook<LogEntry> filter, ChannelNativeAccessor channelNativeAccessor) throws IOException {
    filter.file(file);
    try (StoreChannel in = fileSystem.read(file)) {
        LogHeader logHeader = readLogHeader(ByteBuffers.allocate(CURRENT_FORMAT_LOG_HEADER_SIZE, INSTANCE), in, true, file);
        assert logHeader != null : "Looks like we tried to read a log header of an empty pre-allocated file.";
        PhysicalLogVersionedStoreChannel inChannel = new PhysicalLogVersionedStoreChannel(in, logHeader.getLogVersion(), logHeader.getLogFormatVersion(), file, channelNativeAccessor);
        ReadableLogChannel inBuffer = new ReadAheadLogChannel(inChannel, INSTANCE);
        LogEntryReader entryReader = new VersionAwareLogEntryReader(new TestCommandReaderFactory());
        LogEntry entry;
        while ((entry = entryReader.readLogEntry(inBuffer)) != null) {
            filter.test(entry);
        }
    }
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 17 with PhysicalLogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel in project neo4j by neo4j.

the class CheckTxLogsTest method writeContent.

private void writeContent(File log, ThrowingConsumer<TransactionLogWriter, IOException> consumer) throws IOException {
    FileSystemAbstraction fs = ensureLogExists(log);
    try (StoreChannel channel = fs.open(log, "rw");
        LogVersionedStoreChannel versionedChannel = new PhysicalLogVersionedStoreChannel(channel, 0, (byte) 0);
        PhysicalFlushableChannel writableLogChannel = new PhysicalFlushableChannel(versionedChannel)) {
        long offset = channel.size();
        channel.position(offset);
        consumer.accept(new TransactionLogWriter(new LogEntryWriter(writableLogChannel)));
    }
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PhysicalFlushableChannel) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) TransactionLogWriter(org.neo4j.kernel.impl.transaction.log.TransactionLogWriter) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)

Example 18 with PhysicalLogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel in project neo4j by neo4j.

the class LegacyLogEntryWriter method openWritableChannel.

public LogVersionedStoreChannel openWritableChannel(File file) throws IOException {
    final StoreChannel storeChannel = fs.open(file, "rw");
    final long version = getLegacyLogVersion(file.getName());
    return new PhysicalLogVersionedStoreChannel(storeChannel, version, CURRENT_LOG_VERSION);
}
Also used : LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)

Example 19 with PhysicalLogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel in project neo4j by neo4j.

the class RecoveryTest method writeSomeData.

private void writeSomeData(File file, Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException> visitor) throws IOException {
    try (LogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(fileSystemRule.get().open(file, "rw"), logVersion, CURRENT_LOG_VERSION);
        final PositionAwarePhysicalFlushableChannel writableLogChannel = new PositionAwarePhysicalFlushableChannel(versionedStoreChannel)) {
        writeLogHeader(writableLogChannel, logVersion, 2L);
        Consumer<LogPositionMarker> consumer = marker -> {
            try {
                writableLogChannel.getCurrentPosition(marker);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        };
        LogEntryWriter first = new LogEntryWriter(writableLogChannel);
        visitor.visit(Pair.of(first, consumer));
    }
}
Also used : LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) Assert.fail(org.junit.Assert.fail) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) BASE_TX_COMMIT_TIMESTAMP(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_COMMIT_TIMESTAMP) Recovery(org.neo4j.kernel.recovery.Recovery) StorageEngine(org.neo4j.storageengine.api.StorageEngine) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) CURRENT_LOG_VERSION(org.neo4j.kernel.impl.transaction.log.entry.LogVersions.CURRENT_LOG_VERSION) TestDirectory(org.neo4j.test.rule.TestDirectory) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Matchers.any(org.mockito.Matchers.any) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) Mockito.inOrder(org.mockito.Mockito.inOrder) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) Mockito.mock(org.mockito.Mockito.mock) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Pair(org.neo4j.helpers.collection.Pair) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) Mockito.verifyZeroInteractions(org.mockito.Mockito.verifyZeroInteractions) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) NO_MONITOR(org.neo4j.kernel.recovery.PositionToRecoverFrom.NO_MONITOR) InOrder(org.mockito.InOrder) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) Mockito.times(org.mockito.Mockito.times) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) File(java.io.File) Consumer(java.util.function.Consumer) Rule(org.junit.Rule) DefaultFileSystemRule(org.neo4j.test.rule.fs.DefaultFileSystemRule) Visitor(org.neo4j.helpers.collection.Visitor) Assert.assertEquals(org.junit.Assert.assertEquals) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker)

Example 20 with PhysicalLogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel in project neo4j by neo4j.

the class LogTestUtils method filterNeostoreLogicalLog.

public static File filterNeostoreLogicalLog(FileSystemAbstraction fileSystem, File file, final LogHook<LogEntry> filter) throws IOException {
    filter.file(file);
    File tempFile = new File(file.getAbsolutePath() + ".tmp");
    fileSystem.deleteFile(tempFile);
    try (StoreChannel in = fileSystem.open(file, "r");
        StoreChannel out = fileSystem.open(tempFile, "rw")) {
        LogHeader logHeader = transferLogicalLogHeader(in, out, ByteBuffer.allocate(LOG_HEADER_SIZE));
        PhysicalLogVersionedStoreChannel outChannel = new PhysicalLogVersionedStoreChannel(out, logHeader.logVersion, logHeader.logFormatVersion);
        PhysicalLogVersionedStoreChannel inChannel = new PhysicalLogVersionedStoreChannel(in, logHeader.logVersion, logHeader.logFormatVersion);
        ReadableLogChannel inBuffer = new ReadAheadLogChannel(inChannel, LogVersionBridge.NO_MORE_CHANNELS);
        LogEntryReader<ReadableLogChannel> entryReader = new VersionAwareLogEntryReader<>();
        LogEntry entry;
        while ((entry = entryReader.readLogEntry(inBuffer)) != null) {
            if (filter.test(entry)) {
            // TODO allright, write to outBuffer
            }
        }
    }
    return tempFile;
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) File(java.io.File) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Aggregations

PhysicalLogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)25 StoreChannel (org.neo4j.io.fs.StoreChannel)13 ReadAheadLogChannel (org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel)12 LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)10 LogHeaderReader.readLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader)9 Test (org.junit.jupiter.api.Test)8 LogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel)8 IOException (java.io.IOException)6 Path (java.nio.file.Path)6 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)6 File (java.io.File)5 ReadableLogChannel (org.neo4j.kernel.impl.transaction.log.ReadableLogChannel)5 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)5 ByteBuffer (java.nio.ByteBuffer)4 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)4 LogHeaderWriter.writeLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader)4 HeapScopedBuffer (org.neo4j.io.memory.HeapScopedBuffer)3 LogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader)3 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)3 NoSuchFileException (java.nio.file.NoSuchFileException)2