Search in sources :

Example 16 with LogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel in project neo4j by neo4j.

the class LegacyLogEntryWriterTest method shouldWriteTheHeaderInTheFile.

@Test
public void shouldWriteTheHeaderInTheFile() throws IOException {
    // given
    final LegacyLogEntryWriter writer = new LegacyLogEntryWriter(fs);
    final File output = new File(getLegacyLogFilename(3));
    final LogHeader header = new LogHeader(CURRENT_LOG_VERSION, 1, 42L);
    // when
    try (LogVersionedStoreChannel channel = writer.openWritableChannel(output)) {
        writer.writeLogHeader(channel, header);
    }
    // then
    assertEquals(header, readLogHeader(fs, output));
}
Also used : LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) File(java.io.File) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) Test(org.junit.Test)

Example 17 with LogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel in project neo4j by neo4j.

the class LegacyLogEntryWriterTest method shouldWriteAllTheEntryInSeveralCommitsToTheFile.

@Test
public void shouldWriteAllTheEntryInSeveralCommitsToTheFile() throws IOException {
    // given
    final LogVersionedStoreChannel channel = mock(LogVersionedStoreChannel.class);
    final LogEntryWriter logEntryWriter = mock(LogEntryWriter.class);
    final LegacyLogEntryWriter writer = new LegacyLogEntryWriter(fs, liftToFactory(logEntryWriter));
    final LogEntryStart start1 = new LogEntryStart(0, 1, 2L, 3L, EMPTY_ADDITIONAL_ARRAY, UNSPECIFIED);
    final LogEntryCommand command1 = new LogEntryCommand(new Command.NodeCommand(nodeRecord, nodeRecord));
    final LogEntryCommit commit1 = new OnePhaseCommit(42L, 43L);
    final LogEntryStart start2 = new LogEntryStart(9, 8, 7L, 6L, EMPTY_ADDITIONAL_ARRAY, UNSPECIFIED);
    final LogEntryCommand command2 = new LogEntryCommand(new Command.RelationshipCommand(relRecord, relRecord));
    final LogEntryCommit commit2 = new OnePhaseCommit(84L, 85L);
    // when
    IOCursor<LogEntry> cursor = mockCursor(start1, command1, commit1, start2, command2, commit2);
    writer.writeAllLogEntries(channel, cursor);
    // then
    verify(logEntryWriter, times(1)).writeStartEntry(0, 1, 2L, 3L, EMPTY_ADDITIONAL_ARRAY);
    final TransactionRepresentation expected1 = new PhysicalTransactionRepresentation(Arrays.asList(command1.getXaCommand()));
    verify(logEntryWriter, times(1)).serialize(eq(expected1));
    verify(logEntryWriter, times(1)).writeCommitEntry(42L, 43L);
    verify(logEntryWriter, times(1)).writeStartEntry(9, 8, 7L, 6L, EMPTY_ADDITIONAL_ARRAY);
    final TransactionRepresentation expected2 = new PhysicalTransactionRepresentation(Arrays.asList(command2.getXaCommand()));
    verify(logEntryWriter, times(1)).serialize(eq(expected2));
    verify(logEntryWriter, times(1)).writeCommitEntry(84L, 85L);
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) Command(org.neo4j.kernel.impl.transaction.command.Command) LogEntryCommand(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommand) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) PhysicalTransactionRepresentation(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionRepresentation) Test(org.junit.Test)

Example 18 with LogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel in project neo4j by neo4j.

the class ReaderLogVersionBridgeTest method shouldReturnOldChannelWhenNextChannelHasntGottenCompleteHeaderYet.

@Test
public void shouldReturnOldChannelWhenNextChannelHasntGottenCompleteHeaderYet() throws Exception {
    // given
    final ReaderLogVersionBridge bridge = new ReaderLogVersionBridge(fs, logFiles);
    final StoreChannel nextVersionWithIncompleteHeader = mock(StoreChannel.class);
    when(nextVersionWithIncompleteHeader.read(any(ByteBuffer.class))).thenReturn(LOG_HEADER_SIZE / 2);
    when(channel.getVersion()).thenReturn(version);
    when(fs.fileExists(file)).thenReturn(true);
    when(logFiles.getLogFileForVersion(version + 1)).thenReturn(file);
    when(fs.open(file, "r")).thenReturn(nextVersionWithIncompleteHeader);
    // when
    final LogVersionedStoreChannel result = bridge.next(channel);
    // then
    assertEquals(channel, result);
    verify(channel, never()).close();
}
Also used : LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) ReaderLogVersionBridge(org.neo4j.kernel.impl.transaction.log.ReaderLogVersionBridge) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 19 with LogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel in project neo4j by neo4j.

the class ReaderLogVersionBridgeTest method shouldOpenTheNextChannelWhenItExists.

@Test
void shouldOpenTheNextChannelWhenItExists() throws IOException {
    // given
    final StoreChannel newStoreChannel = mock(StoreChannel.class);
    final ReaderLogVersionBridge bridge = new ReaderLogVersionBridge(logFiles.getLogFile());
    when(channel.getVersion()).thenReturn(version);
    when(channel.getLogFormatVersion()).thenReturn(CURRENT_LOG_FORMAT_VERSION);
    when(fs.fileExists(any(Path.class))).thenReturn(true);
    when(fs.read(any(Path.class))).thenReturn(newStoreChannel);
    when(newStoreChannel.read(ArgumentMatchers.<ByteBuffer>any())).then(new Answer<>() {

        private int count;

        @Override
        public Integer answer(InvocationOnMock invocation) {
            count++;
            ByteBuffer buffer = invocation.getArgument(0);
            if (count == 1) {
                buffer.putLong(encodeLogVersion(version + 1, CURRENT_LOG_FORMAT_VERSION));
                return Long.BYTES;
            }
            if (count == 2) {
                buffer.putLong(42);
                buffer.putLong(1);
                buffer.putLong(2);
                buffer.putLong(3);
                buffer.putLong(4);
                buffer.putLong(5);
                // reserved
                buffer.putLong(0);
                return Long.BYTES * 7;
            }
            throw new AssertionError("Should only be called twice.");
        }
    });
    // when
    final LogVersionedStoreChannel result = bridge.next(channel, false);
    // then
    PhysicalLogVersionedStoreChannel expected = new PhysicalLogVersionedStoreChannel(newStoreChannel, version + 1, CURRENT_LOG_FORMAT_VERSION, Path.of("log.file"), ChannelNativeAccessor.EMPTY_ACCESSOR);
    assertEquals(expected, result);
    verify(channel).close();
}
Also used : Path(java.nio.file.Path) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) InvocationOnMock(org.mockito.invocation.InvocationOnMock) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) ReaderLogVersionBridge(org.neo4j.kernel.impl.transaction.log.ReaderLogVersionBridge) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.jupiter.api.Test)

Example 20 with LogVersionedStoreChannel

use of org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel in project neo4j by neo4j.

the class TransactionLogsRecoveryTest method writeSomeDataWithVersion.

private void writeSomeDataWithVersion(Path file, Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException> visitor, KernelVersion version) throws IOException {
    try (LogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(fileSystem.write(file), logVersion, CURRENT_LOG_FORMAT_VERSION, file, EMPTY_ACCESSOR);
        PositionAwarePhysicalFlushableChecksumChannel writableLogChannel = new PositionAwarePhysicalFlushableChecksumChannel(versionedStoreChannel, new HeapScopedBuffer(1, KibiByte, INSTANCE))) {
        writeLogHeader(writableLogChannel, new LogHeader(logVersion, 2L, StoreId.UNKNOWN));
        writableLogChannel.beginChecksum();
        Consumer<LogPositionMarker> consumer = marker -> {
            try {
                writableLogChannel.getCurrentPosition(marker);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        };
        LogEntryWriter first = new LogEntryWriter(writableLogChannel, version);
        visitor.visit(Pair.of(first, consumer));
    }
}
Also used : HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) Visitor(org.neo4j.internal.helpers.collection.Visitor) BeforeEach(org.junit.jupiter.api.BeforeEach) LifecycleAdapter(org.neo4j.kernel.lifecycle.LifecycleAdapter) MutableInt(org.apache.commons.lang3.mutable.MutableInt) Log(org.neo4j.logging.Log) DatabaseIdFactory.from(org.neo4j.kernel.database.DatabaseIdFactory.from) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) LogFilesBuilder(org.neo4j.kernel.impl.transaction.log.files.LogFilesBuilder) Config(org.neo4j.configuration.Config) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) DatabaseLayout(org.neo4j.io.layout.DatabaseLayout) Mockito.verifyNoInteractions(org.mockito.Mockito.verifyNoInteractions) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) LogCheckPointEvent(org.neo4j.kernel.impl.transaction.tracing.LogCheckPointEvent) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) Monitors(org.neo4j.monitoring.Monitors) BASE_TX_CHECKSUM(org.neo4j.storageengine.api.TransactionIdStore.BASE_TX_CHECKSUM) PageCacheTracer(org.neo4j.io.pagecache.tracing.PageCacheTracer) LogVersionRepository(org.neo4j.storageengine.api.LogVersionRepository) PositionAwarePhysicalFlushableChecksumChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChecksumChannel) Path(java.nio.file.Path) EMPTY_CHECKER(org.neo4j.kernel.recovery.RecoveryStartupChecker.EMPTY_CHECKER) StorageEngine(org.neo4j.storageengine.api.StorageEngine) NO_RECOVERY_REQUIRED(org.neo4j.kernel.recovery.RecoveryStartInformation.NO_RECOVERY_REQUIRED) TestDirectory(org.neo4j.test.rule.TestDirectory) BASE_TX_COMMIT_TIMESTAMP(org.neo4j.storageengine.api.TransactionIdStore.BASE_TX_COMMIT_TIMESTAMP) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) Instant(java.time.Instant) Neo4jLayoutExtension(org.neo4j.test.extension.Neo4jLayoutExtension) Test(org.junit.jupiter.api.Test) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) INSTANCE(org.neo4j.memory.EmptyMemoryTracker.INSTANCE) TestLogEntryReader.logEntryReader(org.neo4j.kernel.impl.transaction.log.TestLogEntryReader.logEntryReader) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) DatabaseStartAbortedException(org.neo4j.dbms.database.DatabaseStartAbortedException) Mockito.mock(org.mockito.Mockito.mock) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) Assertions.fail(org.junit.jupiter.api.Assertions.fail) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) CsvSource(org.junit.jupiter.params.provider.CsvSource) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) ProgressReporter(org.neo4j.common.ProgressReporter) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) KernelVersion(org.neo4j.kernel.KernelVersion) ArrayUtils(org.apache.commons.lang3.ArrayUtils) NULL(org.neo4j.io.pagecache.tracing.PageCacheTracer.NULL) DatabaseStartupController(org.neo4j.kernel.database.DatabaseStartupController) LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) EMPTY_ACCESSOR(org.neo4j.kernel.impl.transaction.log.files.ChannelNativeAccessor.EMPTY_ACCESSOR) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Inject(org.neo4j.test.extension.Inject) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) StoreId(org.neo4j.storageengine.api.StoreId) TransactionApplicationMode(org.neo4j.storageengine.api.TransactionApplicationMode) GraphDatabaseInternalSettings(org.neo4j.configuration.GraphDatabaseInternalSettings) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) ArgumentMatchers.anyInt(org.mockito.ArgumentMatchers.anyInt) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) Files(java.nio.file.Files) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) ExceptionUtils.getRootCause(org.apache.commons.lang3.exception.ExceptionUtils.getRootCause) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) CURRENT_LOG_FORMAT_VERSION(org.neo4j.kernel.impl.transaction.log.entry.LogVersions.CURRENT_LOG_FORMAT_VERSION) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) Mockito.verify(org.mockito.Mockito.verify) Consumer(java.util.function.Consumer) AfterEach(org.junit.jupiter.api.AfterEach) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) UUID.randomUUID(java.util.UUID.randomUUID) Mockito.never(org.mockito.Mockito.never) CURRENT_FORMAT_LOG_HEADER_SIZE(org.neo4j.kernel.impl.transaction.log.entry.LogVersions.CURRENT_FORMAT_LOG_HEADER_SIZE) NO_MONITOR(org.neo4j.kernel.recovery.RecoveryStartInformationProvider.NO_MONITOR) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) Pair(org.neo4j.internal.helpers.collection.Pair) KibiByte(org.neo4j.io.ByteUnit.KibiByte) TransactionIdStore(org.neo4j.storageengine.api.TransactionIdStore) SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PositionAwarePhysicalFlushableChecksumChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChecksumChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker)

Aggregations

LogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel)22 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)11 PhysicalLogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)10 StoreChannel (org.neo4j.io.fs.StoreChannel)8 ReadAheadLogChannel (org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel)8 File (java.io.File)6 Test (org.junit.Test)6 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)6 LogEntryCommit (org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit)6 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)6 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)6 LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)6 Test (org.junit.jupiter.api.Test)5 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)5 ReaderLogVersionBridge (org.neo4j.kernel.impl.transaction.log.ReaderLogVersionBridge)5 IOException (java.io.IOException)4 ByteBuffer (java.nio.ByteBuffer)4 Path (java.nio.file.Path)4 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)3 LogVersionBridge (org.neo4j.kernel.impl.transaction.log.LogVersionBridge)3