Search in sources :

Example 1 with HeapScopedBuffer

use of org.neo4j.io.memory.HeapScopedBuffer in project neo4j by neo4j.

the class BatchingTransactionAppenderTest method shouldNotCallTransactionClosedOnFailedAppendedTransaction.

@Test
void shouldNotCallTransactionClosedOnFailedAppendedTransaction() throws Exception {
    // GIVEN
    long txId = 3;
    String failureMessage = "Forces a failure";
    FlushablePositionAwareChecksumChannel channel = spy(new PositionAwarePhysicalFlushableChecksumChannel(mock(PhysicalLogVersionedStoreChannel.class), new HeapScopedBuffer(Long.BYTES * 2, INSTANCE)));
    IOException failure = new IOException(failureMessage);
    when(channel.putLong(anyLong())).thenThrow(failure);
    when(logFile.getTransactionLogWriter()).thenReturn(new TransactionLogWriter(channel, new DbmsLogEntryWriterFactory(() -> LATEST)));
    when(transactionIdStore.nextCommittingTransactionId()).thenReturn(txId);
    when(transactionIdStore.getLastCommittedTransaction()).thenReturn(new TransactionId(txId, BASE_TX_CHECKSUM, BASE_TX_COMMIT_TIMESTAMP));
    Mockito.reset(databaseHealth);
    TransactionAppender appender = life.add(createTransactionAppender());
    // WHEN
    TransactionRepresentation transaction = mock(TransactionRepresentation.class);
    when(transaction.additionalHeader()).thenReturn(new byte[0]);
    var e = assertThrows(IOException.class, () -> appender.append(new TransactionToApply(transaction, NULL), logAppendEvent));
    assertSame(failure, e);
    verify(transactionIdStore).nextCommittingTransactionId();
    verify(transactionIdStore, never()).transactionClosed(eq(txId), anyLong(), anyLong(), any(CursorContext.class));
    verify(databaseHealth).panic(failure);
}
Also used : TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) IOException(java.io.IOException) CursorContext(org.neo4j.io.pagecache.context.CursorContext) TransactionId(org.neo4j.storageengine.api.TransactionId) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) DbmsLogEntryWriterFactory(org.neo4j.kernel.database.DbmsLogEntryWriterFactory) Test(org.junit.jupiter.api.Test)

Example 2 with HeapScopedBuffer

use of org.neo4j.io.memory.HeapScopedBuffer in project neo4j by neo4j.

the class PhysicalFlushableChannelTest method shouldBeAbleToWriteSmallNumberOfBytes.

@Test
void shouldBeAbleToWriteSmallNumberOfBytes() throws IOException {
    final Path firstFile = directory.homePath().resolve("file1");
    StoreChannel storeChannel = fileSystem.write(firstFile);
    PhysicalLogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(storeChannel, 1, (byte) -1, firstFile, nativeChannelAccessor);
    int length = 26_145;
    byte[] bytes;
    try (PhysicalFlushableChannel channel = new PhysicalFlushableChannel(versionedStoreChannel, new HeapScopedBuffer(100, INSTANCE))) {
        bytes = generateBytes(length);
        channel.put(bytes, length);
    }
    byte[] writtenBytes = new byte[length];
    try (InputStream in = Files.newInputStream(firstFile)) {
        in.read(writtenBytes);
    }
    assertArrayEquals(bytes, writtenBytes);
}
Also used : Path(java.nio.file.Path) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) PhysicalFlushableChannel(org.neo4j.io.fs.PhysicalFlushableChannel) InputStream(java.io.InputStream) StoreChannel(org.neo4j.io.fs.StoreChannel) Test(org.junit.jupiter.api.Test)

Example 3 with HeapScopedBuffer

use of org.neo4j.io.memory.HeapScopedBuffer in project neo4j by neo4j.

the class RecordPropertyCursor method growBuffer.

public ByteBuffer growBuffer(int minAdditionalCapacity) {
    buffer.flip();
    int oldCapacity = buffer.capacity();
    int newCapacity = Math.max(oldCapacity, minAdditionalCapacity) + oldCapacity;
    var oldScopedBuffer = scopedBuffer;
    setScopedBuffer(new HeapScopedBuffer(newCapacity, memoryTracker));
    buffer.put(oldScopedBuffer.getBuffer());
    oldScopedBuffer.close();
    return buffer;
}
Also used : HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer)

Example 4 with HeapScopedBuffer

use of org.neo4j.io.memory.HeapScopedBuffer in project neo4j by neo4j.

the class BlockStorageTest method assertContents.

private static void assertContents(SimpleLongLayout layout, BlockStorage<MutableLong, MutableLong> storage, Iterable<List<BlockEntry<MutableLong, MutableLong>>> expectedBlocks) throws IOException {
    try (BlockReader<MutableLong, MutableLong> reader = storage.reader(false)) {
        for (List<BlockEntry<MutableLong, MutableLong>> expectedBlock : expectedBlocks) {
            try (BlockEntryReader<MutableLong, MutableLong> block = reader.nextBlock(new HeapScopedBuffer(1024, INSTANCE))) {
                assertNotNull(block);
                assertEquals(expectedBlock.size(), block.entryCount());
                for (BlockEntry<MutableLong, MutableLong> expectedEntry : expectedBlock) {
                    assertTrue(block.next());
                    assertEquals(0, layout.compare(expectedEntry.key(), block.key()));
                    assertEquals(expectedEntry.value(), block.value());
                }
            }
        }
    }
}
Also used : HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) MutableLong(org.apache.commons.lang3.mutable.MutableLong)

Example 5 with HeapScopedBuffer

use of org.neo4j.io.memory.HeapScopedBuffer in project neo4j by neo4j.

the class RecoverIndexDropIT method appendDropTransactionToTransactionLog.

private void appendDropTransactionToTransactionLog(Path transactionLogsDirectory, CommittedTransactionRepresentation dropTransaction, StorageEngineFactory storageEngineFactory) throws IOException {
    LogFiles logFiles = LogFilesBuilder.logFilesBasedOnlyBuilder(transactionLogsDirectory, fs).withCommandReaderFactory(storageEngineFactory.commandReaderFactory()).build();
    LogFile logFile = logFiles.getLogFile();
    try (ReadableLogChannel reader = logFile.getReader(logFile.extractHeader(0).getStartPosition())) {
        LogEntryReader logEntryReader = new VersionAwareLogEntryReader(storageEngineFactory.commandReaderFactory());
        while (logEntryReader.readLogEntry(reader) != null) {
        }
        LogPosition position = logEntryReader.lastPosition();
        StoreChannel storeChannel = fs.write(logFile.getLogFileForVersion(logFile.getHighestLogVersion()));
        storeChannel.position(position.getByteOffset());
        try (PhysicalFlushableChecksumChannel writeChannel = new PhysicalFlushableChecksumChannel(storeChannel, new HeapScopedBuffer(100, INSTANCE))) {
            new LogEntryWriter<>(writeChannel, KernelVersion.LATEST).serialize(dropTransaction);
        }
    }
}
Also used : LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) PhysicalFlushableChecksumChannel(org.neo4j.io.fs.PhysicalFlushableChecksumChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Aggregations

HeapScopedBuffer (org.neo4j.io.memory.HeapScopedBuffer)20 StoreChannel (org.neo4j.io.fs.StoreChannel)10 Test (org.junit.jupiter.api.Test)9 Path (java.nio.file.Path)8 PhysicalFlushableChecksumChannel (org.neo4j.io.fs.PhysicalFlushableChecksumChannel)6 ByteBuffer (java.nio.ByteBuffer)5 IOException (java.io.IOException)4 PhysicalLogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)4 InputStream (java.io.InputStream)3 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)3 LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)3 LogFile (org.neo4j.kernel.impl.transaction.log.files.LogFile)3 Checksum (java.util.zip.Checksum)2 LogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel)2 StoreId (org.neo4j.storageengine.api.StoreId)2 UncheckedIOException (java.io.UncheckedIOException)1 Files (java.nio.file.Files)1 NoSuchFileException (java.nio.file.NoSuchFileException)1 Instant (java.time.Instant)1 UUID.randomUUID (java.util.UUID.randomUUID)1