Search in sources :

Example 6 with Visitor

use of org.neo4j.helpers.collection.Visitor in project neo4j by neo4j.

the class RecoveryTest method writeSomeData.

private void writeSomeData(File file, Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException> visitor) throws IOException {
    try (LogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(fileSystemRule.get().open(file, "rw"), logVersion, CURRENT_LOG_VERSION);
        final PositionAwarePhysicalFlushableChannel writableLogChannel = new PositionAwarePhysicalFlushableChannel(versionedStoreChannel)) {
        writeLogHeader(writableLogChannel, logVersion, 2L);
        Consumer<LogPositionMarker> consumer = marker -> {
            try {
                writableLogChannel.getCurrentPosition(marker);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        };
        LogEntryWriter first = new LogEntryWriter(writableLogChannel);
        visitor.visit(Pair.of(first, consumer));
    }
}
Also used : LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) Assert.fail(org.junit.Assert.fail) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) BASE_TX_COMMIT_TIMESTAMP(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_COMMIT_TIMESTAMP) Recovery(org.neo4j.kernel.recovery.Recovery) StorageEngine(org.neo4j.storageengine.api.StorageEngine) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) CURRENT_LOG_VERSION(org.neo4j.kernel.impl.transaction.log.entry.LogVersions.CURRENT_LOG_VERSION) TestDirectory(org.neo4j.test.rule.TestDirectory) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Matchers.any(org.mockito.Matchers.any) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) Mockito.inOrder(org.mockito.Mockito.inOrder) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) Mockito.mock(org.mockito.Mockito.mock) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Pair(org.neo4j.helpers.collection.Pair) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) Mockito.verifyZeroInteractions(org.mockito.Mockito.verifyZeroInteractions) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) NO_MONITOR(org.neo4j.kernel.recovery.PositionToRecoverFrom.NO_MONITOR) InOrder(org.mockito.InOrder) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) Mockito.times(org.mockito.Mockito.times) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) File(java.io.File) Consumer(java.util.function.Consumer) Rule(org.junit.Rule) DefaultFileSystemRule(org.neo4j.test.rule.fs.DefaultFileSystemRule) Visitor(org.neo4j.helpers.collection.Visitor) Assert.assertEquals(org.junit.Assert.assertEquals) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker)

Example 7 with Visitor

use of org.neo4j.helpers.collection.Visitor in project neo4j by neo4j.

the class ResponsePacker method packTransactionStreamResponse.

public <T> Response<T> packTransactionStreamResponse(RequestContext context, T response) {
    final long toStartFrom = context.lastAppliedTransaction() + 1;
    final long toEndAt = transactionIdStore.getLastCommittedTransactionId();
    TransactionStream transactions = visitor -> {
        if (toStartFrom > BASE_TX_ID && toStartFrom <= toEndAt) {
            extractTransactions(toStartFrom, filterVisitor(visitor, toEndAt));
        }
    };
    return new TransactionStreamResponse<>(response, storeId.get(), transactions, ResourceReleaser.NO_OP);
}
Also used : Response(org.neo4j.com.Response) StoreId(org.neo4j.kernel.impl.store.StoreId) TransactionStream(org.neo4j.com.TransactionStream) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) Supplier(java.util.function.Supplier) RequestContext(org.neo4j.com.RequestContext) IOCursor(org.neo4j.cursor.IOCursor) TransactionStreamResponse(org.neo4j.com.TransactionStreamResponse) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) ResourceReleaser(org.neo4j.com.ResourceReleaser) Visitor(org.neo4j.helpers.collection.Visitor) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) BASE_TX_ID(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_ID) TransactionStreamResponse(org.neo4j.com.TransactionStreamResponse) TransactionStream(org.neo4j.com.TransactionStream)

Example 8 with Visitor

use of org.neo4j.helpers.collection.Visitor in project neo4j by neo4j.

the class ResponsePackerIT method shouldPackTheHighestTxCommittedAsObligation.

@Test
public void shouldPackTheHighestTxCommittedAsObligation() throws Exception {
    // GIVEN
    LogicalTransactionStore transactionStore = mock(LogicalTransactionStore.class);
    FileSystemAbstraction fs = fsRule.get();
    PageCache pageCache = pageCacheRule.getPageCache(fs);
    try (NeoStores neoStore = createNeoStore(fs, pageCache)) {
        MetaDataStore store = neoStore.getMetaDataStore();
        store.transactionCommitted(2, 111, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(3, 222, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(4, 333, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(5, 444, BASE_TX_COMMIT_TIMESTAMP);
        store.transactionCommitted(6, 555, BASE_TX_COMMIT_TIMESTAMP);
        // skip 7 to emulate the fact we have an hole in the committed tx ids list
        final long expectedTxId = 8L;
        store.transactionCommitted(expectedTxId, 777, BASE_TX_COMMIT_TIMESTAMP);
        ResponsePacker packer = new ResponsePacker(transactionStore, store, Suppliers.singleton(newStoreIdForCurrentVersion()));
        // WHEN
        Response<Object> response = packer.packTransactionObligationResponse(new RequestContext(0, 0, 0, 0, 0), new Object());
        // THEN
        assertTrue(response instanceof TransactionObligationResponse);
        ((TransactionObligationResponse) response).accept(new Response.Handler() {

            @Override
            public void obligation(long txId) throws IOException {
                assertEquals(expectedTxId, txId);
            }

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> transactions() {
                throw new UnsupportedOperationException("not expected");
            }
        });
    }
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Visitor(org.neo4j.helpers.collection.Visitor) MetaDataStore(org.neo4j.kernel.impl.store.MetaDataStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) IOException(java.io.IOException) Response(org.neo4j.com.Response) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) NeoStores(org.neo4j.kernel.impl.store.NeoStores) TransactionObligationResponse(org.neo4j.com.TransactionObligationResponse) RequestContext(org.neo4j.com.RequestContext) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 9 with Visitor

use of org.neo4j.helpers.collection.Visitor in project neo4j by neo4j.

the class ResponsePackerTest method shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving.

@Test
public void shouldHaveFixedTargetTransactionIdEvenIfLastTransactionIdIsMoving() throws Exception {
    // GIVEN
    LogicalTransactionStore transactionStore = mock(LogicalTransactionStore.class);
    long lastAppliedTransactionId = 5L;
    TransactionCursor endlessCursor = new EndlessCursor(lastAppliedTransactionId + 1);
    when(transactionStore.getTransactions(anyLong())).thenReturn(endlessCursor);
    final long targetTransactionId = 8L;
    final TransactionIdStore transactionIdStore = new DeadSimpleTransactionIdStore(targetTransactionId, 0, BASE_TX_COMMIT_TIMESTAMP, 0, 0);
    ResponsePacker packer = new ResponsePacker(transactionStore, transactionIdStore, Suppliers.singleton(StoreIdTestFactory.newStoreIdForCurrentVersion()));
    // WHEN
    Response<Object> response = packer.packTransactionStreamResponse(requestContextStartingAt(5L), null);
    final AtomicLong nextExpectedVisit = new AtomicLong(lastAppliedTransactionId);
    response.accept(new Response.Handler() {

        @Override
        public void obligation(long txId) throws IOException {
            fail("Should not be called");
        }

        @Override
        public Visitor<CommittedTransactionRepresentation, Exception> transactions() {
            return new Visitor<CommittedTransactionRepresentation, Exception>() {

                @Override
                public boolean visit(CommittedTransactionRepresentation element) {
                    // THEN
                    long txId = element.getCommitEntry().getTxId();
                    assertThat(txId, lessThanOrEqualTo(targetTransactionId));
                    assertEquals(nextExpectedVisit.incrementAndGet(), txId);
                    // Move the target transaction id forward one step, effectively always keeping it out of reach
                    transactionIdStore.setLastCommittedAndClosedTransactionId(transactionIdStore.getLastCommittedTransactionId() + 1, 0, BASE_TX_COMMIT_TIMESTAMP, 3, 4);
                    return true;
                }
            };
        }
    });
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) Visitor(org.neo4j.helpers.collection.Visitor) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) IOException(java.io.IOException) IOException(java.io.IOException) Response(org.neo4j.com.Response) AtomicLong(java.util.concurrent.atomic.AtomicLong) TransactionCursor(org.neo4j.kernel.impl.transaction.log.TransactionCursor) Test(org.junit.Test)

Example 10 with Visitor

use of org.neo4j.helpers.collection.Visitor in project neo4j by neo4j.

the class MadeUpServerImplementation method streamBackTransactions.

@Override
public Response<Integer> streamBackTransactions(int responseToSendBack, final int txCount) {
    TransactionStream transactions = new TransactionStream() {

        @Override
        public void accept(Visitor<CommittedTransactionRepresentation, Exception> visitor) throws Exception {
            for (int i = 1; i <= txCount; i++) {
                CommittedTransactionRepresentation transaction = createTransaction(TransactionIdStore.BASE_TX_ID + i);
                visitor.visit(transaction);
            }
        }
    };
    return new TransactionStreamResponse<>(responseToSendBack, storeIdToRespondWith, transactions, ResourceReleaser.NO_OP);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) Visitor(org.neo4j.helpers.collection.Visitor)

Aggregations

Visitor (org.neo4j.helpers.collection.Visitor)12 IOException (java.io.IOException)7 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)7 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)7 Response (org.neo4j.com.Response)5 TransactionIdStore (org.neo4j.kernel.impl.transaction.log.TransactionIdStore)5 File (java.io.File)4 Test (org.junit.Test)4 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)4 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)4 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)4 RequestContext (org.neo4j.com.RequestContext)3 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)3 NeoStores (org.neo4j.kernel.impl.store.NeoStores)3 LogFile (org.neo4j.kernel.impl.transaction.log.LogFile)3 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)3 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)3 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)3 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)3 StorageEngine (org.neo4j.storageengine.api.StorageEngine)3