Search in sources :

Example 16 with ReadableClosablePositionAwareChannel

use of org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel in project neo4j by neo4j.

the class MasterServerTest method shouldCleanExistentLockSessionOnFinishOffChannel.

@Test
public void shouldCleanExistentLockSessionOnFinishOffChannel() throws Exception {
    Master master = mock(Master.class);
    ConversationManager conversationManager = mock(ConversationManager.class);
    LogEntryReader<ReadableClosablePositionAwareChannel> logEntryReader = new VersionAwareLogEntryReader<>();
    MasterServer masterServer = new MasterServer(master, mock(LogProvider.class), mock(Server.Configuration.class), mock(TxChecksumVerifier.class), mock(ByteCounterMonitor.class), mock(RequestMonitor.class), conversationManager, logEntryReader);
    RequestContext requestContext = new RequestContext(1L, 1, 1, 0, 0L);
    masterServer.stopConversation(requestContext);
    Mockito.verify(conversationManager).stop(requestContext);
}
Also used : LogProvider(org.neo4j.logging.LogProvider) ByteCounterMonitor(org.neo4j.kernel.monitoring.ByteCounterMonitor) TxChecksumVerifier(org.neo4j.com.TxChecksumVerifier) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) RequestContext(org.neo4j.com.RequestContext) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) RequestMonitor(org.neo4j.com.monitor.RequestMonitor) Test(org.junit.Test)

Example 17 with ReadableClosablePositionAwareChannel

use of org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel in project neo4j by neo4j.

the class MasterClientResolverTest method shouldResolveMasterClientFactory.

@Test
public void shouldResolveMasterClientFactory() throws Exception {
    // Given
    LogEntryReader<ReadableClosablePositionAwareChannel> logEntryReader = new VersionAwareLogEntryReader<>();
    MasterClientResolver resolver = new MasterClientResolver(NullLogProvider.getInstance(), ResponseUnpacker.NO_OP_RESPONSE_UNPACKER, mock(InvalidEpochExceptionHandler.class), 1, 1, 1, 1024, Suppliers.singleton(logEntryReader));
    LifeSupport life = new LifeSupport();
    try {
        life.start();
        MasterClient masterClient1 = resolver.instantiate("cluster://localhost", 44, null, new Monitors(), StoreId.DEFAULT, life);
        assertThat(masterClient1, instanceOf(MasterClient320.class));
    } finally {
        life.shutdown();
    }
    IllegalProtocolVersionException illegalProtocolVersionException = new IllegalProtocolVersionException(MasterClient214.PROTOCOL_VERSION.getApplicationProtocol(), MasterClient310.PROTOCOL_VERSION.getApplicationProtocol(), "Protocol is too modern");
    // When
    resolver.handle(illegalProtocolVersionException);
    // Then
    life = new LifeSupport();
    try {
        life.start();
        MasterClient masterClient2 = resolver.instantiate("cluster://localhost", 55, null, new Monitors(), StoreId.DEFAULT, life);
        assertThat(masterClient2, instanceOf(MasterClient214.class));
    } finally {
        life.shutdown();
    }
}
Also used : MasterClient320(org.neo4j.kernel.ha.MasterClient320) Monitors(org.neo4j.kernel.monitoring.Monitors) IllegalProtocolVersionException(org.neo4j.com.IllegalProtocolVersionException) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) MasterClient214(org.neo4j.kernel.ha.MasterClient214) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) Test(org.junit.Test)

Example 18 with ReadableClosablePositionAwareChannel

use of org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel in project neo4j by neo4j.

the class RecoveryRequiredChecker method isRecoveryRequiredAt.

public boolean isRecoveryRequiredAt(File dataDir) throws IOException {
    File neoStore = new File(dataDir, MetaDataStore.DEFAULT_NAME);
    boolean noStoreFound = !NeoStores.isStorePresent(pageCache, dataDir);
    // We need config to determine where the logical log files are
    if (noStoreFound) {
        // No database in the specified directory.
        return false;
    }
    long logVersion = MetaDataStore.getRecord(pageCache, neoStore, MetaDataStore.Position.LOG_VERSION);
    PhysicalLogFiles logFiles = new PhysicalLogFiles(dataDir, fs);
    LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
    LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fs, reader);
    return new PositionToRecoverFrom(finder, NO_MONITOR).apply(logVersion) != LogPosition.UNSPECIFIED;
}
Also used : LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) PositionToRecoverFrom(org.neo4j.kernel.recovery.PositionToRecoverFrom) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) File(java.io.File) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)

Example 19 with ReadableClosablePositionAwareChannel

use of org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel in project neo4j by neo4j.

the class InputEntityReader method readGroup.

protected Group readGroup(int slot, ProcessorState state) throws IOException {
    ReadableClosablePositionAwareChannel channel = state.batchChannel;
    byte groupMode = channel.get();
    switch(groupMode) {
        case SAME_GROUP:
            return state.previousGroups[slot];
        case NEW_GROUP:
            return state.previousGroups[slot] = new Group.Adapter(channel.getInt(), (String) readToken(GROUP_TOKEN, channel));
        default:
            throw new IllegalArgumentException("Unknown group mode " + groupMode);
    }
}
Also used : ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)

Example 20 with ReadableClosablePositionAwareChannel

use of org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel in project neo4j by neo4j.

the class RecoveryTest method shouldRecoverExistingData.

@Test
public void shouldRecoverExistingData() throws Exception {
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            LogPositionMarker marker = new LogPositionMarker();
            // last committed tx
            consumer.accept(marker);
            LogPosition lastCommittedTxPosition = marker.newPosition();
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            lastCommittedTxStartEntry = new LogEntryStart(0, 1, 2L, 3L, new byte[0], lastCommittedTxPosition);
            writer.writeCommitEntry(4L, 5L);
            lastCommittedTxCommitEntry = new OnePhaseCommit(4L, 5L);
            // check point pointing to the previously committed transaction
            writer.writeCheckPointEntry(lastCommittedTxPosition);
            expectedCheckPointEntry = new CheckPoint(lastCommittedTxPosition);
            // tx committed after checkpoint
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 6L, 4L, new byte[0]);
            expectedStartEntry = new LogEntryStart(0, 1, 6L, 4L, new byte[0], marker.newPosition());
            writer.writeCommitEntry(5L, 7L);
            expectedCommitEntry = new OnePhaseCommit(5L, 7L);
            return true;
        }
    });
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    final AtomicBoolean recoveryRequired = new AtomicBoolean();
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            private int nr = 0;

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                recoveryRequired.set(true);
                final Visitor<CommittedTransactionRepresentation, Exception> actual = super.startRecovery();
                return new Visitor<CommittedTransactionRepresentation, Exception>() {

                    @Override
                    public boolean visit(CommittedTransactionRepresentation tx) throws Exception {
                        actual.visit(tx);
                        switch(nr++) {
                            case 0:
                                assertEquals(lastCommittedTxStartEntry, tx.getStartEntry());
                                assertEquals(lastCommittedTxCommitEntry, tx.getCommitEntry());
                                break;
                            case 1:
                                assertEquals(expectedStartEntry, tx.getStartEntry());
                                assertEquals(expectedCommitEntry, tx.getCommitEntry());
                                break;
                            default:
                                fail("Too many recovered transactions");
                        }
                        return false;
                    }
                };
            }
        }, monitor));
        life.start();
        InOrder order = inOrder(monitor);
        order.verify(monitor, times(1)).recoveryRequired(any(LogPosition.class));
        order.verify(monitor, times(1)).recoveryCompleted(2);
        assertTrue(recoveryRequired.get());
    } finally {
        life.shutdown();
    }
}
Also used : DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) Visitor(org.neo4j.helpers.collection.Visitor) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Consumer(java.util.function.Consumer) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) Pair(org.neo4j.helpers.collection.Pair) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) InOrder(org.mockito.InOrder) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) IOException(java.io.IOException) IOException(java.io.IOException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Test(org.junit.Test)

Aggregations

ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)24 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)19 IOException (java.io.IOException)9 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)8 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)8 LatestCheckPointFinder (org.neo4j.kernel.recovery.LatestCheckPointFinder)8 File (java.io.File)7 Test (org.junit.Test)7 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)4 LinkedList (java.util.LinkedList)3 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)3 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)3 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)3 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)3 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)3 LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)3 DefaultRecoverySPI (org.neo4j.kernel.recovery.DefaultRecoverySPI)3 Recovery (org.neo4j.kernel.recovery.Recovery)3 ByteBuffer (java.nio.ByteBuffer)2