Search in sources :

Example 6 with SimpleTriggerInfo

use of org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo in project neo4j by neo4j.

the class TestRecoveryMultipleDataSources method main.

public static void main(String[] args) throws IOException {
    if (args.length != 1) {
        exit(1);
    }
    File storeDir = new File(args[0]);
    GraphDatabaseService db = new TestGraphDatabaseFactory().newEmbeddedDatabase(storeDir);
    try (Transaction tx = db.beginTx()) {
        db.createNode().createRelationshipTo(db.createNode(), MyRelTypes.TEST);
        tx.success();
    }
    ((GraphDatabaseAPI) db).getDependencyResolver().resolveDependency(CheckPointer.class).forceCheckPoint(new SimpleTriggerInfo("test"));
    try (Transaction tx = db.beginTx()) {
        db.index().forNodes("index").add(db.createNode(), storeDir.getAbsolutePath(), db.createNode());
        tx.success();
    }
    exit(0);
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) Transaction(org.neo4j.graphdb.Transaction) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) File(java.io.File)

Example 7 with SimpleTriggerInfo

use of org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo in project neo4j by neo4j.

the class StoreCopyServer method flushStoresAndStreamStoreFiles.

/**
     * Trigger store flush (checkpoint) and write {@link NeoStoreDataSource#listStoreFiles(boolean) store files} to the
     * given {@link StoreWriter}.
     *
     * @param triggerName name of the component asks for store files.
     * @param writer store writer to write files to.
     * @param includeLogs <code>true</code> if transaction logs should be copied, <code>false</code> otherwise.
     * @return a {@link RequestContext} specifying at which point the store copy started.
     */
public RequestContext flushStoresAndStreamStoreFiles(String triggerName, StoreWriter writer, boolean includeLogs) {
    try {
        ThrowingAction<IOException> checkPointAction = () -> {
            monitor.startTryCheckPoint();
            checkPointer.tryCheckPoint(new SimpleTriggerInfo(triggerName));
            monitor.finishTryCheckPoint();
        };
        // Copy the store files
        long lastAppliedTransaction;
        try (Resource lock = mutex.storeCopy(checkPointAction);
            ResourceIterator<StoreFileMetadata> files = dataSource.listStoreFiles(includeLogs)) {
            lastAppliedTransaction = checkPointer.lastCheckPointedTransactionId();
            monitor.startStreamingStoreFiles();
            ByteBuffer temporaryBuffer = ByteBuffer.allocateDirect((int) ByteUnit.mebiBytes(1));
            while (files.hasNext()) {
                StoreFileMetadata meta = files.next();
                File file = meta.file();
                int recordSize = meta.recordSize();
                // Read from paged file if mapping exists. Otherwise read through file system.
                // A file is mapped if it is a store, and we have a running database, which will be the case for
                // both online backup, and when we are the master of an HA cluster.
                final Optional<PagedFile> optionalPagedFile = pageCache.getExistingMapping(file);
                if (optionalPagedFile.isPresent()) {
                    try (PagedFile pagedFile = optionalPagedFile.get()) {
                        long fileSize = pagedFile.fileSize();
                        try (ReadableByteChannel fileChannel = pagedFile.openReadableByteChannel()) {
                            doWrite(writer, temporaryBuffer, file, recordSize, fileChannel, fileSize);
                        }
                    }
                } else {
                    try (ReadableByteChannel fileChannel = fileSystem.open(file, "r")) {
                        long fileSize = fileSystem.getFileSize(file);
                        doWrite(writer, temporaryBuffer, file, recordSize, fileChannel, fileSize);
                    }
                }
            }
        } finally {
            monitor.finishStreamingStoreFiles();
        }
        return anonymous(lastAppliedTransaction);
    } catch (IOException e) {
        throw new ServerFailureException(e);
    }
}
Also used : ReadableByteChannel(java.nio.channels.ReadableByteChannel) ServerFailureException(org.neo4j.com.ServerFailureException) PagedFile(org.neo4j.io.pagecache.PagedFile) Resource(org.neo4j.graphdb.Resource) IOException(java.io.IOException) StoreFileMetadata(org.neo4j.storageengine.api.StoreFileMetadata) ByteBuffer(java.nio.ByteBuffer) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) PagedFile(org.neo4j.io.pagecache.PagedFile) FileUtils.getMostCanonicalFile(org.neo4j.io.fs.FileUtils.getMostCanonicalFile) File(java.io.File)

Example 8 with SimpleTriggerInfo

use of org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo in project neo4j by neo4j.

the class DefaultMasterImplSPITest method flushStoreFilesWithCorrectCheckpointTriggerName.

@Test
public void flushStoreFilesWithCorrectCheckpointTriggerName() throws IOException {
    CheckPointer checkPointer = mock(CheckPointer.class);
    NeoStoreDataSource dataSource = mock(NeoStoreDataSource.class);
    when(dataSource.listStoreFiles(anyBoolean())).thenReturn(Iterators.emptyIterator());
    DefaultMasterImplSPI master = new DefaultMasterImplSPI(mock(GraphDatabaseAPI.class, RETURNS_MOCKS), mock(FileSystemAbstraction.class), new Monitors(), mock(LabelTokenHolder.class), mock(PropertyKeyTokenHolder.class), mock(RelationshipTypeTokenHolder.class), mock(IdGeneratorFactory.class), mock(TransactionCommitProcess.class), checkPointer, mock(TransactionIdStore.class), mock(LogicalTransactionStore.class), dataSource, mock(PageCache.class), new StoreCopyCheckPointMutex());
    master.flushStoresAndStreamStoreFiles(mock(StoreWriter.class));
    TriggerInfo expectedTriggerInfo = new SimpleTriggerInfo(DefaultMasterImplSPI.STORE_COPY_CHECKPOINT_TRIGGER);
    verify(checkPointer).tryCheckPoint(expectedTriggerInfo);
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) RelationshipTypeTokenHolder(org.neo4j.kernel.impl.core.RelationshipTypeTokenHolder) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) TransactionCommitProcess(org.neo4j.kernel.impl.api.TransactionCommitProcess) TriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.TriggerInfo) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) IdGeneratorFactory(org.neo4j.kernel.impl.store.id.IdGeneratorFactory) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) PropertyKeyTokenHolder(org.neo4j.kernel.impl.core.PropertyKeyTokenHolder) StoreWriter(org.neo4j.com.storecopy.StoreWriter) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) Monitors(org.neo4j.kernel.monitoring.Monitors) StoreCopyCheckPointMutex(org.neo4j.kernel.impl.transaction.log.checkpoint.StoreCopyCheckPointMutex) LabelTokenHolder(org.neo4j.kernel.impl.core.LabelTokenHolder) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 9 with SimpleTriggerInfo

use of org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo in project neo4j by neo4j.

the class ReadReplicaStoreCopyIT method forceLogRotationAndPruning.

private static void forceLogRotationAndPruning(CoreClusterMember core) {
    try {
        DependencyResolver dependencyResolver = core.database().getDependencyResolver();
        dependencyResolver.resolveDependency(LogRotation.class).rotateLogFile();
        SimpleTriggerInfo info = new SimpleTriggerInfo("test");
        dependencyResolver.resolveDependency(CheckPointer.class).forceCheckPoint(info);
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    }
}
Also used : SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) UncheckedIOException(java.io.UncheckedIOException) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) DependencyResolver(org.neo4j.graphdb.DependencyResolver)

Example 10 with SimpleTriggerInfo

use of org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo in project neo4j by neo4j.

the class MetricsKernelExtensionFactoryIT method shouldUseEventBasedReportingCorrectly.

@Test
public void shouldUseEventBasedReportingCorrectly() throws Throwable {
    // GIVEN
    addNodes(100);
    // WHEN
    CheckPointer checkPointer = db.getDependencyResolver().resolveDependency(CheckPointer.class);
    checkPointer.checkPointIfNeeded(new SimpleTriggerInfo("test"));
    // wait for the file to be written before shutting down the cluster
    File metricFile = metricsCsv(outputPath, CheckPointingMetrics.CHECK_POINT_DURATION);
    long result = readLongValueAndAssert(metricFile, (newValue, currentValue) -> newValue > 0);
    // THEN
    assertThat(result, greaterThanOrEqualTo(0L));
}
Also used : SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) File(java.io.File) Test(org.junit.Test)

Aggregations

SimpleTriggerInfo (org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo)15 CheckPointer (org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer)14 LogRotation (org.neo4j.kernel.impl.transaction.log.rotation.LogRotation)7 File (java.io.File)4 Test (org.junit.Test)4 Transaction (org.neo4j.graphdb.Transaction)4 GraphDatabaseAPI (org.neo4j.kernel.internal.GraphDatabaseAPI)3 IOException (java.io.IOException)2 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 Node (org.neo4j.graphdb.Node)2 Resource (org.neo4j.graphdb.Resource)2 PagedFile (org.neo4j.io.pagecache.PagedFile)2 TriggerInfo (org.neo4j.kernel.impl.transaction.log.checkpoint.TriggerInfo)2 StoreFileMetadata (org.neo4j.storageengine.api.StoreFileMetadata)2 TestGraphDatabaseFactory (org.neo4j.test.TestGraphDatabaseFactory)2 UncheckedIOException (java.io.UncheckedIOException)1 ByteBuffer (java.nio.ByteBuffer)1 ReadableByteChannel (java.nio.channels.ReadableByteChannel)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1