Search in sources :

Example 6 with Resource

use of org.neo4j.graphdb.Resource in project neo4j by neo4j.

the class StoreCopyCheckPointMutexTest method storeCopyShouldBlockCheckPoint.

@Test
public void storeCopyShouldBlockCheckPoint() throws Exception {
    // GIVEN
    try (Resource lock = mutex.storeCopy(NO_OP)) {
        // WHEN
        t2.execute(state -> mutex.checkPoint());
        // THEN
        t2.get().waitUntilWaiting(details -> details.isAt(StoreCopyCheckPointMutex.class, "checkPoint"));
    }
}
Also used : Resource(org.neo4j.graphdb.Resource) Test(org.junit.Test)

Example 7 with Resource

use of org.neo4j.graphdb.Resource in project neo4j by neo4j.

the class StoreCopyServer method flushStoresAndStreamStoreFiles.

/**
     * Trigger store flush (checkpoint) and write {@link NeoStoreDataSource#listStoreFiles(boolean) store files} to the
     * given {@link StoreWriter}.
     *
     * @param triggerName name of the component asks for store files.
     * @param writer store writer to write files to.
     * @param includeLogs <code>true</code> if transaction logs should be copied, <code>false</code> otherwise.
     * @return a {@link RequestContext} specifying at which point the store copy started.
     */
public RequestContext flushStoresAndStreamStoreFiles(String triggerName, StoreWriter writer, boolean includeLogs) {
    try {
        ThrowingAction<IOException> checkPointAction = () -> {
            monitor.startTryCheckPoint();
            checkPointer.tryCheckPoint(new SimpleTriggerInfo(triggerName));
            monitor.finishTryCheckPoint();
        };
        // Copy the store files
        long lastAppliedTransaction;
        try (Resource lock = mutex.storeCopy(checkPointAction);
            ResourceIterator<StoreFileMetadata> files = dataSource.listStoreFiles(includeLogs)) {
            lastAppliedTransaction = checkPointer.lastCheckPointedTransactionId();
            monitor.startStreamingStoreFiles();
            ByteBuffer temporaryBuffer = ByteBuffer.allocateDirect((int) ByteUnit.mebiBytes(1));
            while (files.hasNext()) {
                StoreFileMetadata meta = files.next();
                File file = meta.file();
                int recordSize = meta.recordSize();
                // Read from paged file if mapping exists. Otherwise read through file system.
                // A file is mapped if it is a store, and we have a running database, which will be the case for
                // both online backup, and when we are the master of an HA cluster.
                final Optional<PagedFile> optionalPagedFile = pageCache.getExistingMapping(file);
                if (optionalPagedFile.isPresent()) {
                    try (PagedFile pagedFile = optionalPagedFile.get()) {
                        long fileSize = pagedFile.fileSize();
                        try (ReadableByteChannel fileChannel = pagedFile.openReadableByteChannel()) {
                            doWrite(writer, temporaryBuffer, file, recordSize, fileChannel, fileSize);
                        }
                    }
                } else {
                    try (ReadableByteChannel fileChannel = fileSystem.open(file, "r")) {
                        long fileSize = fileSystem.getFileSize(file);
                        doWrite(writer, temporaryBuffer, file, recordSize, fileChannel, fileSize);
                    }
                }
            }
        } finally {
            monitor.finishStreamingStoreFiles();
        }
        return anonymous(lastAppliedTransaction);
    } catch (IOException e) {
        throw new ServerFailureException(e);
    }
}
Also used : ReadableByteChannel(java.nio.channels.ReadableByteChannel) ServerFailureException(org.neo4j.com.ServerFailureException) PagedFile(org.neo4j.io.pagecache.PagedFile) Resource(org.neo4j.graphdb.Resource) IOException(java.io.IOException) StoreFileMetadata(org.neo4j.storageengine.api.StoreFileMetadata) ByteBuffer(java.nio.ByteBuffer) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) PagedFile(org.neo4j.io.pagecache.PagedFile) FileUtils.getMostCanonicalFile(org.neo4j.io.fs.FileUtils.getMostCanonicalFile) File(java.io.File)

Example 8 with Resource

use of org.neo4j.graphdb.Resource in project neo4j by neo4j.

the class NeoStoreFileListing method listStoreFiles.

public ResourceIterator<StoreFileMetadata> listStoreFiles(boolean includeLogs) throws IOException {
    Collection<StoreFileMetadata> files = new ArrayList<>();
    gatherNonRecordStores(files, includeLogs);
    gatherNeoStoreFiles(files);
    Resource labelScanStoreSnapshot = gatherLabelScanStoreFiles(files);
    Resource schemaIndexSnapshots = gatherSchemaIndexFiles(files);
    Resource legacyIndexSnapshots = gatherLegacyIndexFiles(files);
    return resourceIterator(files.iterator(), new MultiResource(asList(labelScanStoreSnapshot, schemaIndexSnapshots, legacyIndexSnapshots)));
}
Also used : ArrayList(java.util.ArrayList) Resource(org.neo4j.graphdb.Resource) StoreFileMetadata(org.neo4j.storageengine.api.StoreFileMetadata)

Example 9 with Resource

use of org.neo4j.graphdb.Resource in project neo4j by neo4j.

the class MonoDirectionalTraversalDescription method traverse.

@Override
public Traverser traverse(final Iterable<Node> iterableStartNodes) {
    return new DefaultTraverser(() -> {
        Resource statement = statementSupplier.get();
        MonoDirectionalTraverserIterator iterator = new MonoDirectionalTraverserIterator(statement, uniqueness.create(uniquenessParameter), expander, branchOrdering, evaluator, iterableStartNodes, initialState, uniqueness);
        return sorting != null ? new SortingTraverserIterator(statement, sorting, iterator) : iterator;
    });
}
Also used : Resource(org.neo4j.graphdb.Resource)

Example 10 with Resource

use of org.neo4j.graphdb.Resource in project neo4j by neo4j.

the class TestCommonIterators method iteratorsStreamClosesResourceIterator.

@Test
public void iteratorsStreamClosesResourceIterator() {
    List<Object> list = Arrays.asList("a", "b", "c", "def");
    Resource resource = mock(Resource.class);
    ResourceIterator<Object> iterator = Iterators.resourceIterator(list.iterator(), resource);
    try (Stream<Object> stream = Iterators.stream(iterator)) {
        assertEquals(list, stream.collect(toList()));
    }
    verify(resource).close();
}
Also used : Resource(org.neo4j.graphdb.Resource) Test(org.junit.Test)

Aggregations

Resource (org.neo4j.graphdb.Resource)12 Test (org.junit.Test)8 StoreFileMetadata (org.neo4j.storageengine.api.StoreFileMetadata)3 File (java.io.File)2 IOException (java.io.IOException)2 PagedFile (org.neo4j.io.pagecache.PagedFile)2 SimpleTriggerInfo (org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo)2 Race (org.neo4j.test.Race)2 ByteBuffer (java.nio.ByteBuffer)1 ReadableByteChannel (java.nio.channels.ReadableByteChannel)1 ArrayList (java.util.ArrayList)1 ExecutionException (java.util.concurrent.ExecutionException)1 Future (java.util.concurrent.Future)1 ThreadLocalRandom (java.util.concurrent.ThreadLocalRandom)1 MILLISECONDS (java.util.concurrent.TimeUnit.MILLISECONDS)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 AtomicReference (java.util.concurrent.atomic.AtomicReference)1 LockSupport (java.util.concurrent.locks.LockSupport)1 Matchers.containsString (org.hamcrest.Matchers.containsString)1 Matchers.lessThan (org.hamcrest.Matchers.lessThan)1