Search in sources :

Example 96 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class PropertyKeyTest method lazyLoadWithinWriteTransaction.

@Test
public void lazyLoadWithinWriteTransaction() throws Exception {
    // Given
    FileSystemAbstraction fileSystem = fs.get();
    File dir = new File("dir").getAbsoluteFile();
    BatchInserter inserter = BatchInserters.inserter(dir, fileSystem);
    int count = 3000;
    long nodeId = inserter.createNode(mapWithManyProperties(count));
    inserter.shutdown();
    GraphDatabaseService db = new TestGraphDatabaseFactory().setFileSystem(fileSystem).newImpermanentDatabase(dir);
    // When
    try (Transaction tx = db.beginTx()) {
        db.createNode();
        Node node = db.getNodeById(nodeId);
        // Then
        assertEquals(count, Iterables.count(node.getPropertyKeys()));
        tx.success();
    } finally {
        db.shutdown();
    }
}
Also used : BatchInserter(org.neo4j.unsafe.batchinsert.BatchInserter) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Transaction(org.neo4j.graphdb.Transaction) Node(org.neo4j.graphdb.Node) TestGraphDatabaseFactory(org.neo4j.test.TestGraphDatabaseFactory) File(java.io.File) Test(org.junit.Test)

Example 97 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class InMemoryCountsStoreCountsSnapshotSerializerIntegrationTest method largeWorkloadOnPhysicalLogTest.

@Test
public void largeWorkloadOnPhysicalLogTest() throws IOException {
    //GIVEN
    try (FileSystemAbstraction fs = new DefaultFileSystemAbstraction()) {
        File tempFile = new File(testDir.directory(), "temp");
        StoreChannel rawChannel = fs.create(tempFile);
        Map<CountsKey, long[]> map = CountsStoreMapGenerator.simpleCountStoreMap(100000);
        CountsSnapshot countsSnapshot = new CountsSnapshot(1, map);
        CountsSnapshot recovered;
        //WHEN
        try (FlushableChannel tempChannel = new PhysicalFlushableChannel(rawChannel)) {
            serialize(tempChannel, countsSnapshot);
        }
        // close() here is necessary to flush the temp buffer into the channel so we can read it next
        // The try-with-resources closes the channel, need to reopen
        rawChannel = fs.open(tempFile, "r");
        try (ReadAheadChannel<StoreChannel> readAheadChannel = new ReadAheadChannel<>(rawChannel)) {
            recovered = deserialize(readAheadChannel);
            //THEN
            Assert.assertEquals(countsSnapshot.getTxId(), recovered.getTxId());
            for (Map.Entry<CountsKey, long[]> pair : countsSnapshot.getMap().entrySet()) {
                long[] value = recovered.getMap().get(pair.getKey());
                Assert.assertNotNull(value);
                Assert.assertArrayEquals(value, pair.getValue());
            }
            for (Map.Entry<CountsKey, long[]> pair : recovered.getMap().entrySet()) {
                long[] value = countsSnapshot.getMap().get(pair.getKey());
                Assert.assertNotNull(value);
                Assert.assertArrayEquals(value, pair.getValue());
            }
        }
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) PhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PhysicalFlushableChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) CountsKey(org.neo4j.kernel.impl.store.counts.keys.CountsKey) FlushableChannel(org.neo4j.kernel.impl.transaction.log.FlushableChannel) PhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PhysicalFlushableChannel) File(java.io.File) Map(java.util.Map) ReadAheadChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadChannel) Test(org.junit.Test)

Example 98 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class StoreFactoryTest method setUp.

@Before
public void setUp() throws IOException {
    FileSystemAbstraction fs = fsRule.get();
    pageCache = pageCacheRule.getPageCache(fs);
    idGeneratorFactory = new DefaultIdGeneratorFactory(fs);
    storeDir = directory("dir");
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultIdGeneratorFactory(org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory) Before(org.junit.Before)

Example 99 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class PhysicalLogFileTest method skipLogFileWithoutHeader.

@Test
public void skipLogFileWithoutHeader() throws IOException {
    LifeSupport life = new LifeSupport();
    FileSystemAbstraction fs = fileSystemRule.get();
    PhysicalLogFiles physicalLogFiles = new PhysicalLogFiles(directory.directory(), "logs", fs);
    PhysicalLogFile physicalLogFile = new PhysicalLogFile(fs, physicalLogFiles, 1000, () -> 1L, logVersionRepository, mock(Monitor.class), new LogHeaderCache(10));
    life.add(physicalLogFile);
    life.start();
    // simulate new file without header presence
    PhysicalLogFile logFileToSearchFrom = new PhysicalLogFile(fs, physicalLogFiles, 1000, () -> 10L, logVersionRepository, mock(Monitor.class), new LogHeaderCache(10));
    logVersionRepository.incrementAndGetVersion();
    fs.create(physicalLogFiles.getLogFileForVersion(logVersionRepository.getCurrentLogVersion())).close();
    PhysicalLogicalTransactionStore.LogVersionLocator versionLocator = new PhysicalLogicalTransactionStore.LogVersionLocator(4L);
    logFileToSearchFrom.accept(versionLocator);
    LogPosition logPosition = versionLocator.getLogPosition();
    assertEquals(1, logPosition.getLogVersion());
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Monitor(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.Monitor) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Test(org.junit.Test)

Example 100 with FileSystemAbstraction

use of org.neo4j.io.fs.FileSystemAbstraction in project neo4j by neo4j.

the class ReadAheadChannelTest method shouldThrowExceptionForReadAfterEOFIfNotEnoughBytesExist.

@Test
public void shouldThrowExceptionForReadAfterEOFIfNotEnoughBytesExist() throws Exception {
    // Given
    FileSystemAbstraction fileSystem = fileSystemRule.get();
    StoreChannel storeChannel = fileSystem.open(new File("foo.txt"), "rw");
    ByteBuffer buffer = ByteBuffer.allocate(1);
    buffer.put((byte) 1);
    buffer.flip();
    storeChannel.writeAll(buffer);
    storeChannel.force(false);
    storeChannel.close();
    storeChannel = fileSystem.open(new File("foo.txt"), "r");
    ReadAheadChannel<StoreChannel> channel = new ReadAheadChannel<>(storeChannel);
    assertEquals((byte) 1, channel.get());
    try {
        channel.get();
        fail("Should have thrown exception signalling end of file reached");
    } catch (ReadPastEndException endOfFile) {
    // outstanding
    }
    try {
        channel.get();
        fail("Should have thrown exception signalling end of file reached");
    } catch (ReadPastEndException endOfFile) {
    // outstanding
    }
}
Also used : EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) StoreChannel(org.neo4j.io.fs.StoreChannel) File(java.io.File) ByteBuffer(java.nio.ByteBuffer) ReadPastEndException(org.neo4j.storageengine.api.ReadPastEndException) Test(org.junit.Test)

Aggregations

FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)125 File (java.io.File)88 Test (org.junit.Test)82 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)34 IOException (java.io.IOException)28 Config (org.neo4j.kernel.configuration.Config)23 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)22 PageCache (org.neo4j.io.pagecache.PageCache)22 DelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.DelegatingFileSystemAbstraction)20 ByteBuffer (java.nio.ByteBuffer)13 StoreChannel (org.neo4j.io.fs.StoreChannel)11 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)10 UncloseableDelegatingFileSystemAbstraction (org.neo4j.graphdb.mockfs.UncloseableDelegatingFileSystemAbstraction)9 DefaultIdGeneratorFactory (org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory)9 OutputStream (java.io.OutputStream)8 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)8 DelegatingStoreChannel (org.neo4j.graphdb.mockfs.DelegatingStoreChannel)8 Map (java.util.Map)7 Matchers.containsString (org.hamcrest.Matchers.containsString)7 AdversarialPagedFile (org.neo4j.adversaries.pagecache.AdversarialPagedFile)7