Examples with MemoryAllocator org.neo4j.io.mem.MemoryAllocator used on opensource projects

Example 1 with MemoryAllocator

use of org.neo4j.io.mem.MemoryAllocator in project neo4j by neo4j.

the class LargePageListIT method veryLargePageListsMustBeFullyAccessible.

@Test
void veryLargePageListsMustBeFullyAccessible() {
    // We need roughly 2 GiBs of memory for the meta-data here, which is why this is an IT and not a Test.
    // We add one extra page worth of data to the size here, to avoid ending up on a "convenient" boundary.
    int pageSize = (int) ByteUnit.kibiBytes(8);
    long pageCacheSize = ByteUnit.gibiBytes(513) + pageSize;
    int pages = Math.toIntExact(pageCacheSize / pageSize);
    MemoryAllocator mman = MemoryAllocator.createAllocator(GibiByte.toBytes(2), EmptyMemoryTracker.INSTANCE);
    SwapperSet swappers = new SwapperSet();
    long victimPage = VictimPageReference.getVictimPage(pageSize, INSTANCE);
    PageList pageList = new PageList(pages, pageSize, mman, swappers, victimPage, Long.BYTES);
    // Verify we end up with the correct number of pages.
    assertThat(pageList.getPageCount()).isEqualTo(pages);
    // Spot-check the accessibility in the bulk of the pages.
    IntStream.range(0, pages / 32).parallel().forEach(id -> verifyPageMetaDataIsAccessible(pageList, id * 32));
    // Thoroughly check the accessibility around the tail end of the page list.
    IntStream.range(pages - 2000, pages).parallel().forEach(id -> verifyPageMetaDataIsAccessible(pageList, id));
}

Example 2 with MemoryAllocator

use of org.neo4j.io.mem.MemoryAllocator in project neo4j by neo4j.

the class BatchingNeoStores method createPageCache.

private static PageCache createPageCache(FileSystemAbstraction fileSystem, Config config, PageCacheTracer tracer, JobScheduler jobScheduler, MemoryTracker memoryTracker) {
    SingleFilePageSwapperFactory swapperFactory = new SingleFilePageSwapperFactory(fileSystem);
    MemoryAllocator memoryAllocator = createAllocator(ByteUnit.parse(config.get(pagecache_memory)), memoryTracker);
    MuninnPageCache.Configuration configuration = MuninnPageCache.config(memoryAllocator).pageCacheTracer(tracer).memoryTracker(memoryTracker).bufferFactory(new ConfigurableIOBufferFactory(config, memoryTracker)).faultLockStriping(1 << 11).disableEvictionThread();
    return new MuninnPageCache(swapperFactory, jobScheduler, configuration);
}

Example 3 with MemoryAllocator

use of org.neo4j.io.mem.MemoryAllocator in project neo4j by neo4j.

the class ConfiguringPageCacheFactory method createPageCache.

protected PageCache createPageCache() {
    long pageCacheMaxMemory = getPageCacheMaxMemory(config);
    var memoryPool = memoryPools.pool(PAGE_CACHE, pageCacheMaxMemory, false, null);
    var memoryTracker = memoryPool.getPoolMemoryTracker();
    MemoryAllocator memoryAllocator = buildMemoryAllocator(pageCacheMaxMemory, memoryTracker);
    var bufferFactory = new ConfigurableIOBufferFactory(config, memoryTracker);
    MuninnPageCache.Configuration configuration = MuninnPageCache.config(memoryAllocator).memoryTracker(memoryTracker).bufferFactory(bufferFactory).preallocateStoreFiles(config.get(preallocate_store_files)).clock(clock).pageCacheTracer(pageCacheTracer);
    return new MuninnPageCache(swapperFactory, scheduler, configuration);
}

Example 4 with MemoryAllocator

use of org.neo4j.io.mem.MemoryAllocator in project neo4j by neo4j.

the class PageCacheRule method getPageCache.

/**
 * Opens a new {@link PageCache} with the provided file system and config.
 *
 * @param factory {@link PageSwapperFactory} to use for the {@link PageCache}.
 * @param overriddenConfig specific {@link PageCacheConfig} overriding config provided in {@link PageCacheRule}
 * constructor, if any.
 * @return the opened {@link PageCache}.
 */
public PageCache getPageCache(PageSwapperFactory factory, PageCacheConfig overriddenConfig) {
    closeExistingPageCache();
    var memoryTracker = new LocalMemoryTracker();
    MemoryAllocator mman = MemoryAllocator.createAllocator(parse(selectConfig(baseConfig.memory, overriddenConfig.memory, "8 MiB")), memoryTracker);
    if (clock == null) {
        clock = Clocks.nanoClock();
    }
    MuninnPageCache.Configuration configuration = MuninnPageCache.config(mman).memoryTracker(memoryTracker).clock(clock);
    Integer pageSize = selectConfig(baseConfig.pageSize, overriddenConfig.pageSize, null);
    configuration = pageSize == null ? configuration : configuration.pageSize(pageSize);
    PageCacheTracer cacheTracer = selectConfig(baseConfig.tracer, overriddenConfig.tracer, PageCacheTracer.NULL);
    configuration = configuration.pageCacheTracer(cacheTracer);
    initializeJobScheduler();
    pageCache = new MuninnPageCache(factory, jobScheduler, configuration);
    pageCachePostConstruct(overriddenConfig);
    return pageCache;
}

Example 5 with MemoryAllocator

use of org.neo4j.io.mem.MemoryAllocator in project neo4j by neo4j.

the class StandalonePageCacheFactory method createPageCache.

private static PageCache createPageCache(PageSwapperFactory factory, JobScheduler jobScheduler, PageCacheTracer cacheTracer, int pageSize) {
    long expectedMemory = Math.max(MebiByte.toBytes(8), 10 * pageSize);
    MemoryAllocator memoryAllocator = MemoryAllocator.createAllocator(expectedMemory, EmptyMemoryTracker.INSTANCE);
    return new MuninnPageCache(factory, jobScheduler, config(memoryAllocator).pageCacheTracer(cacheTracer).pageSize(pageSize));
}