Search in sources :

Example 6 with PageMemory

use of org.apache.ignite.internal.pagememory.PageMemory in project ignite-3 by apache.

the class PageMemoryNoLoadSelfTest method testPageHandleDeallocation.

@Test
public void testPageHandleDeallocation() throws Exception {
    PageMemory mem = memory();
    mem.start();
    try {
        int pages = 3 * 1024 * 1024 / (8 * 1024);
        Collection<FullPageId> handles = new HashSet<>();
        for (int i = 0; i < pages; i++) {
            handles.add(allocatePage(mem));
        }
        for (FullPageId fullId : handles) {
            mem.freePage(fullId.groupId(), fullId.pageId());
        }
        for (int i = 0; i < pages; i++) {
            assertFalse(handles.add(allocatePage(mem)));
        }
    } finally {
        mem.stop(true);
    }
}
Also used : PageMemory(org.apache.ignite.internal.pagememory.PageMemory) FullPageId(org.apache.ignite.internal.pagememory.FullPageId) HashSet(java.util.HashSet) BaseIgniteAbstractTest(org.apache.ignite.internal.testframework.BaseIgniteAbstractTest) Test(org.junit.jupiter.api.Test)

Example 7 with PageMemory

use of org.apache.ignite.internal.pagememory.PageMemory in project ignite-3 by apache.

the class PageMemoryNoLoadSelfTest method testPageIdRotation.

@Test
public void testPageIdRotation() throws Exception {
    PageMemory mem = memory();
    mem.start();
    try {
        int pages = 5;
        Collection<FullPageId> old = new ArrayList<>();
        Collection<FullPageId> updated = new ArrayList<>();
        for (int i = 0; i < pages; i++) {
            old.add(allocatePage(mem));
        }
        // Check that initial pages are accessible.
        for (FullPageId id : old) {
            long pageApsPtr = mem.acquirePage(id.groupId(), id.pageId());
            try {
                long pageAddr = mem.writeLock(id.groupId(), id.pageId(), pageApsPtr);
                assertNotNull(pageAddr);
                try {
                    PAGE_IO.initNewPage(pageAddr, id.pageId(), mem.realPageSize(id.groupId()));
                    long updId = PageIdUtils.rotatePageId(id.pageId());
                    PageIo.setPageId(pageAddr, updId);
                    updated.add(new FullPageId(updId, id.groupId()));
                } finally {
                    mem.writeUnlock(id.groupId(), id.pageId(), pageApsPtr, true);
                }
            } finally {
                mem.releasePage(id.groupId(), id.pageId(), pageApsPtr);
            }
        }
        // Check that updated pages are inaccessible using old IDs.
        for (FullPageId id : old) {
            long pageApsPtr = mem.acquirePage(id.groupId(), id.pageId());
            try {
                long pageAddr = mem.writeLock(id.groupId(), id.pageId(), pageApsPtr);
                if (pageAddr != 0L) {
                    mem.writeUnlock(id.groupId(), id.pageId(), pageApsPtr, false);
                    fail("Was able to acquire page write lock.");
                }
                mem.readLock(id.groupId(), id.pageId(), pageApsPtr);
                if (pageAddr != 0) {
                    mem.readUnlock(id.groupId(), id.pageId(), pageApsPtr);
                    fail("Was able to acquire page read lock.");
                }
            } finally {
                mem.releasePage(id.groupId(), id.pageId(), pageApsPtr);
            }
        }
        // Check that updated pages are accessible using new IDs.
        for (FullPageId id : updated) {
            long pageApsPtr = mem.acquirePage(id.groupId(), id.pageId());
            try {
                long pageAddr = mem.writeLock(id.groupId(), id.pageId(), pageApsPtr);
                assertNotSame(0L, pageAddr);
                try {
                    assertEquals(id.pageId(), PageIo.getPageId(pageAddr));
                } finally {
                    mem.writeUnlock(id.groupId(), id.pageId(), pageApsPtr, false);
                }
                pageAddr = mem.readLock(id.groupId(), id.pageId(), pageApsPtr);
                assertNotSame(0L, pageAddr);
                try {
                    assertEquals(id.pageId(), PageIo.getPageId(pageAddr));
                } finally {
                    mem.readUnlock(id.groupId(), id.pageId(), pageApsPtr);
                }
            } finally {
                mem.releasePage(id.groupId(), id.pageId(), pageApsPtr);
            }
        }
    } finally {
        mem.stop(true);
    }
}
Also used : PageMemory(org.apache.ignite.internal.pagememory.PageMemory) ArrayList(java.util.ArrayList) FullPageId(org.apache.ignite.internal.pagememory.FullPageId) BaseIgniteAbstractTest(org.apache.ignite.internal.testframework.BaseIgniteAbstractTest) Test(org.junit.jupiter.api.Test)

Example 8 with PageMemory

use of org.apache.ignite.internal.pagememory.PageMemory in project ignite-3 by apache.

the class PageMemoryImplNoLoadTest method memory.

/**
 * {@inheritDoc}
 */
@Override
protected PageMemory memory() throws Exception {
    dataRegionCfg.change(cfg -> cfg.convert(PageMemoryDataRegionChange.class).changePageSize(PAGE_SIZE).changeInitSize(MAX_MEMORY_SIZE).changeMaxSize(MAX_MEMORY_SIZE)).get(1, SECONDS);
    PageIoRegistry ioRegistry = new PageIoRegistry();
    ioRegistry.loadFromServiceLoader();
    return new PageMemoryImpl(new UnsafeMemoryProvider(null), (PageMemoryDataRegionConfiguration) fixConfiguration(dataRegionCfg), ioRegistry, LongStream.range(0, 10).map(i -> 5 * MiB).toArray(), new TestPageReadWriteManager(), (page, fullPageId, pageMemoryEx) -> {
    });
}
Also used : ConfigurationTestUtils.fixConfiguration(org.apache.ignite.internal.configuration.ConfigurationTestUtils.fixConfiguration) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider) Test(org.junit.jupiter.api.Test) PageMemory(org.apache.ignite.internal.pagememory.PageMemory) LongStream(java.util.stream.LongStream) PageIoRegistry(org.apache.ignite.internal.pagememory.io.PageIoRegistry) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) MiB(org.apache.ignite.internal.util.Constants.MiB) PageMemoryDataRegionConfiguration(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfiguration) SECONDS(java.util.concurrent.TimeUnit.SECONDS) PageMemoryNoLoadSelfTest(org.apache.ignite.internal.pagememory.impl.PageMemoryNoLoadSelfTest) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) PageIoRegistry(org.apache.ignite.internal.pagememory.io.PageIoRegistry) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider)

Example 9 with PageMemory

use of org.apache.ignite.internal.pagememory.PageMemory in project ignite-3 by apache.

the class ItBplusTreeSelfTest method createPageMemory.

/**
 * Returns page memory.
 *
 * @throws Exception If failed.
 */
protected PageMemory createPageMemory() throws Exception {
    dataRegionCfg.change(c -> c.convert(PageMemoryDataRegionChange.class).changePageSize(PAGE_SIZE).changeInitSize(MAX_MEMORY_SIZE).changeMaxSize(MAX_MEMORY_SIZE)).get(1, TimeUnit.SECONDS);
    TestPageIoRegistry ioRegistry = new TestPageIoRegistry();
    ioRegistry.loadFromServiceLoader();
    return new PageMemoryNoStoreImpl(new UnsafeMemoryProvider(null), (PageMemoryDataRegionConfiguration) fixConfiguration(dataRegionCfg), ioRegistry);
}
Also used : Collections.shuffle(java.util.Collections.shuffle) BeforeEach(org.junit.jupiter.api.BeforeEach) Arrays(java.util.Arrays) TestPageIoRegistry(org.apache.ignite.internal.pagememory.TestPageIoRegistry) REMOVE(org.apache.ignite.internal.pagememory.tree.IgniteTree.OperationType.REMOVE) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) IgniteUtils.hexLong(org.apache.ignite.internal.util.IgniteUtils.hexLong) Random(java.util.Random) IgniteStringBuilder(org.apache.ignite.lang.IgniteStringBuilder) TreeRowClosure(org.apache.ignite.internal.pagememory.tree.BplusTree.TreeRowClosure) BplusIo(org.apache.ignite.internal.pagememory.tree.io.BplusIo) IgniteLogger(org.apache.ignite.lang.IgniteLogger) IgniteTestUtils.runMultiThreadedAsync(org.apache.ignite.internal.testframework.IgniteTestUtils.runMultiThreadedAsync) DataStructure(org.apache.ignite.internal.pagememory.datastructure.DataStructure) InvokeClosure(org.apache.ignite.internal.pagememory.tree.IgniteTree.InvokeClosure) Collections.singleton(java.util.Collections.singleton) PageIdUtils.effectivePageId(org.apache.ignite.internal.pagememory.util.PageIdUtils.effectivePageId) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) UnsafeMemoryAllocatorConfigurationSchema(org.apache.ignite.configuration.schemas.store.UnsafeMemoryAllocatorConfigurationSchema) ExtendWith(org.junit.jupiter.api.extension.ExtendWith) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) PageMemoryNoStoreImpl(org.apache.ignite.internal.pagememory.impl.PageMemoryNoStoreImpl) Map(java.util.Map) BplusLeafIo(org.apache.ignite.internal.pagememory.tree.io.BplusLeafIo) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider) CyclicBarrier(java.util.concurrent.CyclicBarrier) DataRegionConfiguration(org.apache.ignite.configuration.schemas.store.DataRegionConfiguration) Predicate(java.util.function.Predicate) BplusMetaIo(org.apache.ignite.internal.pagememory.tree.io.BplusMetaIo) Collection(java.util.Collection) OperationType(org.apache.ignite.internal.pagememory.tree.IgniteTree.OperationType) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) BlockingQueue(java.util.concurrent.BlockingQueue) Collections.emptyIterator(java.util.Collections.emptyIterator) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) BplusInnerIo(org.apache.ignite.internal.pagememory.tree.io.BplusInnerIo) ConfigurationTestUtils.fixConfiguration(org.apache.ignite.internal.configuration.ConfigurationTestUtils.fixConfiguration) Test(org.junit.jupiter.api.Test) ArrayBlockingQueue(java.util.concurrent.ArrayBlockingQueue) Nullable(org.jetbrains.annotations.Nullable) INDEX_PARTITION(org.apache.ignite.internal.pagememory.PageIdAllocator.INDEX_PARTITION) List(java.util.List) IgniteInternalCheckedException(org.apache.ignite.lang.IgniteInternalCheckedException) PUT(org.apache.ignite.internal.pagememory.tree.IgniteTree.OperationType.PUT) ReuseList(org.apache.ignite.internal.pagememory.reuse.ReuseList) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) PageIo.getPageId(org.apache.ignite.internal.pagememory.io.PageIo.getPageId) Queue(java.util.Queue) ConcurrentLinkedQueue(java.util.concurrent.ConcurrentLinkedQueue) AtomicLongArray(java.util.concurrent.atomic.AtomicLongArray) Assertions.fail(org.junit.jupiter.api.Assertions.fail) NOOP(org.apache.ignite.internal.pagememory.tree.IgniteTree.OperationType.NOOP) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) PageMemory(org.apache.ignite.internal.pagememory.PageMemory) IgniteRandom(org.apache.ignite.internal.util.IgniteRandom) PageUtils.getLong(org.apache.ignite.internal.pagememory.util.PageUtils.getLong) Assertions.assertNull(org.junit.jupiter.api.Assertions.assertNull) PageMemoryDataRegionConfigurationSchema(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfigurationSchema) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) FullPageId(org.apache.ignite.internal.pagememory.FullPageId) DataStructure.rnd(org.apache.ignite.internal.pagememory.datastructure.DataStructure.rnd) TreeSet(java.util.TreeSet) FLAG_AUX(org.apache.ignite.internal.pagememory.PageIdAllocator.FLAG_AUX) ArrayList(java.util.ArrayList) ConcurrentMap(java.util.concurrent.ConcurrentMap) InjectConfiguration(org.apache.ignite.internal.configuration.testframework.InjectConfiguration) PageLockListenerNoOp(org.apache.ignite.internal.pagememory.util.PageLockListenerNoOp) ConfigurationExtension(org.apache.ignite.internal.configuration.testframework.ConfigurationExtension) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) Iterator(java.util.Iterator) IgniteStripedLock(org.apache.ignite.internal.util.IgniteStripedLock) BrokenBarrierException(java.util.concurrent.BrokenBarrierException) BaseIgniteAbstractTest(org.apache.ignite.internal.testframework.BaseIgniteAbstractTest) IgniteTestUtils.runMultiThreaded(org.apache.ignite.internal.testframework.IgniteTestUtils.runMultiThreaded) Assertions.assertSame(org.junit.jupiter.api.Assertions.assertSame) TimeUnit(java.util.concurrent.TimeUnit) AtomicLong(java.util.concurrent.atomic.AtomicLong) Lock(java.util.concurrent.locks.Lock) AfterEach(org.junit.jupiter.api.AfterEach) TreeMap(java.util.TreeMap) TestTree.threadId(org.apache.ignite.internal.pagememory.tree.ItBplusTreeSelfTest.TestTree.threadId) PageUtils.putLong(org.apache.ignite.internal.pagememory.util.PageUtils.putLong) GiB(org.apache.ignite.internal.util.Constants.GiB) IoVersions(org.apache.ignite.internal.pagememory.io.IoVersions) IgniteCursor(org.apache.ignite.internal.util.IgniteCursor) PageMemoryDataRegionConfiguration(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfiguration) PageLockListener(org.apache.ignite.internal.pagememory.util.PageLockListener) PageMemoryNoStoreImpl(org.apache.ignite.internal.pagememory.impl.PageMemoryNoStoreImpl) TestPageIoRegistry(org.apache.ignite.internal.pagememory.TestPageIoRegistry) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider)

Example 10 with PageMemory

use of org.apache.ignite.internal.pagememory.PageMemory in project ignite-3 by apache.

the class ItBplusTreePageMemoryImplTest method createPageMemory.

/**
 * {@inheritDoc}
 */
@Override
protected PageMemory createPageMemory() throws Exception {
    dataRegionCfg.change(c -> c.convert(PageMemoryDataRegionChange.class).changePageSize(PAGE_SIZE).changeInitSize(MAX_MEMORY_SIZE).changeMaxSize(MAX_MEMORY_SIZE)).get(1, TimeUnit.SECONDS);
    long[] sizes = LongStream.range(0, CPUS + 1).map(i -> MAX_MEMORY_SIZE / CPUS).toArray();
    sizes[CPUS] = 10 * MiB;
    TestPageIoRegistry ioRegistry = new TestPageIoRegistry();
    ioRegistry.loadFromServiceLoader();
    return new PageMemoryImpl(new UnsafeMemoryProvider(null), (PageMemoryDataRegionConfiguration) fixConfiguration(dataRegionCfg), ioRegistry, sizes, new TestPageReadWriteManager(), (page, fullPageId, pageMemoryEx) -> {
    });
}
Also used : ConfigurationTestUtils.fixConfiguration(org.apache.ignite.internal.configuration.ConfigurationTestUtils.fixConfiguration) TimeUnit(java.util.concurrent.TimeUnit) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider) PageMemory(org.apache.ignite.internal.pagememory.PageMemory) TestPageIoRegistry(org.apache.ignite.internal.pagememory.TestPageIoRegistry) LongStream(java.util.stream.LongStream) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) MiB(org.apache.ignite.internal.util.Constants.MiB) PageMemoryDataRegionConfiguration(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfiguration) BplusTree(org.apache.ignite.internal.pagememory.tree.BplusTree) ItBplusTreeSelfTest(org.apache.ignite.internal.pagememory.tree.ItBplusTreeSelfTest) TestPageIoRegistry(org.apache.ignite.internal.pagememory.TestPageIoRegistry) PageMemoryDataRegionChange(org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange) UnsafeMemoryProvider(org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider)

Aggregations

PageMemory (org.apache.ignite.internal.pagememory.PageMemory)12 BaseIgniteAbstractTest (org.apache.ignite.internal.testframework.BaseIgniteAbstractTest)9 Test (org.junit.jupiter.api.Test)9 PageMemoryDataRegionChange (org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionChange)7 PageMemoryDataRegionConfiguration (org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfiguration)7 ConfigurationTestUtils.fixConfiguration (org.apache.ignite.internal.configuration.ConfigurationTestUtils.fixConfiguration)7 FullPageId (org.apache.ignite.internal.pagememory.FullPageId)7 UnsafeMemoryProvider (org.apache.ignite.internal.pagememory.mem.unsafe.UnsafeMemoryProvider)7 ArrayList (java.util.ArrayList)5 TimeUnit (java.util.concurrent.TimeUnit)5 TestPageIoRegistry (org.apache.ignite.internal.pagememory.TestPageIoRegistry)5 MiB (org.apache.ignite.internal.util.Constants.MiB)5 DataRegionConfiguration (org.apache.ignite.configuration.schemas.store.DataRegionConfiguration)4 PageMemoryDataRegionConfigurationSchema (org.apache.ignite.configuration.schemas.store.PageMemoryDataRegionConfigurationSchema)4 UnsafeMemoryAllocatorConfigurationSchema (org.apache.ignite.configuration.schemas.store.UnsafeMemoryAllocatorConfigurationSchema)4 ConfigurationExtension (org.apache.ignite.internal.configuration.testframework.ConfigurationExtension)4 InjectConfiguration (org.apache.ignite.internal.configuration.testframework.InjectConfiguration)4 IgniteInternalCheckedException (org.apache.ignite.lang.IgniteInternalCheckedException)4 Assertions.assertEquals (org.junit.jupiter.api.Assertions.assertEquals)4 ExtendWith (org.junit.jupiter.api.extension.ExtendWith)4