Examples with IgniteOutOfMemoryException org.apache.ignite.internal.mem.IgniteOutOfMemoryException used on opensource projects

Example 6 with IgniteOutOfMemoryException

use of org.apache.ignite.internal.mem.IgniteOutOfMemoryException in project ignite by apache.

the class CacheMemoryPolicyConfigurationTest method testTooSmallMemoryPolicy.

/**
     * Verifies that {@link IgniteOutOfMemoryException} is thrown when cache is configured with too small MemoryPolicy.
     */
public void testTooSmallMemoryPolicy() throws Exception {
    memCfg = new MemoryConfiguration();
    MemoryPolicyConfiguration dfltPlcCfg = new MemoryPolicyConfiguration();
    dfltPlcCfg.setName("dfltPlc");
    dfltPlcCfg.setInitialSize(10 * 1024 * 1024);
    dfltPlcCfg.setMaxSize(10 * 1024 * 1024);
    MemoryPolicyConfiguration bigPlcCfg = new MemoryPolicyConfiguration();
    bigPlcCfg.setName("bigPlc");
    bigPlcCfg.setMaxSize(1024 * 1024 * 1024);
    memCfg.setMemoryPolicies(dfltPlcCfg, bigPlcCfg);
    memCfg.setDefaultMemoryPolicyName("dfltPlc");
    ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
    IgniteEx ignite0 = startGrid(0);
    IgniteCache<Object, Object> cache = ignite0.cache(DEFAULT_CACHE_NAME);
    boolean oomeThrown = false;
    try {
        for (int i = 0; i < 500_000; i++) cache.put(i, "abc");
    } catch (Exception e) {
        Throwable cause = e;
        do {
            if (cause instanceof IgniteOutOfMemoryException) {
                oomeThrown = true;
                break;
            }
            if (cause == null)
                break;
            if (cause.getSuppressed() == null || cause.getSuppressed().length == 0)
                cause = cause.getCause();
            else
                cause = cause.getSuppressed()[0];
        } while (true);
    }
    if (!oomeThrown)
        fail("OutOfMemoryException hasn't been thrown");
}

Example 7 with IgniteOutOfMemoryException

use of org.apache.ignite.internal.mem.IgniteOutOfMemoryException in project ignite by apache.

the class IgniteCacheDatabaseSharedManager method ensureFreeSpaceForInsert.

/**
 * Checks that the given {@code region} has enough space for putting a new entry.
 *
 * This method makes sense then and only then
 * the data region is not persisted {@link DataRegionConfiguration#isPersistenceEnabled()}
 * and page eviction is disabled {@link DataPageEvictionMode#DISABLED}.
 *
 * The non-persistent region should reserve a number of pages to support a free list {@link AbstractFreeList}.
 * For example, removing a row from underlying store may require allocating a new data page
 * in order to move a tracked page from one bucket to another one which does not have a free space for a new stripe.
 * See {@link AbstractFreeList#removeDataRowByLink}.
 * Therefore, inserting a new entry should be prevented in case of some threshold is exceeded.
 *
 * @param region Data region to be checked.
 * @param dataRowSize Size of data row to be inserted.
 * @throws IgniteOutOfMemoryException In case of the given data region does not have enough free space
 * for putting a new entry.
 */
public void ensureFreeSpaceForInsert(DataRegion region, int dataRowSize) throws IgniteOutOfMemoryException {
    if (region == null)
        return;
    DataRegionConfiguration regCfg = region.config();
    if (regCfg.getPageEvictionMode() != DataPageEvictionMode.DISABLED || regCfg.isPersistenceEnabled())
        return;
    long memorySize = regCfg.getMaxSize();
    PageMemory pageMem = region.pageMemory();
    CacheFreeList freeList = freeListMap.get(regCfg.getName());
    long nonEmptyPages = (pageMem.loadedPages() - freeList.emptyDataPages());
    // The maximum number of pages that can be allocated (memorySize / systemPageSize)
    // should be greater or equal to pages required for inserting a new entry plus
    // the current number of non-empty pages plus the number of pages that may be required in order to move
    // all pages to a reuse bucket, that is equal to nonEmptyPages * 8 / pageSize, where 8 is the size of a link.
    // Note that not the whole page can be used to storing links,
    // see PagesListNodeIO and PagesListMetaIO#getCapacity(), so we pessimistically multiply the result on 1.5,
    // in any way, the number of required pages is less than 1 percent.
    boolean oomThreshold = (memorySize / pageMem.systemPageSize()) < ((double) dataRowSize / pageMem.pageSize() + nonEmptyPages * (8.0 * 1.5 / pageMem.pageSize() + 1) + 256);
    if (oomThreshold) {
        IgniteOutOfMemoryException oom = new IgniteOutOfMemoryException("Out of memory in data region [" + "name=" + regCfg.getName() + ", initSize=" + U.readableSize(regCfg.getInitialSize(), false) + ", maxSize=" + U.readableSize(regCfg.getMaxSize(), false) + ", persistenceEnabled=" + regCfg.isPersistenceEnabled() + "] Try the following:" + U.nl() + "  ^-- Increase maximum off-heap memory size (DataRegionConfiguration.maxSize)" + U.nl() + "  ^-- Enable Ignite persistence (DataRegionConfiguration.persistenceEnabled)" + U.nl() + "  ^-- Enable eviction or expiration policies");
        if (cctx.kernalContext() != null)
            cctx.kernalContext().failure().process(new FailureContext(FailureType.CRITICAL_ERROR, oom));
        throw oom;
    }
}

Example 8 with IgniteOutOfMemoryException

use of org.apache.ignite.internal.mem.IgniteOutOfMemoryException in project ignite by apache.

the class PageMemoryImplTest method testCheckpointProtocolCannotReplaceUnwrittenPage.

/**
 * @throws Exception if failed.
 */
@Test
public void testCheckpointProtocolCannotReplaceUnwrittenPage() throws Exception {
    TestPageStoreManager pageStoreMgr = new TestPageStoreManager();
    // Create a 1 mb page memory.
    PageMemoryImpl memory = createPageMemory(1, PageMemoryImpl.ThrottlingPolicy.TARGET_RATIO_BASED, pageStoreMgr, pageStoreMgr, null);
    int initPageCnt = 500;
    List<FullPageId> allocated = new ArrayList<>(initPageCnt);
    for (int i = 0; i < initPageCnt; i++) {
        long id = memory.allocatePage(1, INDEX_PARTITION, FLAG_IDX);
        FullPageId fullId = new FullPageId(id, 1);
        allocated.add(fullId);
        writePage(memory, fullId, (byte) 1);
    }
    // CP Write lock.
    memory.beginCheckpoint(new GridFinishedFuture());
    // CP Write unlock.
    byte[] buf = new byte[PAGE_SIZE];
    memory.checkpointWritePage(allocated.get(0), ByteBuffer.wrap(buf), (fullPageId, buf0, tag) -> {
        assertNotNull(tag);
        boolean oom = false;
        try {
            // Try force page replacement.
            while (true) {
                memory.allocatePage(1, INDEX_PARTITION, FLAG_IDX);
            }
        } catch (IgniteOutOfMemoryException ex) {
            oom = true;
        }
        assertTrue("Should oom before check replaced page.", oom);
        assertTrue("Missing page: " + fullPageId, memory.hasLoadedPage(fullPageId));
    }, null);
}

Example 9 with IgniteOutOfMemoryException

use of org.apache.ignite.internal.mem.IgniteOutOfMemoryException in project ignite by apache.

the class PageMemoryNoLoadSelfTest method testLoadedPagesCount.

/**
 * @throws Exception If failed.
 */
@Test
public void testLoadedPagesCount() throws Exception {
    PageMemory mem = memory();
    mem.start();
    int expPages = MAX_MEMORY_SIZE / mem.systemPageSize();
    try {
        for (int i = 0; i < expPages * 2; i++) allocatePage(mem);
    } catch (IgniteOutOfMemoryException e) {
        e.printStackTrace();
        // Expected.
        assertEquals(mem.loadedPages(), expPages);
    } finally {
        mem.stop(true);
    }
}

Example 10 with IgniteOutOfMemoryException

use of org.apache.ignite.internal.mem.IgniteOutOfMemoryException in project ignite by apache.

the class AlignedBuffers method allocate.

/**
 * Allocates align memory for use with O_DIRECT and returns native byte buffer.
 * @param fsBlockSize alignment, FS ans OS block size.
 * @param size capacity.
 * @return byte buffer, to be released by {@link #free(ByteBuffer)}.
 */
public static ByteBuffer allocate(int fsBlockSize, int size) {
    assert fsBlockSize > 0;
    assert size > 0;
    PointerByReference refToPtr = new PointerByReference();
    int retVal = IgniteNativeIoLib.posix_memalign(refToPtr, new NativeLong(fsBlockSize), new NativeLong(size));
    if (retVal != 0)
        throw new IgniteOutOfMemoryException("Failed to allocate memory: " + IgniteNativeIoLib.strerror(retVal));
    return GridUnsafe.wrapPointer(Pointer.nativeValue(refToPtr.getValue()), size);
}