Search in sources :

Example 96 with DrillBuf

use of io.netty.buffer.DrillBuf in project drill by axbaretto.

the class TestBaseAllocator method testRootAllocator_createChildAndUse.

@Test
public void testRootAllocator_createChildAndUse() throws Exception {
    try (final RootAllocator rootAllocator = new RootAllocator(MAX_ALLOCATION)) {
        try (final BufferAllocator childAllocator = rootAllocator.newChildAllocator("createChildAndUse", 0, MAX_ALLOCATION)) {
            final DrillBuf drillBuf = childAllocator.buffer(512);
            assertNotNull("allocation failed", drillBuf);
            drillBuf.release();
        }
    }
}
Also used : DrillBuf(io.netty.buffer.DrillBuf) MemoryTest(org.apache.drill.categories.MemoryTest) Test(org.junit.Test)

Example 97 with DrillBuf

use of io.netty.buffer.DrillBuf in project drill by axbaretto.

the class TestBaseAllocator method testAllocator_claimedReservation.

@Test
public void testAllocator_claimedReservation() throws Exception {
    try (final RootAllocator rootAllocator = new RootAllocator(MAX_ALLOCATION)) {
        try (final BufferAllocator childAllocator1 = rootAllocator.newChildAllocator("claimedReservation", 0, MAX_ALLOCATION)) {
            try (final AllocationReservation reservation = childAllocator1.newReservation()) {
                assertTrue(reservation.add(32));
                assertTrue(reservation.add(32));
                final DrillBuf drillBuf = reservation.allocateBuffer();
                assertEquals(64, drillBuf.capacity());
                rootAllocator.verify();
                drillBuf.release();
                rootAllocator.verify();
            }
            rootAllocator.verify();
        }
    }
}
Also used : DrillBuf(io.netty.buffer.DrillBuf) MemoryTest(org.apache.drill.categories.MemoryTest) Test(org.junit.Test)

Example 98 with DrillBuf

use of io.netty.buffer.DrillBuf in project drill by axbaretto.

the class TestBaseAllocator method testAllocator_overAllocateParent.

@Test
public void testAllocator_overAllocateParent() throws Exception {
    try (final RootAllocator rootAllocator = new RootAllocator(MAX_ALLOCATION)) {
        try (final BufferAllocator childAllocator = rootAllocator.newChildAllocator("overAllocateParent", 0, MAX_ALLOCATION)) {
            final DrillBuf drillBuf1 = rootAllocator.buffer(MAX_ALLOCATION / 2);
            assertNotNull("allocation failed", drillBuf1);
            final DrillBuf drillBuf2 = childAllocator.buffer(MAX_ALLOCATION / 2);
            assertNotNull("allocation failed", drillBuf2);
            try {
                childAllocator.buffer(MAX_ALLOCATION / 4);
                fail("allocated memory beyond max allowed");
            } catch (OutOfMemoryException e) {
            // expected
            }
            drillBuf1.release();
            drillBuf2.release();
        }
    }
}
Also used : OutOfMemoryException(org.apache.drill.exec.exception.OutOfMemoryException) DrillBuf(io.netty.buffer.DrillBuf) MemoryTest(org.apache.drill.categories.MemoryTest) Test(org.junit.Test)

Example 99 with DrillBuf

use of io.netty.buffer.DrillBuf in project drill by axbaretto.

the class BaseAllocator method bufferWithoutReservation.

/**
 * Used by usual allocation as well as for allocating a pre-reserved buffer. Skips the typical accounting associated
 * with creating a new buffer.
 */
private DrillBuf bufferWithoutReservation(final int size, BufferManager bufferManager) throws OutOfMemoryException {
    assertOpen();
    final AllocationManager manager = new AllocationManager(this, size);
    // +1 ref cnt (required)
    final BufferLedger ledger = manager.associate(this);
    final DrillBuf buffer = ledger.newDrillBuf(0, size, bufferManager);
    // make sure that our allocation is equal to what we expected.
    Preconditions.checkArgument(buffer.capacity() == size, "Allocated capacity %d was not equal to requested capacity %d.", buffer.capacity(), size);
    return buffer;
}
Also used : BufferLedger(org.apache.drill.exec.memory.AllocationManager.BufferLedger) DrillBuf(io.netty.buffer.DrillBuf)

Example 100 with DrillBuf

use of io.netty.buffer.DrillBuf in project drill by axbaretto.

the class BaseAllocator method buffer.

@Override
public DrillBuf buffer(final int initialRequestSize, BufferManager manager) {
    assertOpen();
    Preconditions.checkArgument(initialRequestSize >= 0, "the requested size must be non-negative");
    if (initialRequestSize == 0) {
        return empty;
    }
    // round to next largest power of two if we're within a chunk since that is how our allocator operates
    final int actualRequestSize = initialRequestSize < CHUNK_SIZE ? nextPowerOfTwo(initialRequestSize) : initialRequestSize;
    AllocationOutcome outcome = allocateBytes(actualRequestSize);
    if (!outcome.isOk()) {
        throw new OutOfMemoryException(createErrorMsg(this, actualRequestSize, initialRequestSize));
    }
    boolean success = false;
    try {
        DrillBuf buffer = bufferWithoutReservation(actualRequestSize, manager);
        success = true;
        return buffer;
    } finally {
        if (!success) {
            releaseBytes(actualRequestSize);
        }
    }
}
Also used : OutOfMemoryException(org.apache.drill.exec.exception.OutOfMemoryException) DrillBuf(io.netty.buffer.DrillBuf)

Aggregations

DrillBuf (io.netty.buffer.DrillBuf)187 Test (org.junit.Test)63 MemoryTest (org.apache.drill.categories.MemoryTest)38 SelectionVector4 (org.apache.drill.exec.record.selection.SelectionVector4)22 ValueVector (org.apache.drill.exec.vector.ValueVector)18 BaseTest (org.apache.drill.test.BaseTest)18 MaterializedField (org.apache.drill.exec.record.MaterializedField)16 IOException (java.io.IOException)13 VectorTest (org.apache.drill.categories.VectorTest)13 OutOfMemoryException (org.apache.drill.exec.exception.OutOfMemoryException)13 ExecTest (org.apache.drill.exec.ExecTest)11 BufferAllocator (org.apache.drill.exec.memory.BufferAllocator)11 VectorContainer (org.apache.drill.exec.record.VectorContainer)10 Stopwatch (com.google.common.base.Stopwatch)9 ByteBuffer (java.nio.ByteBuffer)9 BatchSchema (org.apache.drill.exec.record.BatchSchema)9 UnlikelyTest (org.apache.drill.categories.UnlikelyTest)8 UserBitShared (org.apache.drill.exec.proto.UserBitShared)8 SerializedField (org.apache.drill.exec.proto.UserBitShared.SerializedField)8 DrillTest (org.apache.drill.test.DrillTest)8