Search in sources :

Example 1 with NativePool

use of org.apache.cassandra.utils.memory.NativePool in project cassandra by apache.

the class Memtable method createMemtableAllocatorPool.

private static MemtablePool createMemtableAllocatorPool() {
    long heapLimit = DatabaseDescriptor.getMemtableHeapSpaceInMiB() << 20;
    long offHeapLimit = DatabaseDescriptor.getMemtableOffheapSpaceInMiB() << 20;
    final float cleaningThreshold = DatabaseDescriptor.getMemtableCleanupThreshold();
    final MemtableCleaner cleaner = ColumnFamilyStore::flushLargestMemtable;
    switch(DatabaseDescriptor.getMemtableAllocationType()) {
        case unslabbed_heap_buffers_logged:
            return new HeapPool.Logged(heapLimit, cleaningThreshold, cleaner);
        case unslabbed_heap_buffers:
            return new HeapPool(heapLimit, cleaningThreshold, cleaner);
        case heap_buffers:
            return new SlabPool(heapLimit, 0, cleaningThreshold, cleaner);
        case offheap_buffers:
            return new SlabPool(heapLimit, offHeapLimit, cleaningThreshold, cleaner);
        case offheap_objects:
            return new NativePool(heapLimit, offHeapLimit, cleaningThreshold, cleaner);
        default:
            throw new AssertionError();
    }
}
Also used : MemtableCleaner(org.apache.cassandra.utils.memory.MemtableCleaner) SlabPool(org.apache.cassandra.utils.memory.SlabPool) NativePool(org.apache.cassandra.utils.memory.NativePool) HeapPool(org.apache.cassandra.utils.memory.HeapPool)

Example 2 with NativePool

use of org.apache.cassandra.utils.memory.NativePool in project cassandra by apache.

the class CellSpecTest method data.

@Parameterized.Parameters(name = "{0}")
public static Collection<Object[]> data() {
    TableMetadata table = TableMetadata.builder("testing", "testing").addPartitionKeyColumn("pk", BytesType.instance).build();
    byte[] rawBytes = { 0, 1, 2, 3, 4, 5, 6 };
    ByteBuffer bbBytes = ByteBuffer.wrap(rawBytes);
    NativePool pool = new NativePool(1024, 1024, 1, () -> ImmediateFuture.success(true));
    NativeAllocator allocator = pool.newAllocator(null);
    OpOrder order = new OpOrder();
    List<Cell<?>> tests = new ArrayList<>();
    BiConsumer<ColumnMetadata, CellPath> fn = (column, path) -> {
        tests.add(new ArrayCell(column, 1234, 1, 1, rawBytes, path));
        tests.add(new BufferCell(column, 1234, 1, 1, bbBytes, path));
        tests.add(new NativeCell(allocator, order.getCurrent(), column, 1234, 1, 1, bbBytes, path));
    };
    // simple
    fn.accept(ColumnMetadata.regularColumn(table, bytes("simple"), BytesType.instance), null);
    // complex
    // seems NativeCell does not allow CellPath.TOP, or CellPath.BOTTOM
    fn.accept(ColumnMetadata.regularColumn(table, bytes("complex"), ListType.getInstance(BytesType.instance, true)), CellPath.create(bytes(UUIDGen.getTimeUUID())));
    return tests.stream().map(a -> new Object[] { a.getClass().getSimpleName() + ":" + (a.path() == null ? "simple" : "complex"), a }).collect(Collectors.toList());
}
Also used : TableMetadata(org.apache.cassandra.schema.TableMetadata) CellPath(org.apache.cassandra.db.rows.CellPath) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) ByteBufferUtil.bytes(org.apache.cassandra.utils.ByteBufferUtil.bytes) ByteBuffer(java.nio.ByteBuffer) ArrayList(java.util.ArrayList) OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) ListType(org.apache.cassandra.db.marshal.ListType) BufferCell(org.apache.cassandra.db.rows.BufferCell) BiConsumer(java.util.function.BiConsumer) Parameterized(org.junit.runners.Parameterized) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) NativeCell(org.apache.cassandra.db.rows.NativeCell) Collection(java.util.Collection) BytesType(org.apache.cassandra.db.marshal.BytesType) Test(org.junit.Test) NativeAllocator(org.apache.cassandra.utils.memory.NativeAllocator) Collectors(java.util.stream.Collectors) UUIDGen(org.apache.cassandra.utils.UUIDGen) CellPath(org.apache.cassandra.db.rows.CellPath) List(java.util.List) Cell(org.apache.cassandra.db.rows.Cell) ImmediateFuture(org.apache.cassandra.utils.concurrent.ImmediateFuture) TableMetadata(org.apache.cassandra.schema.TableMetadata) ObjectSizes(org.apache.cassandra.utils.ObjectSizes) NativePool(org.apache.cassandra.utils.memory.NativePool) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) NativePool(org.apache.cassandra.utils.memory.NativePool) ArrayList(java.util.ArrayList) NativeCell(org.apache.cassandra.db.rows.NativeCell) BufferCell(org.apache.cassandra.db.rows.BufferCell) ByteBuffer(java.nio.ByteBuffer) OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) NativeAllocator(org.apache.cassandra.utils.memory.NativeAllocator) BufferCell(org.apache.cassandra.db.rows.BufferCell) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) NativeCell(org.apache.cassandra.db.rows.NativeCell) Cell(org.apache.cassandra.db.rows.Cell)

Example 3 with NativePool

use of org.apache.cassandra.utils.memory.NativePool in project cassandra by apache.

the class ClusteringHeapSizeTest method data.

@Parameterized.Parameters(name = "{0}")
public static Collection<Object[]> data() {
    byte[] rawBytes = { 0, 1, 2, 3, 4, 5, 6 };
    NativePool pool = new NativePool(1024, 1024, 1, () -> ImmediateFuture.success(true));
    OpOrder order = new OpOrder();
    ArrayClustering array = ArrayClustering.make(rawBytes);
    BufferClustering buffer = BufferClustering.make(ByteBuffer.wrap(rawBytes));
    return Arrays.asList(new Object[][] { { array }, { buffer }, { new NativeClustering(pool.newAllocator(null), order.getCurrent(), array) } });
}
Also used : OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) NativePool(org.apache.cassandra.utils.memory.NativePool)

Aggregations

NativePool (org.apache.cassandra.utils.memory.NativePool)3 OpOrder (org.apache.cassandra.utils.concurrent.OpOrder)2 ByteBuffer (java.nio.ByteBuffer)1 ArrayList (java.util.ArrayList)1 Collection (java.util.Collection)1 List (java.util.List)1 BiConsumer (java.util.function.BiConsumer)1 Collectors (java.util.stream.Collectors)1 BytesType (org.apache.cassandra.db.marshal.BytesType)1 ListType (org.apache.cassandra.db.marshal.ListType)1 ArrayCell (org.apache.cassandra.db.rows.ArrayCell)1 BufferCell (org.apache.cassandra.db.rows.BufferCell)1 Cell (org.apache.cassandra.db.rows.Cell)1 CellPath (org.apache.cassandra.db.rows.CellPath)1 NativeCell (org.apache.cassandra.db.rows.NativeCell)1 ColumnMetadata (org.apache.cassandra.schema.ColumnMetadata)1 TableMetadata (org.apache.cassandra.schema.TableMetadata)1 ByteBufferUtil.bytes (org.apache.cassandra.utils.ByteBufferUtil.bytes)1 ObjectSizes (org.apache.cassandra.utils.ObjectSizes)1 UUIDGen (org.apache.cassandra.utils.UUIDGen)1