Search in sources :

Example 1 with OpOrder

use of org.apache.cassandra.utils.concurrent.OpOrder in project cassandra by apache.

the class CellSpecTest method data.

@Parameterized.Parameters(name = "{0}")
public static Collection<Object[]> data() {
    TableMetadata table = TableMetadata.builder("testing", "testing").addPartitionKeyColumn("pk", BytesType.instance).build();
    byte[] rawBytes = { 0, 1, 2, 3, 4, 5, 6 };
    ByteBuffer bbBytes = ByteBuffer.wrap(rawBytes);
    NativePool pool = new NativePool(1024, 1024, 1, () -> ImmediateFuture.success(true));
    NativeAllocator allocator = pool.newAllocator(null);
    OpOrder order = new OpOrder();
    List<Cell<?>> tests = new ArrayList<>();
    BiConsumer<ColumnMetadata, CellPath> fn = (column, path) -> {
        tests.add(new ArrayCell(column, 1234, 1, 1, rawBytes, path));
        tests.add(new BufferCell(column, 1234, 1, 1, bbBytes, path));
        tests.add(new NativeCell(allocator, order.getCurrent(), column, 1234, 1, 1, bbBytes, path));
    };
    // simple
    fn.accept(ColumnMetadata.regularColumn(table, bytes("simple"), BytesType.instance), null);
    // complex
    // seems NativeCell does not allow CellPath.TOP, or CellPath.BOTTOM
    fn.accept(ColumnMetadata.regularColumn(table, bytes("complex"), ListType.getInstance(BytesType.instance, true)), CellPath.create(bytes(UUIDGen.getTimeUUID())));
    return tests.stream().map(a -> new Object[] { a.getClass().getSimpleName() + ":" + (a.path() == null ? "simple" : "complex"), a }).collect(Collectors.toList());
}
Also used : TableMetadata(org.apache.cassandra.schema.TableMetadata) CellPath(org.apache.cassandra.db.rows.CellPath) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) ByteBufferUtil.bytes(org.apache.cassandra.utils.ByteBufferUtil.bytes) ByteBuffer(java.nio.ByteBuffer) ArrayList(java.util.ArrayList) OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) ListType(org.apache.cassandra.db.marshal.ListType) BufferCell(org.apache.cassandra.db.rows.BufferCell) BiConsumer(java.util.function.BiConsumer) Parameterized(org.junit.runners.Parameterized) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) NativeCell(org.apache.cassandra.db.rows.NativeCell) Collection(java.util.Collection) BytesType(org.apache.cassandra.db.marshal.BytesType) Test(org.junit.Test) NativeAllocator(org.apache.cassandra.utils.memory.NativeAllocator) Collectors(java.util.stream.Collectors) UUIDGen(org.apache.cassandra.utils.UUIDGen) CellPath(org.apache.cassandra.db.rows.CellPath) List(java.util.List) Cell(org.apache.cassandra.db.rows.Cell) ImmediateFuture(org.apache.cassandra.utils.concurrent.ImmediateFuture) TableMetadata(org.apache.cassandra.schema.TableMetadata) ObjectSizes(org.apache.cassandra.utils.ObjectSizes) NativePool(org.apache.cassandra.utils.memory.NativePool) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) ColumnMetadata(org.apache.cassandra.schema.ColumnMetadata) NativePool(org.apache.cassandra.utils.memory.NativePool) ArrayList(java.util.ArrayList) NativeCell(org.apache.cassandra.db.rows.NativeCell) BufferCell(org.apache.cassandra.db.rows.BufferCell) ByteBuffer(java.nio.ByteBuffer) OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) NativeAllocator(org.apache.cassandra.utils.memory.NativeAllocator) BufferCell(org.apache.cassandra.db.rows.BufferCell) ArrayCell(org.apache.cassandra.db.rows.ArrayCell) NativeCell(org.apache.cassandra.db.rows.NativeCell) Cell(org.apache.cassandra.db.rows.Cell)

Example 2 with OpOrder

use of org.apache.cassandra.utils.concurrent.OpOrder in project cassandra by apache.

the class Memtable method estimateRowOverhead.

private static int estimateRowOverhead(final int count) {
    // calculate row overhead
    try (final OpOrder.Group group = new OpOrder().start()) {
        int rowOverhead;
        MemtableAllocator allocator = MEMORY_POOL.newAllocator(null);
        ConcurrentNavigableMap<PartitionPosition, Object> partitions = new ConcurrentSkipListMap<>();
        final Object val = new Object();
        for (int i = 0; i < count; i++) partitions.put(allocator.clone(new BufferDecoratedKey(new LongToken(i), ByteBufferUtil.EMPTY_BYTE_BUFFER), group), val);
        double avgSize = ObjectSizes.measureDeep(partitions) / (double) count;
        rowOverhead = (int) ((avgSize - Math.floor(avgSize)) < 0.05 ? Math.floor(avgSize) : Math.ceil(avgSize));
        rowOverhead -= ObjectSizes.measureDeep(new LongToken(0));
        rowOverhead += AtomicBTreePartition.EMPTY_SIZE;
        rowOverhead += AbstractBTreePartition.HOLDER_UNSHARED_HEAP_SIZE;
        allocator.setDiscarding();
        allocator.setDiscarded();
        return rowOverhead;
    }
}
Also used : ConcurrentSkipListMap(java.util.concurrent.ConcurrentSkipListMap) OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) MemtableAllocator(org.apache.cassandra.utils.memory.MemtableAllocator) LongToken(org.apache.cassandra.dht.Murmur3Partitioner.LongToken)

Example 3 with OpOrder

use of org.apache.cassandra.utils.concurrent.OpOrder in project cassandra by apache.

the class ClusteringHeapSizeTest method data.

@Parameterized.Parameters(name = "{0}")
public static Collection<Object[]> data() {
    byte[] rawBytes = { 0, 1, 2, 3, 4, 5, 6 };
    NativePool pool = new NativePool(1024, 1024, 1, () -> ImmediateFuture.success(true));
    OpOrder order = new OpOrder();
    ArrayClustering array = ArrayClustering.make(rawBytes);
    BufferClustering buffer = BufferClustering.make(ByteBuffer.wrap(rawBytes));
    return Arrays.asList(new Object[][] { { array }, { buffer }, { new NativeClustering(pool.newAllocator(null), order.getCurrent(), array) } });
}
Also used : OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) NativePool(org.apache.cassandra.utils.memory.NativePool)

Example 4 with OpOrder

use of org.apache.cassandra.utils.concurrent.OpOrder in project cassandra by apache.

the class NativeAllocatorTest method setUp.

@Before
public void setUp() {
    exec = Executors.newScheduledThreadPool(2);
    order = new OpOrder();
    group = order.start();
    canClean = new CountDownLatch(1);
    isClean = new CountDownLatch(1);
    allocatorRef = new AtomicReference<>();
    barrier = new AtomicReference<>();
    pool = new NativePool(1, 100, 0.75f, () -> {
        try {
            canClean.await();
        } catch (InterruptedException e) {
            throw new AssertionError();
        }
        if (isClean.getCount() > 0) {
            allocatorRef.get().offHeap().released(80);
            isClean.countDown();
        }
        return ImmediateFuture.success(true);
    });
    allocator = new NativeAllocator(pool);
    allocatorRef.set(allocator);
    markBlocking = () -> {
        barrier.set(order.newBarrier());
        barrier.get().issue();
        barrier.get().markBlocking();
    };
}
Also used : OpOrder(org.apache.cassandra.utils.concurrent.OpOrder) Before(org.junit.Before)

Aggregations

OpOrder (org.apache.cassandra.utils.concurrent.OpOrder)4 NativePool (org.apache.cassandra.utils.memory.NativePool)2 ByteBuffer (java.nio.ByteBuffer)1 ArrayList (java.util.ArrayList)1 Collection (java.util.Collection)1 List (java.util.List)1 ConcurrentSkipListMap (java.util.concurrent.ConcurrentSkipListMap)1 BiConsumer (java.util.function.BiConsumer)1 Collectors (java.util.stream.Collectors)1 BytesType (org.apache.cassandra.db.marshal.BytesType)1 ListType (org.apache.cassandra.db.marshal.ListType)1 ArrayCell (org.apache.cassandra.db.rows.ArrayCell)1 BufferCell (org.apache.cassandra.db.rows.BufferCell)1 Cell (org.apache.cassandra.db.rows.Cell)1 CellPath (org.apache.cassandra.db.rows.CellPath)1 NativeCell (org.apache.cassandra.db.rows.NativeCell)1 LongToken (org.apache.cassandra.dht.Murmur3Partitioner.LongToken)1 ColumnMetadata (org.apache.cassandra.schema.ColumnMetadata)1 TableMetadata (org.apache.cassandra.schema.TableMetadata)1 ByteBufferUtil.bytes (org.apache.cassandra.utils.ByteBufferUtil.bytes)1