Search in sources :

Example 1 with BenchmarkOptions

use of com.carrotsearch.junitbenchmarks.BenchmarkOptions in project druid by druid-io.

the class BitmapCreationBenchmark method testToImmutableByteArray.

@BenchmarkOptions(warmupRounds = 10, benchmarkRounds = 1000)
@Test
public void testToImmutableByteArray() {
    ImmutableBitmap immutableBitmap = factory.makeImmutableBitmap(baseMutableBitmap);
    Assert.assertArrayEquals(baseBytes, immutableBitmap.toBytes());
}
Also used : ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Example 2 with BenchmarkOptions

use of com.carrotsearch.junitbenchmarks.BenchmarkOptions in project druid by druid-io.

the class BenchmarkIndexibleWrites method testConcurrentReads.

/**
   BenchmarkIndexibleWrites.TestConcurrentReads[0]: [measured 100 out of 200 rounds, threads: 1 (sequential)]
   round: 0.28 [+- 0.02], round.block: 0.00 [+- 0.00], round.gc: 0.02 [+- 0.00], GC.calls: 396, GC.time: 1.84, time.total: 59.98, time.warmup: 30.51, time.bench: 29.48
   BenchmarkIndexibleWrites.TestConcurrentReads[1]: [measured 100 out of 200 rounds, threads: 1 (sequential)]
   round: 0.12 [+- 0.01], round.block: 0.00 [+- 0.00], round.gc: 0.02 [+- 0.00], GC.calls: 396, GC.time: 2.05, time.total: 29.21, time.warmup: 14.65, time.bench: 14.55

   */
@BenchmarkOptions(warmupRounds = 100, benchmarkRounds = 100, clock = Clock.REAL_TIME, callgc = true)
@Ignore
@Test
public void testConcurrentReads() throws ExecutionException, InterruptedException {
    final ListeningExecutorService executorService = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(concurrentThreads, new ThreadFactoryBuilder().setDaemon(false).setNameFormat("indexible-writes-benchmark-reader-%d").build()));
    final AtomicInteger index = new AtomicInteger(0);
    final AtomicInteger queryableIndex = new AtomicInteger(0);
    List<ListenableFuture<?>> futures = new LinkedList<>();
    final Integer loops = totalIndexSize / concurrentThreads;
    final AtomicBoolean done = new AtomicBoolean(false);
    final CountDownLatch start = new CountDownLatch(1);
    for (int i = 0; i < concurrentThreads; ++i) {
        futures.add(executorService.submit(new Runnable() {

            @Override
            public void run() {
                try {
                    start.await();
                } catch (InterruptedException e) {
                    throw Throwables.propagate(e);
                }
                final Random rndGen = new Random();
                while (!done.get()) {
                    Integer idx = rndGen.nextInt(queryableIndex.get() + 1);
                    Assert.assertEquals(idx, concurrentIndexible.get(idx));
                }
            }
        }));
    }
    {
        final Integer idx = index.getAndIncrement();
        concurrentIndexible.set(idx, idx);
        start.countDown();
    }
    for (int i = 1; i < totalIndexSize; ++i) {
        final Integer idx = index.getAndIncrement();
        concurrentIndexible.set(idx, idx);
        queryableIndex.incrementAndGet();
    }
    done.set(true);
    Futures.allAsList(futures).get();
    executorService.shutdown();
    Assert.assertTrue(String.format("Index too small %d, expected %d across %d loops", index.get(), totalIndexSize, loops), index.get() >= totalIndexSize);
    for (int i = 0; i < index.get(); ++i) {
        Assert.assertEquals(i, concurrentIndexible.get(i).intValue());
    }
    concurrentIndexible.clear();
    futures.clear();
}
Also used : CountDownLatch(java.util.concurrent.CountDownLatch) LinkedList(java.util.LinkedList) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Random(java.util.Random) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ThreadFactoryBuilder(com.google.common.util.concurrent.ThreadFactoryBuilder) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ListeningExecutorService(com.google.common.util.concurrent.ListeningExecutorService) Ignore(org.junit.Ignore) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Example 3 with BenchmarkOptions

use of com.carrotsearch.junitbenchmarks.BenchmarkOptions in project parquet-mr by apache.

the class SmallRangeWritingBenchmarkTest method writeRLEWithSmallBitWidthTest.

@BenchmarkOptions(benchmarkRounds = 10, warmupRounds = 2)
@Test
public void writeRLEWithSmallBitWidthTest() {
    ValuesWriter writer = new RunLengthBitPackingHybridValuesWriter(2, 100, 20000, new DirectByteBufferAllocator());
    runWriteTest(writer);
}
Also used : DirectByteBufferAllocator(org.apache.parquet.bytes.DirectByteBufferAllocator) RunLengthBitPackingHybridValuesWriter(org.apache.parquet.column.values.rle.RunLengthBitPackingHybridValuesWriter) ValuesWriter(org.apache.parquet.column.values.ValuesWriter) RunLengthBitPackingHybridValuesWriter(org.apache.parquet.column.values.rle.RunLengthBitPackingHybridValuesWriter) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Example 4 with BenchmarkOptions

use of com.carrotsearch.junitbenchmarks.BenchmarkOptions in project parquet-mr by apache.

the class BenchmarkDeltaLengthByteArray method benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter.

@BenchmarkOptions(benchmarkRounds = 20, warmupRounds = 4)
@Test
public void benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter() throws IOException {
    DeltaLengthByteArrayValuesWriter writer = new DeltaLengthByteArrayValuesWriter(64 * 1024, 64 * 1024, new DirectByteBufferAllocator());
    DeltaLengthByteArrayValuesReader reader = new DeltaLengthByteArrayValuesReader();
    Utils.writeData(writer, values);
    ByteBufferInputStream data = writer.getBytes().toInputStream();
    Binary[] bin = Utils.readData(reader, data, values.length);
    System.out.println("size " + data.position());
}
Also used : DirectByteBufferAllocator(org.apache.parquet.bytes.DirectByteBufferAllocator) DeltaLengthByteArrayValuesReader(org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesReader) ByteBufferInputStream(org.apache.parquet.bytes.ByteBufferInputStream) DeltaLengthByteArrayValuesWriter(org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesWriter) Binary(org.apache.parquet.io.api.Binary) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Example 5 with BenchmarkOptions

use of com.carrotsearch.junitbenchmarks.BenchmarkOptions in project parquet-mr by apache.

the class BenchmarkDeltaByteArray method benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter.

@BenchmarkOptions(benchmarkRounds = 20, warmupRounds = 4)
@Test
public void benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter() throws IOException {
    DeltaByteArrayWriter writer = new DeltaByteArrayWriter(64 * 1024, 64 * 1024, new DirectByteBufferAllocator());
    DeltaByteArrayReader reader = new DeltaByteArrayReader();
    Utils.writeData(writer, values);
    ByteBufferInputStream data = writer.getBytes().toInputStream();
    Binary[] bin = Utils.readData(reader, data, values.length);
    System.out.println("size " + data.position());
}
Also used : DirectByteBufferAllocator(org.apache.parquet.bytes.DirectByteBufferAllocator) DeltaByteArrayWriter(org.apache.parquet.column.values.deltastrings.DeltaByteArrayWriter) ByteBufferInputStream(org.apache.parquet.bytes.ByteBufferInputStream) DeltaByteArrayReader(org.apache.parquet.column.values.deltastrings.DeltaByteArrayReader) Binary(org.apache.parquet.io.api.Binary) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Aggregations

BenchmarkOptions (com.carrotsearch.junitbenchmarks.BenchmarkOptions)42 Test (org.junit.Test)42 DirectByteBufferAllocator (org.apache.parquet.bytes.DirectByteBufferAllocator)10 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)6 ListeningExecutorService (com.google.common.util.concurrent.ListeningExecutorService)6 ThreadFactoryBuilder (com.google.common.util.concurrent.ThreadFactoryBuilder)6 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)6 ByteBufferInputStream (org.apache.parquet.bytes.ByteBufferInputStream)6 Binary (org.apache.parquet.io.api.Binary)6 Ignore (org.junit.Ignore)6 ArrayList (java.util.ArrayList)4 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)4 Pointer (org.directmemory.memory.Pointer)4 BasicDBObject (com.mongodb.BasicDBObject)3 MutableBitmap (io.druid.collections.bitmap.MutableBitmap)3 LinkedList (java.util.LinkedList)3 MutableBitmap (org.apache.druid.collections.bitmap.MutableBitmap)3 DeltaBinaryPackingValuesWriter (org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriter)3 BinaryPlainValuesReader (org.apache.parquet.column.values.plain.BinaryPlainValuesReader)3 PlainValuesWriter (org.apache.parquet.column.values.plain.PlainValuesWriter)3