Examples with DeltaBinaryPackingValuesWriterForInteger org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriterForInteger used on opensource projects

Example 1 with DeltaBinaryPackingValuesWriterForInteger

use of org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriterForInteger in project parquet-mr by apache.

the class BenchmarkIntegerOutputSize method testRandomIntegers.

public void testRandomIntegers(IntFunc func, int bitWidth) {
    DeltaBinaryPackingValuesWriter delta = new DeltaBinaryPackingValuesWriterForInteger(blockSize, miniBlockNum, 100, 20000, new DirectByteBufferAllocator());
    RunLengthBitPackingHybridValuesWriter rle = new RunLengthBitPackingHybridValuesWriter(bitWidth, 100, 20000, new DirectByteBufferAllocator());
    for (int i = 0; i < dataSize; i++) {
        int v = func.getIntValue();
        delta.writeInteger(v);
        rle.writeInteger(v);
    }
    System.out.println("delta size: " + delta.getBytes().size());
    System.out.println("estimated size" + estimatedSize());
    System.out.println("rle size: " + rle.getBytes().size());
}

Example 2 with DeltaBinaryPackingValuesWriterForInteger

use of org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriterForInteger in project parquet-mr by apache.

the class RandomWritingBenchmarkTest method writeDeltaPackingTest2.

@BenchmarkOptions(benchmarkRounds = 10, warmupRounds = 2)
@Test
public void writeDeltaPackingTest2() {
    DeltaBinaryPackingValuesWriter writer = new DeltaBinaryPackingValuesWriterForInteger(blockSize, miniBlockNum, 100, 20000, new DirectByteBufferAllocator());
    runWriteTest(writer);
}

Example 3 with DeltaBinaryPackingValuesWriterForInteger

use of org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriterForInteger in project parquet-mr by apache.

the class RandomWritingBenchmarkTest method writeDeltaPackingTest.

@BenchmarkOptions(benchmarkRounds = 10, warmupRounds = 2)
@Test
public void writeDeltaPackingTest() {
    DeltaBinaryPackingValuesWriter writer = new DeltaBinaryPackingValuesWriterForInteger(blockSize, miniBlockNum, 100, 20000, new DirectByteBufferAllocator());
    runWriteTest(writer);
}

Example 4 with DeltaBinaryPackingValuesWriterForInteger

use of org.apache.parquet.column.values.delta.DeltaBinaryPackingValuesWriterForInteger in project parquet-mr by apache.

the class BenchmarkReadingRandomIntegers method prepare.

@BeforeClass
public static void prepare() throws IOException {
    Random random = new Random();
    data = new int[100000 * blockSize];
    for (int i = 0; i < data.length; i++) {
        data[i] = random.nextInt(100) - 200;
    }
    ValuesWriter delta = new DeltaBinaryPackingValuesWriterForInteger(blockSize, miniBlockNum, 100, 20000, new DirectByteBufferAllocator());
    ValuesWriter rle = new RunLengthBitPackingHybridValuesWriter(32, 100, 20000, new DirectByteBufferAllocator());
    for (int i = 0; i < data.length; i++) {
        delta.writeInteger(data[i]);
        rle.writeInteger(data[i]);
    }
    deltaBytes = delta.getBytes().toByteArray();
    rleBytes = rle.getBytes().toByteArray();
}