Search in sources :

Example 1 with DeltaLengthByteArrayValuesWriter

use of org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesWriter in project parquet-mr by apache.

the class BenchmarkDeltaLengthByteArray method benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter.

@BenchmarkOptions(benchmarkRounds = 20, warmupRounds = 4)
@Test
public void benchmarkRandomStringsWithDeltaLengthByteArrayValuesWriter() throws IOException {
    DeltaLengthByteArrayValuesWriter writer = new DeltaLengthByteArrayValuesWriter(64 * 1024, 64 * 1024, new DirectByteBufferAllocator());
    DeltaLengthByteArrayValuesReader reader = new DeltaLengthByteArrayValuesReader();
    Utils.writeData(writer, values);
    ByteBufferInputStream data = writer.getBytes().toInputStream();
    Binary[] bin = Utils.readData(reader, data, values.length);
    System.out.println("size " + data.position());
}
Also used : DirectByteBufferAllocator(org.apache.parquet.bytes.DirectByteBufferAllocator) DeltaLengthByteArrayValuesReader(org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesReader) ByteBufferInputStream(org.apache.parquet.bytes.ByteBufferInputStream) DeltaLengthByteArrayValuesWriter(org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesWriter) Binary(org.apache.parquet.io.api.Binary) Test(org.junit.Test) BenchmarkOptions(com.carrotsearch.junitbenchmarks.BenchmarkOptions)

Aggregations

BenchmarkOptions (com.carrotsearch.junitbenchmarks.BenchmarkOptions)1 ByteBufferInputStream (org.apache.parquet.bytes.ByteBufferInputStream)1 DirectByteBufferAllocator (org.apache.parquet.bytes.DirectByteBufferAllocator)1 DeltaLengthByteArrayValuesReader (org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesReader)1 DeltaLengthByteArrayValuesWriter (org.apache.parquet.column.values.deltalengthbytearray.DeltaLengthByteArrayValuesWriter)1 Binary (org.apache.parquet.io.api.Binary)1 Test (org.junit.Test)1