Examples with PlainValuesWriter org.apache.parquet.column.values.plain.PlainValuesWriter used on opensource projects

Example 11 with PlainValuesWriter

use of org.apache.parquet.column.values.plain.PlainValuesWriter in project parquet-mr by apache.

the class BenchmarkDeltaByteArray method benchmarkSortedStringsWithPlainValuesWriter.

@BenchmarkOptions(benchmarkRounds = 20, warmupRounds = 4)
@Test
public void benchmarkSortedStringsWithPlainValuesWriter() throws IOException {
    PlainValuesWriter writer = new PlainValuesWriter(64 * 1024, 64 * 1024, new DirectByteBufferAllocator());
    BinaryPlainValuesReader reader = new BinaryPlainValuesReader();
    Utils.writeData(writer, sortedVals);
    ByteBufferInputStream data = writer.getBytes().toInputStream();
    Binary[] bin = Utils.readData(reader, data, values.length);
    System.out.println("size " + data.position());
}

Example 12 with PlainValuesWriter

use of org.apache.parquet.column.values.plain.PlainValuesWriter in project parquet-mr by apache.

the class TestDictionary method testZeroValues.

@Test
public void testZeroValues() throws IOException {
    FallbackValuesWriter<PlainIntegerDictionaryValuesWriter, PlainValuesWriter> cw = newPlainIntegerDictionaryValuesWriter(100, 100);
    cw.writeInteger(34);
    cw.writeInteger(34);
    getBytesAndCheckEncoding(cw, PLAIN_DICTIONARY);
    DictionaryValuesReader reader = initDicReader(cw, INT32);
    // pretend there are 100 nulls. what matters is offset = bytes.length.
    // data doesn't matter
    ByteBuffer bytes = ByteBuffer.wrap(new byte[] { 0x00, 0x01, 0x02, 0x03 });
    ByteBufferInputStream stream = ByteBufferInputStream.wrap(bytes);
    stream.skipFully(stream.available());
    reader.initFromPage(100, stream);
}