Search in sources :

Example 1 with SliceDictionaryColumnWriter

use of io.prestosql.orc.writer.SliceDictionaryColumnWriter in project hetu-core by openlookeng.

the class TestSliceDictionaryColumnWriter method testDirectConversion.

@Test
public void testDirectConversion() {
    SliceDictionaryColumnWriter writer = new SliceDictionaryColumnWriter(ROOT_COLUMN, VARCHAR, CompressionKind.NONE, toIntExact(DEFAULT_MAX_COMPRESSION_BUFFER_SIZE.toBytes()), DEFAULT_MAX_STRING_STATISTICS_LIMIT);
    // a single row group exceeds 2G after direct conversion
    byte[] value = new byte[megabytes(1)];
    ThreadLocalRandom.current().nextBytes(value);
    Block data = RunLengthEncodedBlock.create(VARCHAR, Slices.wrappedBuffer(value), 3000);
    writer.beginRowGroup();
    writer.writeBlock(data);
    writer.finishRowGroup();
    assertFalse(writer.tryConvertToDirect(megabytes(64)).isPresent());
}
Also used : SliceDictionaryColumnWriter(io.prestosql.orc.writer.SliceDictionaryColumnWriter) RunLengthEncodedBlock(io.prestosql.spi.block.RunLengthEncodedBlock) Block(io.prestosql.spi.block.Block) Test(org.testng.annotations.Test)

Aggregations

SliceDictionaryColumnWriter (io.prestosql.orc.writer.SliceDictionaryColumnWriter)1 Block (io.prestosql.spi.block.Block)1 RunLengthEncodedBlock (io.prestosql.spi.block.RunLengthEncodedBlock)1 Test (org.testng.annotations.Test)1