Search in sources :

Example 1 with WriterVersion

use of parquet.column.ParquetProperties.WriterVersion in project presto by prestodb.

the class ParquetTester method assertRoundTrip.

public void assertRoundTrip(ObjectInspector objectInspector, Iterable<?> writeValues, Iterable<?> readValues, Type type) throws Exception {
    for (WriterVersion version : versions) {
        for (CompressionCodecName compressionCodecName : compressions) {
            try (TempFile tempFile = new TempFile("test", "parquet")) {
                JobConf jobConf = new JobConf();
                jobConf.setEnum(ParquetOutputFormat.COMPRESSION, compressionCodecName);
                jobConf.setBoolean(ParquetOutputFormat.ENABLE_DICTIONARY, true);
                jobConf.setEnum(ParquetOutputFormat.WRITER_VERSION, version);
                writeParquetColumn(jobConf, tempFile.getFile(), compressionCodecName, objectInspector, writeValues.iterator());
                assertFileContents(jobConf, tempFile, readValues, type);
            }
        }
    }
}
Also used : CompressionCodecName(parquet.hadoop.metadata.CompressionCodecName) JobConf(org.apache.hadoop.mapred.JobConf) WriterVersion(parquet.column.ParquetProperties.WriterVersion)

Aggregations

JobConf (org.apache.hadoop.mapred.JobConf)1 WriterVersion (parquet.column.ParquetProperties.WriterVersion)1 CompressionCodecName (parquet.hadoop.metadata.CompressionCodecName)1