Examples with CompressionType org.apache.hadoop.io.SequenceFile.CompressionType used on opensource projects

Example 11 with CompressionType

use of org.apache.hadoop.io.SequenceFile.CompressionType in project hive by apache.

the class Utilities method createSequenceWriter.

/**
 * Create a sequencefile output stream based on job configuration Uses user supplied compression
 * flag (rather than obtaining it from the Job Configuration).
 *
 * @param jc
 *          Job configuration
 * @param fs
 *          File System to create file in
 * @param file
 *          Path to be created
 * @param keyClass
 *          Java Class for key
 * @param valClass
 *          Java Class for value
 * @return output stream over the created sequencefile
 */
public static SequenceFile.Writer createSequenceWriter(JobConf jc, FileSystem fs, Path file, Class<?> keyClass, Class<?> valClass, boolean isCompressed, Progressable progressable) throws IOException {
    CompressionCodec codec = null;
    CompressionType compressionType = CompressionType.NONE;
    Class<? extends CompressionCodec> codecClass = null;
    if (isCompressed) {
        compressionType = SequenceFileOutputFormat.getOutputCompressionType(jc);
        codecClass = FileOutputFormat.getOutputCompressorClass(jc, DefaultCodec.class);
        codec = ReflectionUtil.newInstance(codecClass, jc);
    }
    return SequenceFile.createWriter(fs, jc, file, keyClass, valClass, compressionType, codec, progressable);
}