Search in sources :

Example 1 with LazyBSONDecoder

use of org.bson.LazyBSONDecoder in project mongo-hadoop by mongodb.

the class BSONFileRecordReader method init.

public void init(final InputSplit inputSplit, final Configuration configuration) throws IOException, InterruptedException {
    this.configuration = configuration;
    fileSplit = (FileSplit) inputSplit;
    if (LOG.isDebugEnabled()) {
        LOG.debug("reading split " + fileSplit);
    }
    Path file = fileSplit.getPath();
    FileSystem fs = file.getFileSystem(configuration);
    CompressionCodec codec = new CompressionCodecFactory(configuration).getCodec(fileSplit.getPath());
    inRaw = fs.open(file, 16 * 1024 * 1024);
    inRaw.seek(startingPosition == BSON_RR_POSITION_NOT_GIVEN ? fileSplit.getStart() : startingPosition);
    if (codec != null) {
        decompressor = CodecPool.getDecompressor(codec);
        in = codec.createInputStream(inRaw, decompressor);
    } else {
        in = inRaw;
    }
    if (MongoConfigUtil.getLazyBSON(configuration)) {
        callback = new LazyBSONCallback();
        decoder = new LazyBSONDecoder();
    } else {
        callback = new BasicBSONCallback();
        decoder = new BasicBSONDecoder();
    }
}
Also used : Path(org.apache.hadoop.fs.Path) BasicBSONCallback(org.bson.BasicBSONCallback) CompressionCodecFactory(org.apache.hadoop.io.compress.CompressionCodecFactory) FileSystem(org.apache.hadoop.fs.FileSystem) LazyBSONCallback(org.bson.LazyBSONCallback) CompressionCodec(org.apache.hadoop.io.compress.CompressionCodec) LazyBSONDecoder(org.bson.LazyBSONDecoder) BasicBSONDecoder(org.bson.BasicBSONDecoder)

Aggregations

FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1 CompressionCodec (org.apache.hadoop.io.compress.CompressionCodec)1 CompressionCodecFactory (org.apache.hadoop.io.compress.CompressionCodecFactory)1 BasicBSONCallback (org.bson.BasicBSONCallback)1 BasicBSONDecoder (org.bson.BasicBSONDecoder)1 LazyBSONCallback (org.bson.LazyBSONCallback)1 LazyBSONDecoder (org.bson.LazyBSONDecoder)1