Search in sources :

Example 1 with LazyBSONCallback

use of org.bson.LazyBSONCallback in project mongo-hadoop by mongodb.

the class BSONFileRecordReader method init.

public void init(final InputSplit inputSplit, final Configuration configuration) throws IOException, InterruptedException {
    this.configuration = configuration;
    fileSplit = (FileSplit) inputSplit;
    if (LOG.isDebugEnabled()) {
        LOG.debug("reading split " + fileSplit);
    }
    Path file = fileSplit.getPath();
    FileSystem fs = file.getFileSystem(configuration);
    CompressionCodec codec = new CompressionCodecFactory(configuration).getCodec(fileSplit.getPath());
    inRaw = fs.open(file, 16 * 1024 * 1024);
    inRaw.seek(startingPosition == BSON_RR_POSITION_NOT_GIVEN ? fileSplit.getStart() : startingPosition);
    if (codec != null) {
        decompressor = CodecPool.getDecompressor(codec);
        in = codec.createInputStream(inRaw, decompressor);
    } else {
        in = inRaw;
    }
    if (MongoConfigUtil.getLazyBSON(configuration)) {
        callback = new LazyBSONCallback();
        decoder = new LazyBSONDecoder();
    } else {
        callback = new BasicBSONCallback();
        decoder = new BasicBSONDecoder();
    }
}
Also used : Path(org.apache.hadoop.fs.Path) BasicBSONCallback(org.bson.BasicBSONCallback) CompressionCodecFactory(org.apache.hadoop.io.compress.CompressionCodecFactory) FileSystem(org.apache.hadoop.fs.FileSystem) LazyBSONCallback(org.bson.LazyBSONCallback) CompressionCodec(org.apache.hadoop.io.compress.CompressionCodec) LazyBSONDecoder(org.bson.LazyBSONDecoder) BasicBSONDecoder(org.bson.BasicBSONDecoder)

Example 2 with LazyBSONCallback

use of org.bson.LazyBSONCallback in project mongo-hadoop by mongodb.

the class BSONComparator method compare.

@Override
public int compare(final byte[] b1, final int s1, final int l1, final byte[] b2, final int s2, final int l2) {
    LazyBSONCallback cb = new LazyBSONCallback();
    DECODER.decode(b1, cb);
    BSONObject a = (BSONObject) cb.get();
    cb.reset();
    DECODER.decode(b2, cb);
    BSONObject b = (BSONObject) cb.get();
    return compare(a, b);
}
Also used : LazyBSONCallback(org.bson.LazyBSONCallback) LazyBSONObject(org.bson.LazyBSONObject) BasicBSONObject(org.bson.BasicBSONObject) BSONObject(org.bson.BSONObject)

Example 3 with LazyBSONCallback

use of org.bson.LazyBSONCallback in project mongo-java-driver by mongodb.

the class DBObjectCodecTest method shouldEncodedNestedMapsListsAndDocuments.

@Test
public void shouldEncodedNestedMapsListsAndDocuments() {
    // {"0" : 0, "1", 1}
    byte[] zeroOneDocumentBytes = new byte[] { 19, 0, 0, 0, 16, 48, 0, 0, 0, 0, 0, 16, 49, 0, 1, 0, 0, 0, 0 };
    Map<String, Object> zeroOneMap = new HashMap<String, Object>();
    zeroOneMap.put("0", 0);
    zeroOneMap.put("1", 1);
    DBObject zeroOneDBObject = new BasicDBObject();
    zeroOneDBObject.putAll(zeroOneMap);
    DBObject zeroOneDBList = new BasicDBList();
    zeroOneDBList.putAll(zeroOneMap);
    List<Integer> zeroOneList = asList(0, 1);
    DBObjectCodec dbObjectCodec = new DBObjectCodec(fromProviders(asList(new ValueCodecProvider(), new DBObjectCodecProvider(), new BsonValueCodecProvider())));
    DBObject doc = new BasicDBObject().append("map", zeroOneMap).append("dbDocument", zeroOneDBObject).append("dbList", zeroOneDBList).append("list", zeroOneList).append("array", new int[] { 0, 1 }).append("lazyDoc", new LazyDBObject(zeroOneDocumentBytes, new LazyBSONCallback())).append("lazyArray", new LazyDBList(zeroOneDocumentBytes, new LazyBSONCallback()));
    BsonDocumentWriter writer = new BsonDocumentWriter(new BsonDocument());
    dbObjectCodec.encode(writer, doc, EncoderContext.builder().build());
    BsonDocument zeroOneBsonDocument = new BsonDocument().append("0", new BsonInt32(0)).append("1", new BsonInt32(1));
    BsonArray zeroOneBsonArray = new BsonArray(asList(new BsonInt32(0), new BsonInt32(1)));
    assertEquals(new BsonDocument("map", zeroOneBsonDocument).append("dbDocument", zeroOneBsonDocument).append("dbList", zeroOneBsonArray).append("list", zeroOneBsonArray).append("array", zeroOneBsonArray).append("lazyDoc", zeroOneBsonDocument).append("lazyArray", zeroOneBsonArray), writer.getDocument());
}
Also used : HashMap(java.util.HashMap) ValueCodecProvider(org.bson.codecs.ValueCodecProvider) BsonValueCodecProvider(org.bson.codecs.BsonValueCodecProvider) BsonInt32(org.bson.BsonInt32) BsonDocumentWriter(org.bson.BsonDocumentWriter) BsonDocument(org.bson.BsonDocument) LazyBSONCallback(org.bson.LazyBSONCallback) BsonArray(org.bson.BsonArray) BsonValueCodecProvider(org.bson.codecs.BsonValueCodecProvider) Test(org.junit.Test)

Aggregations

LazyBSONCallback (org.bson.LazyBSONCallback)3 HashMap (java.util.HashMap)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 Path (org.apache.hadoop.fs.Path)1 CompressionCodec (org.apache.hadoop.io.compress.CompressionCodec)1 CompressionCodecFactory (org.apache.hadoop.io.compress.CompressionCodecFactory)1 BSONObject (org.bson.BSONObject)1 BasicBSONCallback (org.bson.BasicBSONCallback)1 BasicBSONDecoder (org.bson.BasicBSONDecoder)1 BasicBSONObject (org.bson.BasicBSONObject)1 BsonArray (org.bson.BsonArray)1 BsonDocument (org.bson.BsonDocument)1 BsonDocumentWriter (org.bson.BsonDocumentWriter)1 BsonInt32 (org.bson.BsonInt32)1 LazyBSONDecoder (org.bson.LazyBSONDecoder)1 LazyBSONObject (org.bson.LazyBSONObject)1 BsonValueCodecProvider (org.bson.codecs.BsonValueCodecProvider)1 ValueCodecProvider (org.bson.codecs.ValueCodecProvider)1 Test (org.junit.Test)1