Examples with BasicBSONDecoder org.bson.BasicBSONDecoder used on opensource projects

Example 6 with BasicBSONDecoder

use of org.bson.BasicBSONDecoder in project mongo-hadoop by mongodb.

the class BSONSplitterTest method testReadSplitsForFile.

@Test
public void testReadSplitsForFile() throws IOException {
    Configuration readSplitsConfig = new Configuration(conf);
    SPLITTER.setConf(readSplitsConfig);
    // Only one split if reading splits is disabled.
    MongoConfigUtil.setBSONReadSplits(readSplitsConfig, false);
    SPLITTER.readSplitsForFile(file);
    List<BSONFileSplit> splitsList = SPLITTER.getAllSplits();
    assertEquals(1, splitsList.size());
    BSONFileSplit theSplit = splitsList.get(0);
    assertOneSplit(theSplit);
    // Actually compute splits.
    MongoConfigUtil.setBSONReadSplits(readSplitsConfig, true);
    // Set split size to be really small so we get a lot of them.
    readSplitsConfig.set("mapreduce.input.fileinputformat.split.maxsize", "5000");
    SPLITTER.readSplitsForFile(file);
    splitsList = SPLITTER.getAllSplits();
    // Value found through manual inspection.
    assertEquals(40, splitsList.size());
    // Make sure that all splits start on document boundaries.
    FSDataInputStream stream = fs.open(file.getPath());
    BSONDecoder decoder = new BasicBSONDecoder();
    BSONCallback callback = new BasicBSONCallback();
    for (BSONFileSplit split : splitsList) {
        stream.seek(split.getStart());
        decoder.decode(stream, callback);
        BSONObject doc = (BSONObject) callback.get();
        assertTrue(doc.containsField("_id"));
    }
}

Example 7 with BasicBSONDecoder

use of org.bson.BasicBSONDecoder in project camel by apache.

the class MongoDbBasicConverters method fromInputStreamToDBObject.

@Converter
public static BasicDBObject fromInputStreamToDBObject(InputStream is, Exchange exchange) {
    BasicDBObject answer = null;
    try {
        byte[] input = IOConverter.toBytes(is);
        if (isBson(input)) {
            BSONCallback callback = new JSONCallback();
            new BasicBSONDecoder().decode(input, callback);
            answer = (BasicDBObject) callback.get();
        } else {
            answer = (BasicDBObject) JSON.parse(IOConverter.toString(input, exchange));
        }
    } catch (Exception e) {
        LOG.warn("String -> DBObject conversion selected, but the following exception occurred. Returning null.", e);
    } finally {
        // we need to make sure to close the input stream
        IOHelper.close(is, "InputStream", LOG);
    }
    return answer;
}