Examples with BSONObject org.bson.BSONObject used on opensource projects

Example 91 with BSONObject

use of org.bson.BSONObject in project mongo-hadoop by mongodb.

the class DeviceMapper method map.

@Override
public void map(final Object key, final BSONWritable value, final OutputCollector<Text, Text> output, final Reporter reporter) throws IOException {
    BSONObject val = value.getDoc();
    String keyOut = (String) val.get("owner") + " " + (String) val.get("type");
    keyText.set(keyOut);
    valueText.set(val.get("_id").toString());
    output.collect(keyText, valueText);
}

Example 92 with BSONObject

use of org.bson.BSONObject in project mongo-hadoop by mongodb.

the class TagsMapper method map.

@Override
protected void map(final Object key, final BSONObject value, final Context context) throws IOException, InterruptedException {
    BasicBSONList tags = (BasicBSONList) value.get("tags");
    Text text = new Text();
    value.removeField("tags");
    for (Object tag : tags) {
        text.set((String) tag);
        writable.setDoc(value);
        context.write(text, writable);
    }
}

Example 93 with BSONObject

use of org.bson.BSONObject in project mongo-hadoop by mongodb.

the class TagsReducer method reduce.

@Override
public void reduce(final Text key, final Iterator<BSONWritable> values, final OutputCollector<NullWritable, MongoUpdateWritable> output, final Reporter reporter) throws IOException {
    BasicDBObject query = new BasicDBObject("_id", key.toString());
    ArrayList<BSONObject> books = new ArrayList<BSONObject>();
    while (values.hasNext()) {
        books.add(values.next().getDoc());
    }
    BasicBSONObject update = new BasicBSONObject("$set", new BasicBSONObject("books", books));
    reduceResult.setQuery(query);
    reduceResult.setModifiers(update);
    output.collect(null, reduceResult);
}

Example 94 with BSONObject

use of org.bson.BSONObject in project mongo-hadoop by mongodb.

the class BSONSplitterTest method testReadSplitsForFile.

@Test
public void testReadSplitsForFile() throws IOException {
    Configuration readSplitsConfig = new Configuration(conf);
    SPLITTER.setConf(readSplitsConfig);
    // Only one split if reading splits is disabled.
    MongoConfigUtil.setBSONReadSplits(readSplitsConfig, false);
    SPLITTER.readSplitsForFile(file);
    List<BSONFileSplit> splitsList = SPLITTER.getAllSplits();
    assertEquals(1, splitsList.size());
    BSONFileSplit theSplit = splitsList.get(0);
    assertOneSplit(theSplit);
    // Actually compute splits.
    MongoConfigUtil.setBSONReadSplits(readSplitsConfig, true);
    // Set split size to be really small so we get a lot of them.
    readSplitsConfig.set("mapreduce.input.fileinputformat.split.maxsize", "5000");
    SPLITTER.readSplitsForFile(file);
    splitsList = SPLITTER.getAllSplits();
    // Value found through manual inspection.
    assertEquals(40, splitsList.size());
    // Make sure that all splits start on document boundaries.
    FSDataInputStream stream = fs.open(file.getPath());
    BSONDecoder decoder = new BasicBSONDecoder();
    BSONCallback callback = new BasicBSONCallback();
    for (BSONFileSplit split : splitsList) {
        stream.seek(split.getStart());
        decoder.decode(stream, callback);
        BSONObject doc = (BSONObject) callback.get();
        assertTrue(doc.containsField("_id"));
    }
}

Example 95 with BSONObject

use of org.bson.BSONObject in project mongo-hadoop by mongodb.

the class BSONComparator method compare.

@Override
public int compare(final byte[] b1, final int s1, final int l1, final byte[] b2, final int s2, final int l2) {
    LazyBSONCallback cb = new LazyBSONCallback();
    DECODER.decode(b1, cb);
    BSONObject a = (BSONObject) cb.get();
    cb.reset();
    DECODER.decode(b2, cb);
    BSONObject b = (BSONObject) cb.get();
    return compare(a, b);
}