Search in sources :

Example 86 with BasicBSONObject

use of org.bson.BasicBSONObject in project mongo-hadoop by mongodb.

the class JSONPigReplaceTest method testUnnamedArrayReplace.

@Test
public void testUnnamedArrayReplace() throws Exception {
    // create tuple ({("a"), ("b"), ("c")})
    // with schema 'cars:{f:(t:chararray)}'
    DataBag b = bagFactory.newDefaultBag();
    b.add(tupleFactory.newTuple("a"));
    b.add(tupleFactory.newTuple("b"));
    b.add(tupleFactory.newTuple("c"));
    JSONPigReplace j = new JSONPigReplace(new String[] { "{days : [1,2,3], age : 19, cars : '$cars'}" });
    BasicBSONObject[] bs = j.substitute(tupleFactory.newTuple(b), "cars : {f:(t:chararray)}", "t");
    assertNotNull(bs);
    assertTrue(bs.length == 1);
    // should produce
    // { "name" : "Daniel" , "age" : 19 , "property" : { "cars" : [ "a" , "b" , "c"]} , "school" : "Carleton College"}
    BasicBSONObject res = bs[0];
    ArrayList cars = (ArrayList) res.get("cars");
    assertEquals(cars.size(), 3);
    assertEquals(cars.get(0), "a");
}
Also used : BasicBSONObject(org.bson.BasicBSONObject) DataBag(org.apache.pig.data.DataBag) ArrayList(java.util.ArrayList) Test(org.junit.Test)

Example 87 with BasicBSONObject

use of org.bson.BasicBSONObject in project mongo-hadoop by mongodb.

the class JSONPigReplaceTest method testSimpleNestedReplace.

@Test
public void testSimpleNestedReplace() throws Exception {
    // create tuple ({("Daniel", "Alabi")}, "Carleton College")
    // with schema 'b:{t:(f:chararray,l:chararray)}, s:chararray'
    Tuple t1 = tupleFactory.newTuple(2);
    t1.set(0, "Daniel");
    t1.set(1, "Alabi");
    DataBag b = bagFactory.newDefaultBag();
    b.add(t1);
    Tuple t = tupleFactory.newTuple(2);
    t.set(0, b);
    t.set(1, "Carleton College");
    JSONPigReplace j = new JSONPigReplace(new String[] { "{first:'$f', last:'$l', school:'$s'}" });
    BasicBSONObject[] bs = j.substitute(t, "b:{t:(f:chararray,l:chararray)}, s:chararray", null);
    assertNotNull(bs);
    assertTrue(bs.length == 1);
    // should produce
    // { "first" : "Daniel" , "last" : "Alabi" , "school" : "Carleton College"}
    BasicBSONObject res = bs[0];
    assertEquals(res.get("first"), "Daniel");
    assertEquals(res.get("last"), "Alabi");
    assertEquals(res.get("school"), "Carleton College");
}
Also used : BasicBSONObject(org.bson.BasicBSONObject) DataBag(org.apache.pig.data.DataBag) Tuple(org.apache.pig.data.Tuple) Test(org.junit.Test)

Example 88 with BasicBSONObject

use of org.bson.BasicBSONObject in project mongo-hadoop by mongodb.

the class MongoUpdateOutputReaderTest method testUpdate.

@Test
public void testUpdate() throws IOException {
    BasicBSONObject query = new BasicDBObject("i", 42);
    BasicBSONObject modifiers = new BasicDBObject("$set", new BasicDBObject("a", "b"));
    DBObject update = new BasicDBObjectBuilder().add("_id", query).add("modifiers", modifiers).push("options").add("multi", true).add("upsert", false).pop().get();
    MongoUpdateWritable muw = new MongoUpdateWritable(query, modifiers, false, true, false);
    PipeMapRed pipeMapRed = mock(PipeMapRed.class);
    when(pipeMapRed.getClientInput()).thenReturn(inputFromBSONObject(update));
    MongoUpdateOutputReader reader = new MongoUpdateOutputReader();
    reader.initialize(pipeMapRed);
    assertTrue(reader.readKeyValue());
    assertEquals(muw, reader.getCurrentValue());
}
Also used : BasicBSONObject(org.bson.BasicBSONObject) BasicDBObject(com.mongodb.BasicDBObject) BasicDBObjectBuilder(com.mongodb.BasicDBObjectBuilder) MongoUpdateWritable(com.mongodb.hadoop.io.MongoUpdateWritable) BasicDBObject(com.mongodb.BasicDBObject) DBObject(com.mongodb.DBObject) PipeMapRed(org.apache.hadoop.streaming.PipeMapRed) Test(org.junit.Test)

Aggregations

BasicBSONObject (org.bson.BasicBSONObject)88 Test (org.junit.Test)39 BSONObject (org.bson.BSONObject)37 ArrayList (java.util.ArrayList)15 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)14 BSONWritable (com.mongodb.hadoop.io.BSONWritable)13 ListObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector)13 MapObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector)13 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)13 BasicDBObject (com.mongodb.BasicDBObject)11 ObjectId (org.bson.types.ObjectId)11 IOException (java.io.IOException)8 ByteArrayInputStream (java.io.ByteArrayInputStream)6 DataBag (org.apache.pig.data.DataBag)6 Map (java.util.Map)5 Tuple (org.apache.pig.data.Tuple)5 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 Mongo (com.mongodb.Mongo)4 Date (java.util.Date)4 DoubleWritable (org.apache.hadoop.io.DoubleWritable)4