Examples with PipeMapRed org.apache.hadoop.streaming.PipeMapRed used on opensource projects

Example 1 with PipeMapRed

use of org.apache.hadoop.streaming.PipeMapRed in project hadoop by apache.

the class TestKeyOnlyTextOutputReader method testKeyOnlyTextOutputReader.

@Test
public void testKeyOnlyTextOutputReader() throws IOException {
    String text = "key,value\nkey2,value2\nnocomma\n";
    PipeMapRed pipeMapRed = new MyPipeMapRed(text);
    KeyOnlyTextOutputReader outputReader = new KeyOnlyTextOutputReader();
    outputReader.initialize(pipeMapRed);
    outputReader.readKeyValue();
    Assert.assertEquals(new Text("key,value"), outputReader.getCurrentKey());
    outputReader.readKeyValue();
    Assert.assertEquals(new Text("key2,value2"), outputReader.getCurrentKey());
    outputReader.readKeyValue();
    Assert.assertEquals(new Text("nocomma"), outputReader.getCurrentKey());
    Assert.assertEquals(false, outputReader.readKeyValue());
}

Example 2 with PipeMapRed

use of org.apache.hadoop.streaming.PipeMapRed in project mongo-hadoop by mongodb.

the class MongoUpdateOutputReaderTest method testNoUpdate.

@Test
public void testNoUpdate() throws IOException {
    // Test document, does not describe an update.
    DBObject notAnUpdate = new BasicDBObject("_id", 42);
    PipeMapRed pipeMapRed = mock(PipeMapRed.class);
    when(pipeMapRed.getClientInput()).thenReturn(inputFromBSONObject(notAnUpdate));
    MongoUpdateOutputReader reader = new MongoUpdateOutputReader();
    reader.initialize(pipeMapRed);
    assertTrue(reader.readKeyValue());
    assertEquals(notAnUpdate, reader.getCurrentValue().getQuery());
}

Example 3 with PipeMapRed

use of org.apache.hadoop.streaming.PipeMapRed in project mongo-hadoop by mongodb.

the class MongoUpdateOutputReaderTest method testUpdate.

@Test
public void testUpdate() throws IOException {
    BasicBSONObject query = new BasicDBObject("i", 42);
    BasicBSONObject modifiers = new BasicDBObject("$set", new BasicDBObject("a", "b"));
    DBObject update = new BasicDBObjectBuilder().add("_id", query).add("modifiers", modifiers).push("options").add("multi", true).add("upsert", false).pop().get();
    MongoUpdateWritable muw = new MongoUpdateWritable(query, modifiers, false, true, false);
    PipeMapRed pipeMapRed = mock(PipeMapRed.class);
    when(pipeMapRed.getClientInput()).thenReturn(inputFromBSONObject(update));
    MongoUpdateOutputReader reader = new MongoUpdateOutputReader();
    reader.initialize(pipeMapRed);
    assertTrue(reader.readKeyValue());
    assertEquals(muw, reader.getCurrentValue());
}