Search in sources :

Example 1 with SortedMapWritable

use of org.apache.hadoop.io.SortedMapWritable in project hadoop by apache.

the class TypedBytesWritableInput method readSortedMap.

public <K extends WritableComparable<? super K>> SortedMapWritable<K> readSortedMap(SortedMapWritable<K> mw) throws IOException {
    if (mw == null) {
        mw = new SortedMapWritable<K>();
    }
    int length = in.readMapHeader();
    for (int i = 0; i < length; i++) {
        @SuppressWarnings("unchecked") K key = (K) read();
        Writable value = read();
        mw.put(key, value);
    }
    return mw;
}
Also used : SortedMapWritable(org.apache.hadoop.io.SortedMapWritable) VLongWritable(org.apache.hadoop.io.VLongWritable) Writable(org.apache.hadoop.io.Writable) MapWritable(org.apache.hadoop.io.MapWritable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) ByteWritable(org.apache.hadoop.io.ByteWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) VIntWritable(org.apache.hadoop.io.VIntWritable) ArrayWritable(org.apache.hadoop.io.ArrayWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) IntWritable(org.apache.hadoop.io.IntWritable)

Example 2 with SortedMapWritable

use of org.apache.hadoop.io.SortedMapWritable in project hive by apache.

the class TypedBytesWritableInput method readSortedMap.

public SortedMapWritable readSortedMap(SortedMapWritable mw) throws IOException {
    if (mw == null) {
        mw = new SortedMapWritable();
    }
    int length = in.readMapHeader();
    for (int i = 0; i < length; i++) {
        WritableComparable key = (WritableComparable) read();
        Writable value = read();
        mw.put(key, value);
    }
    return mw;
}
Also used : WritableComparable(org.apache.hadoop.io.WritableComparable) ByteWritable(org.apache.hadoop.hive.serde2.io.ByteWritable) NullWritable(org.apache.hadoop.io.NullWritable) VLongWritable(org.apache.hadoop.io.VLongWritable) Writable(org.apache.hadoop.io.Writable) MapWritable(org.apache.hadoop.io.MapWritable) LongWritable(org.apache.hadoop.io.LongWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) DoubleWritable(org.apache.hadoop.hive.serde2.io.DoubleWritable) ShortWritable(org.apache.hadoop.hive.serde2.io.ShortWritable) ArrayWritable(org.apache.hadoop.io.ArrayWritable) IntWritable(org.apache.hadoop.io.IntWritable) SortedMapWritable(org.apache.hadoop.io.SortedMapWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) VIntWritable(org.apache.hadoop.io.VIntWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) SortedMapWritable(org.apache.hadoop.io.SortedMapWritable)

Example 3 with SortedMapWritable

use of org.apache.hadoop.io.SortedMapWritable in project mongo-hadoop by mongodb.

the class BSONWritableTest method testToBSON.

@Test
public void testToBSON() {
    assertEquals(null, toBSON(null));
    assertEquals(null, toBSON(NullWritable.get()));
    assertEquals("hello", toBSON(new Text("hello")));
    DBObject obj = new BasicDBObject("hello", "world");
    assertEquals(obj, toBSON(new BSONWritable(obj)));
    final BasicBSONObject bsonResult = new BasicBSONObject("one", 1);
    SortedMapWritable smw = new SortedMapWritable();
    smw.put(new Text("one"), new IntWritable(1));
    assertEquals(bsonResult, toBSON(smw));
    MapWritable mw = new MapWritable();
    mw.put(new Text("one"), new IntWritable(1));
    assertEquals(bsonResult, toBSON(mw));
    String[] expectedObjects = new String[] { "one", "two" };
    Writable[] writableObjects = new Writable[] { new Text("one"), new Text("two") };
    ArrayWritable aw = new ArrayWritable(Text.class, writableObjects);
    Object[] actual = (Object[]) toBSON(aw);
    assertTrue(Arrays.equals(expectedObjects, actual));
    assertEquals(false, toBSON(new BooleanWritable(false)));
    byte[] bytes = new byte[] { '0', '1', '2' };
    assertEquals(bytes, toBSON(new BytesWritable(bytes)));
    byte b = (byte) 'c';
    assertEquals(b, toBSON(new ByteWritable(b)));
    assertEquals(3.14159, toBSON(new DoubleWritable(3.14159)));
    assertEquals(3.14159f, toBSON(new FloatWritable(3.14159f)));
    assertEquals(42L, toBSON(new LongWritable(42L)));
    assertEquals(42, toBSON(new IntWritable(42)));
    // Catchall
    assertEquals("hi", toBSON("hi"));
}
Also used : NullWritable(org.apache.hadoop.io.NullWritable) SortedMapWritable(org.apache.hadoop.io.SortedMapWritable) Writable(org.apache.hadoop.io.Writable) MapWritable(org.apache.hadoop.io.MapWritable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) ByteWritable(org.apache.hadoop.io.ByteWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) ArrayWritable(org.apache.hadoop.io.ArrayWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) IntWritable(org.apache.hadoop.io.IntWritable) Text(org.apache.hadoop.io.Text) BytesWritable(org.apache.hadoop.io.BytesWritable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) SortedMapWritable(org.apache.hadoop.io.SortedMapWritable) SortedMapWritable(org.apache.hadoop.io.SortedMapWritable) MapWritable(org.apache.hadoop.io.MapWritable) BasicDBObject(com.mongodb.BasicDBObject) DBObject(com.mongodb.DBObject) BasicDBObject(com.mongodb.BasicDBObject) BasicBSONObject(org.bson.BasicBSONObject) FloatWritable(org.apache.hadoop.io.FloatWritable) ArrayWritable(org.apache.hadoop.io.ArrayWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) BasicBSONObject(org.bson.BasicBSONObject) BasicDBObject(com.mongodb.BasicDBObject) DBObject(com.mongodb.DBObject) LongWritable(org.apache.hadoop.io.LongWritable) ByteWritable(org.apache.hadoop.io.ByteWritable) IntWritable(org.apache.hadoop.io.IntWritable) Test(org.junit.Test)

Aggregations

ArrayWritable (org.apache.hadoop.io.ArrayWritable)3 BooleanWritable (org.apache.hadoop.io.BooleanWritable)3 BytesWritable (org.apache.hadoop.io.BytesWritable)3 FloatWritable (org.apache.hadoop.io.FloatWritable)3 IntWritable (org.apache.hadoop.io.IntWritable)3 LongWritable (org.apache.hadoop.io.LongWritable)3 MapWritable (org.apache.hadoop.io.MapWritable)3 SortedMapWritable (org.apache.hadoop.io.SortedMapWritable)3 Writable (org.apache.hadoop.io.Writable)3 ByteWritable (org.apache.hadoop.io.ByteWritable)2 DoubleWritable (org.apache.hadoop.io.DoubleWritable)2 NullWritable (org.apache.hadoop.io.NullWritable)2 VIntWritable (org.apache.hadoop.io.VIntWritable)2 VLongWritable (org.apache.hadoop.io.VLongWritable)2 BasicDBObject (com.mongodb.BasicDBObject)1 DBObject (com.mongodb.DBObject)1 ByteWritable (org.apache.hadoop.hive.serde2.io.ByteWritable)1 DoubleWritable (org.apache.hadoop.hive.serde2.io.DoubleWritable)1 ShortWritable (org.apache.hadoop.hive.serde2.io.ShortWritable)1 Text (org.apache.hadoop.io.Text)1