Search in sources :

Example 1 with LazyNonPrimitive

use of org.apache.hadoop.hive.serde2.lazy.LazyNonPrimitive in project cdap by caskdata.

the class ObjectSerializer method serialize.

public Writable serialize(Object o, ObjectInspector objectInspector) {
    //overwrite field names (as they get lost by Hive)
    StructTypeInfo structTypeInfo = (StructTypeInfo) TypeInfoUtils.getTypeInfoFromObjectInspector(objectInspector);
    structTypeInfo.setAllStructFieldNames(columnNames);
    List<TypeInfo> info = structTypeInfo.getAllStructFieldTypeInfos();
    List<String> names = structTypeInfo.getAllStructFieldNames();
    Map<String, Object> recordMap = new HashMap<>();
    List<Object> recordObjects = ((StructObjectInspector) objectInspector).getStructFieldsDataAsList(o);
    for (int structIndex = 0; structIndex < info.size(); structIndex++) {
        Object obj = recordObjects.get(structIndex);
        TypeInfo objType = info.get(structIndex);
        if (obj instanceof LazyNonPrimitive || obj instanceof LazyPrimitive) {
            // In case the SerDe that deserialized the object is the one of a native table
            recordMap.put(names.get(structIndex), fromLazyObject(objType, obj));
        } else if (obj instanceof Writable) {
            // Native tables sometimes introduce primitive Writable objects at this point
            recordMap.put(names.get(structIndex), fromWritable((Writable) obj));
        } else {
            // In case the deserializer is the DatasetSerDe
            recordMap.put(names.get(structIndex), serialize(obj, objType));
        }
    }
    // TODO Improve serialization logic - CDAP-11
    return new Text(GSON.toJson(recordMap));
}
Also used : HashMap(java.util.HashMap) ByteWritable(org.apache.hadoop.hive.serde2.io.ByteWritable) NullWritable(org.apache.hadoop.io.NullWritable) DateWritable(org.apache.hadoop.hive.serde2.io.DateWritable) Writable(org.apache.hadoop.io.Writable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) ShortWritable(org.apache.hadoop.io.ShortWritable) IntWritable(org.apache.hadoop.io.IntWritable) HiveBaseCharWritable(org.apache.hadoop.hive.serde2.io.HiveBaseCharWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) HiveDecimalWritable(org.apache.hadoop.hive.serde2.io.HiveDecimalWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) Text(org.apache.hadoop.io.Text) MapTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.MapTypeInfo) ListTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) LazyPrimitive(org.apache.hadoop.hive.serde2.lazy.LazyPrimitive) LazyNonPrimitive(org.apache.hadoop.hive.serde2.lazy.LazyNonPrimitive) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)

Aggregations

HashMap (java.util.HashMap)1 ByteWritable (org.apache.hadoop.hive.serde2.io.ByteWritable)1 DateWritable (org.apache.hadoop.hive.serde2.io.DateWritable)1 HiveBaseCharWritable (org.apache.hadoop.hive.serde2.io.HiveBaseCharWritable)1 HiveDecimalWritable (org.apache.hadoop.hive.serde2.io.HiveDecimalWritable)1 TimestampWritable (org.apache.hadoop.hive.serde2.io.TimestampWritable)1 LazyNonPrimitive (org.apache.hadoop.hive.serde2.lazy.LazyNonPrimitive)1 LazyPrimitive (org.apache.hadoop.hive.serde2.lazy.LazyPrimitive)1 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)1 ListTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo)1 MapTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.MapTypeInfo)1 StructTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo)1 TypeInfo (org.apache.hadoop.hive.serde2.typeinfo.TypeInfo)1 BooleanWritable (org.apache.hadoop.io.BooleanWritable)1 BytesWritable (org.apache.hadoop.io.BytesWritable)1 DoubleWritable (org.apache.hadoop.io.DoubleWritable)1 FloatWritable (org.apache.hadoop.io.FloatWritable)1 IntWritable (org.apache.hadoop.io.IntWritable)1 LongWritable (org.apache.hadoop.io.LongWritable)1 NullWritable (org.apache.hadoop.io.NullWritable)1