Search in sources :

Example 1 with IonStructCaseInsensitiveDecorator

use of com.amazon.ionhiveserde.caseinsensitivedecorator.IonStructCaseInsensitiveDecorator in project ion-hive-serde by amzn.

the class IonHiveSerDe method deserialize.

@Override
public Object deserialize(final Writable blob) throws SerDeException {
    final byte[] bytes;
    final int length;
    // TextFormat which produces Text
    if (blob instanceof Text) {
        Text text = (Text) blob;
        // getBytes returns a reference to the current buffer which is only valid up to length
        bytes = text.getBytes();
        length = text.getLength();
    } else if (blob instanceof BytesWritable) {
        BytesWritable bytesWritable = (BytesWritable) blob;
        // getBytes returns a reference to the current buffer which is only valid up to length
        bytes = bytesWritable.getBytes();
        length = bytesWritable.getLength();
    } else {
        throw new SerDeException("Invalid Writable instance, must be either Text or BytesWritable, was " + blob.getClass());
    }
    final IonSystem domFactory = ionFactory.getDomFactory();
    try (final IonReader reader = ionFactory.newReader(bytes, 0, length)) {
        /*
                We're using an IonStruct here because:
                1. We need a key-value store to carry column values
                2. The top-level IonStruct as a context object carries the IonSystem which we use as a ValueFactory in
                   the callbacks created in PathExtractionConfig
                Refer to https://github.com/amzn/ion-hive-serde/issues/61.
            */
        IonStruct struct = domFactory.newEmptyStruct();
        if (!serDeProperties.pathExtractorCaseSensitivity()) {
            struct = new IonStructCaseInsensitiveDecorator(struct);
        }
        final PathExtractor<IonStruct> pathExtractor = serDeProperties.pathExtractor();
        pathExtractor.match(reader, struct);
        return struct;
    } catch (IonException e) {
        // skips if ignoring malformed
        if (serDeProperties.getIgnoreMalformed()) {
            return null;
        }
        throw e;
    } catch (IOException e) {
        throw new SerDeException(e);
    }
}
Also used : IonSystem(com.amazon.ion.IonSystem) IonStruct(com.amazon.ion.IonStruct) IonException(com.amazon.ion.IonException) IonReader(com.amazon.ion.IonReader) IonStructCaseInsensitiveDecorator(com.amazon.ionhiveserde.caseinsensitivedecorator.IonStructCaseInsensitiveDecorator) Text(org.apache.hadoop.io.Text) BytesWritable(org.apache.hadoop.io.BytesWritable) IOException(java.io.IOException) SerDeException(org.apache.hadoop.hive.serde2.SerDeException)

Aggregations

IonException (com.amazon.ion.IonException)1 IonReader (com.amazon.ion.IonReader)1 IonStruct (com.amazon.ion.IonStruct)1 IonSystem (com.amazon.ion.IonSystem)1 IonStructCaseInsensitiveDecorator (com.amazon.ionhiveserde.caseinsensitivedecorator.IonStructCaseInsensitiveDecorator)1 IOException (java.io.IOException)1 SerDeException (org.apache.hadoop.hive.serde2.SerDeException)1 BytesWritable (org.apache.hadoop.io.BytesWritable)1 Text (org.apache.hadoop.io.Text)1