Search in sources :

Example 1 with ExpressionEncoder

use of org.apache.spark.sql.catalyst.encoders.ExpressionEncoder in project beam by apache.

the class EncoderFactory method fromBeamCoder.

public static <T> Encoder<T> fromBeamCoder(Coder<T> coder) {
    Class<? super T> clazz = coder.getEncodedTypeDescriptor().getRawType();
    ClassTag<T> classTag = ClassTag$.MODULE$.apply(clazz);
    Expression serializer = new EncoderHelpers.EncodeUsingBeamCoder<>(new BoundReference(0, new ObjectType(clazz), true), coder);
    Expression deserializer = new EncoderHelpers.DecodeUsingBeamCoder<>(new Cast(new GetColumnByOrdinal(0, BinaryType), BinaryType, scala.Option.<String>empty()), classTag, coder);
    return new ExpressionEncoder<>(serializer, deserializer, classTag);
}
Also used : Cast(org.apache.spark.sql.catalyst.expressions.Cast) ObjectType(org.apache.spark.sql.types.ObjectType) GetColumnByOrdinal(org.apache.spark.sql.catalyst.analysis.GetColumnByOrdinal) Expression(org.apache.spark.sql.catalyst.expressions.Expression) ExpressionEncoder(org.apache.spark.sql.catalyst.encoders.ExpressionEncoder) BoundReference(org.apache.spark.sql.catalyst.expressions.BoundReference)

Aggregations

GetColumnByOrdinal (org.apache.spark.sql.catalyst.analysis.GetColumnByOrdinal)1 ExpressionEncoder (org.apache.spark.sql.catalyst.encoders.ExpressionEncoder)1 BoundReference (org.apache.spark.sql.catalyst.expressions.BoundReference)1 Cast (org.apache.spark.sql.catalyst.expressions.Cast)1 Expression (org.apache.spark.sql.catalyst.expressions.Expression)1 ObjectType (org.apache.spark.sql.types.ObjectType)1