Examples with UserDefinedFunction org.apache.spark.sql.UserDefinedFunction used on opensource projects

Example 1 with UserDefinedFunction

use of org.apache.spark.sql.UserDefinedFunction in project kylo by Teradata.

the class DataSetProviderUtil method map.

/**
 * Applies the specified function to the specified field of the data set.
 */
@Nonnull
@SuppressWarnings("unchecked")
static DataFrame map(@Nonnull final DataFrame dataSet, @Nonnull final String fieldName, @Nonnull final Function1 function, @Nonnull final DataType returnType) {
    final Seq<Column> inputs = Seq$.MODULE$.<Column>newBuilder().$plus$eq(dataSet.col(fieldName)).result();
    final UserDefinedFunction udf = new UserDefinedFunction(function, returnType, (Seq<DataType>) Seq$.MODULE$.<DataType>empty());
    return dataSet.withColumn(fieldName, udf.apply(inputs));
}

Example 2 with UserDefinedFunction

use of org.apache.spark.sql.UserDefinedFunction in project kylo by Teradata.

the class DataSetProviderUtilV1 method map.

/**
 * Applies the specified function to the specified field of the data set.
 */
@Nonnull
@SuppressWarnings("unchecked")
static DataFrame map(@Nonnull final DataFrame dataSet, @Nonnull final String fieldName, @Nonnull final Function1 function, @Nonnull final DataType returnType) {
    final Seq<Column> inputs = Seq$.MODULE$.<Column>newBuilder().$plus$eq(dataSet.col(fieldName)).result();
    final UserDefinedFunction udf = new UserDefinedFunction(function, returnType, (Seq<DataType>) Seq$.MODULE$.<DataType>empty());
    return dataSet.withColumn(fieldName, udf.apply(inputs));
}