Search in sources :

Example 1 with DenseMatrix

use of org.apache.spark.ml.linalg.DenseMatrix in project jpmml-sparkml by jpmml.

the class PCAModelConverter method encodeFeatures.

@Override
public List<Feature> encodeFeatures(SparkMLEncoder encoder) {
    PCAModel transformer = getTransformer();
    List<Feature> features = encoder.getFeatures(transformer.getInputCol());
    DenseMatrix pc = transformer.pc();
    if (pc.numRows() != features.size()) {
        throw new IllegalArgumentException();
    }
    List<Feature> result = new ArrayList<>();
    for (int i = 0; i < transformer.getK(); i++) {
        Apply apply = new Apply("sum");
        for (int j = 0; j < features.size(); j++) {
            Feature feature = features.get(j);
            ContinuousFeature continuousFeature = feature.toContinuousFeature();
            Expression expression = continuousFeature.ref();
            Double coefficient = pc.apply(j, i);
            if (!ValueUtil.isOne(coefficient)) {
                expression = PMMLUtil.createApply("*", expression, PMMLUtil.createConstant(coefficient));
            }
            apply.addExpressions(expression);
        }
        DerivedField derivedField = encoder.createDerivedField(formatName(transformer, i), OpType.CONTINUOUS, DataType.DOUBLE, apply);
        result.add(new ContinuousFeature(encoder, derivedField));
    }
    return result;
}
Also used : PCAModel(org.apache.spark.ml.feature.PCAModel) ContinuousFeature(org.jpmml.converter.ContinuousFeature) Expression(org.dmg.pmml.Expression) Apply(org.dmg.pmml.Apply) ArrayList(java.util.ArrayList) Feature(org.jpmml.converter.Feature) ContinuousFeature(org.jpmml.converter.ContinuousFeature) DerivedField(org.dmg.pmml.DerivedField) DenseMatrix(org.apache.spark.ml.linalg.DenseMatrix)

Aggregations

ArrayList (java.util.ArrayList)1 PCAModel (org.apache.spark.ml.feature.PCAModel)1 DenseMatrix (org.apache.spark.ml.linalg.DenseMatrix)1 Apply (org.dmg.pmml.Apply)1 DerivedField (org.dmg.pmml.DerivedField)1 Expression (org.dmg.pmml.Expression)1 ContinuousFeature (org.jpmml.converter.ContinuousFeature)1 Feature (org.jpmml.converter.Feature)1