Examples with IDFModel org.apache.spark.ml.feature.IDFModel used on opensource projects

Example 1 with IDFModel

use of org.apache.spark.ml.feature.IDFModel in project jpmml-sparkml by jpmml.

the class IDFModelConverter method encodeFeatures.

@Override
public List<Feature> encodeFeatures(SparkMLEncoder encoder) {
    IDFModel transformer = getTransformer();
    List<Feature> features = encoder.getFeatures(transformer.getInputCol());
    Vector idf = transformer.idf();
    if (idf.size() != features.size()) {
        throw new IllegalArgumentException();
    }
    List<Feature> result = new ArrayList<>();
    for (int i = 0; i < features.size(); i++) {
        Feature feature = features.get(i);
        TermFeature termFeature = (TermFeature) feature;
        result.add(termFeature.toWeightedTermFeature(idf.apply(i)));
    }
    return result;
}