Examples with KeySequenceConvertFunction org.deeplearning4j.spark.models.paragraphvectors.functions.KeySequenceConvertFunction used on opensource projects

Example 1 with KeySequenceConvertFunction

use of org.deeplearning4j.spark.models.paragraphvectors.functions.KeySequenceConvertFunction in project deeplearning4j by deeplearning4j.

the class SparkParagraphVectors method fitMultipleFiles.

/**
     * This method builds ParagraphVectors model, expecting JavaPairRDD with key as label, and value as document-in-a-string.
     *
     * @param documentsRdd
     */
public void fitMultipleFiles(JavaPairRDD<String, String> documentsRdd) {
    /*
            All we want here, is to transform JavaPairRDD into JavaRDD<Sequence<VocabWord>>
         */
    validateConfiguration();
    broadcastEnvironment(new JavaSparkContext(documentsRdd.context()));
    JavaRDD<Sequence<VocabWord>> sequenceRdd = documentsRdd.map(new KeySequenceConvertFunction(configurationBroadcast));
    super.fitSequences(sequenceRdd);
}