Search in sources :

Example 1 with TextObjectFactory

use of com.optimaize.langdetect.text.TextObjectFactory in project languagetool by languagetool-org.

the class LanguageDetectionTrainer method main.

public static void main(String[] args) throws IOException {
    if (args.length != 3) {
        System.out.println("Usage: " + LanguageDetectionTrainer.class.getName() + " <languageCode> <plainTextFile> <minimalFrequency>");
        System.exit(1);
    }
    String langCode = args[0];
    String fileName = args[1];
    int minimalFrequency = Integer.parseInt(args[2]);
    String text = IOUtils.toString(new FileReader(fileName));
    TextObjectFactory textObjectFactory = CommonTextObjectFactories.forIndexingCleanText();
    TextObject inputText = textObjectFactory.create().append(text);
    LanguageProfile languageProfile = new LanguageProfileBuilder(langCode).ngramExtractor(NgramExtractors.standard()).minimalFrequency(minimalFrequency).addText(inputText).build();
    // current dir
    File outputDir = new File(System.getProperty("user.dir"));
    new LanguageProfileWriter().writeToDirectory(languageProfile, outputDir);
    System.out.println("Language profile written to " + new File(outputDir, langCode).getAbsolutePath());
}
Also used : TextObject(com.optimaize.langdetect.text.TextObject) LanguageProfile(com.optimaize.langdetect.profiles.LanguageProfile) LanguageProfileBuilder(com.optimaize.langdetect.profiles.LanguageProfileBuilder) LanguageProfileWriter(com.optimaize.langdetect.profiles.LanguageProfileWriter) FileReader(java.io.FileReader) File(java.io.File) TextObjectFactory(com.optimaize.langdetect.text.TextObjectFactory)

Aggregations

LanguageProfile (com.optimaize.langdetect.profiles.LanguageProfile)1 LanguageProfileBuilder (com.optimaize.langdetect.profiles.LanguageProfileBuilder)1 LanguageProfileWriter (com.optimaize.langdetect.profiles.LanguageProfileWriter)1 TextObject (com.optimaize.langdetect.text.TextObject)1 TextObjectFactory (com.optimaize.langdetect.text.TextObjectFactory)1 File (java.io.File)1 FileReader (java.io.FileReader)1