Examples with LdLocale com.optimaize.langdetect.i18n.LdLocale used on opensource projects

Example 1 with LdLocale

use of com.optimaize.langdetect.i18n.LdLocale in project languagetool by languagetool-org.

the class LanguageIdentifier method detectLanguageCode.

/**
   * @return language or {@code null} if language could not be identified
   */
@Nullable
private String detectLanguageCode(String text) {
    TextObject textObject = textObjectFactory.forText(text);
    Optional<LdLocale> lang = languageDetector.detect(textObject);
    //System.out.println(languageDetector.getProbabilities(textObject));
    if (lang.isPresent()) {
        return lang.get().getLanguage();
    } else {
        return null;
    }
}

Example 2 with LdLocale

use of com.optimaize.langdetect.i18n.LdLocale in project tika by apache.

the class OptimaizeLangDetector method createDetector.

private com.optimaize.langdetect.LanguageDetector createDetector(List<LanguageProfile> languageProfiles) {
    // FUTURE currently the short text algorithm doesn't normalize probabilities until the end, which
    // means you can often get 0 probabilities. So we pick a very short length for this limit.
    LanguageDetectorBuilder builder = LanguageDetectorBuilder.create(NgramExtractors.standard()).shortTextAlgorithm(30).withProfiles(languageProfiles);
    if (languageProbabilities != null) {
        Map<LdLocale, Double> languageWeights = new HashMap<>(languageProbabilities.size());
        for (String language : languageProbabilities.keySet()) {
            Double priority = (double) languageProbabilities.get(language);
            languageWeights.put(LdLocale.fromString(language), priority);
        }
        builder.languagePriorities(languageWeights);
    }
    return builder.build();
}

Example 3 with LdLocale

use of com.optimaize.langdetect.i18n.LdLocale in project tika by apache.

the class OptimaizeLangDetector method loadModels.

@Override
public LanguageDetector loadModels(Set<String> languages) throws IOException {
    // Normalize languages.
    this.languages = new HashSet<>(languages.size());
    for (String language : languages) {
        this.languages.add(LanguageNames.normalizeName(language));
    }
    // TODO what happens if you request a language that has no profile?
    Set<LdLocale> locales = new HashSet<>();
    for (LdLocale locale : BuiltInLanguages.getLanguages()) {
        String languageName = makeLanguageName(locale);
        if (this.languages.contains(languageName)) {
            locales.add(locale);
        }
    }
    detector = createDetector(new LanguageProfileReader().readBuiltIn(locales));
    return this;
}

Example 4 with LdLocale

use of com.optimaize.langdetect.i18n.LdLocale in project KaellyBot by Kaysoro.

the class Translator method getLanguageFrom.

public static Language getLanguageFrom(String source) {
    TextObject textObject = CommonTextObjectFactories.forDetectingOnLargeText().forText(source);
    Optional<LdLocale> lang = getLanguageDetector().detect(textObject);
    if (lang.isPresent())
        for (Language lg : Language.values()) if (lang.get().getLanguage().equals(lg.getAbrev().toLowerCase()))
            return lg;
    return null;
}

Example 5 with LdLocale

use of com.optimaize.langdetect.i18n.LdLocale in project neo4j-nlp by graphaware.

the class LanguageManager method detectLanguage.

public String detectLanguage(String text) {
    if (!initialized) {
        initialize();
    }
    if (text != null) {
        TextObject textObject = textObjectFactory.forText(text);
        Optional<LdLocale> lang = languageDetector.detect(textObject);
        if (lang.isPresent()) {
            return lang.get().getLanguage();
        }
    }
    return LANGUAGE_NA;
}