Search in sources :

Example 21 with RootLexicon

use of zemberek.morphology.lexicon.RootLexicon in project zemberek-nlp by ahmetaa.

the class DictionaryOperations method findAbbreviations.

public static void findAbbreviations() throws IOException {
    // TurkishMorphology morphology = TurkishMorphology.createWithDefaults();
    RootLexicon lexicon = TurkishDictionaryLoader.loadFromResources("tr/non-tdk.dict");
    Set<String> set = new HashSet<>();
    for (DictionaryItem item : lexicon) {
        String lemma = item.lemma;
        if (item.attributes.contains(RootAttribute.Dummy)) {
            continue;
        }
        if (item.secondaryPos != SecondaryPos.ProperNoun) {
            continue;
        }
        TurkishAlphabet alphabet = TurkishAlphabet.INSTANCE;
        if (!alphabet.containsVowel(lemma) || (lemma.length() > 3 && !alphabet.containsVowel(lemma.substring(0, 3)))) {
            set.add(lemma + " [P:Abbrv]");
        }
    }
    List<String> list = new ArrayList<>(set);
    list.sort(Turkish.STRING_COMPARATOR_ASC);
    Files.write(Paths.get("zemberek.possible.abbrv2"), list);
}
Also used : DictionaryItem(zemberek.morphology.lexicon.DictionaryItem) TurkishAlphabet(zemberek.core.turkish.TurkishAlphabet) ArrayList(java.util.ArrayList) RootLexicon(zemberek.morphology.lexicon.RootLexicon) HashSet(java.util.HashSet) LinkedHashSet(java.util.LinkedHashSet)

Aggregations

RootLexicon (zemberek.morphology.lexicon.RootLexicon)21 DictionaryItem (zemberek.morphology.lexicon.DictionaryItem)12 Test (org.junit.Test)10 ArrayList (java.util.ArrayList)9 TurkishDictionaryLoader (zemberek.morphology.lexicon.tr.TurkishDictionaryLoader)6 File (java.io.File)4 LinkedHashSet (java.util.LinkedHashSet)4 Stopwatch (com.google.common.base.Stopwatch)3 HashSet (java.util.HashSet)3 Ignore (org.junit.Ignore)3 TurkishMorphology (zemberek.morphology.TurkishMorphology)3 DynamicLexiconGraph (zemberek.morphology.lexicon.graph.DynamicLexiconGraph)3 TurkishMorphotactics (zemberek.morphology.morphotactics.TurkishMorphotactics)3 Path (java.nio.file.Path)2 List (java.util.List)2 Set (java.util.Set)2 ExecutorCompletionService (java.util.concurrent.ExecutorCompletionService)2 ExecutorService (java.util.concurrent.ExecutorService)2 Collectors (java.util.stream.Collectors)2 Assert (org.junit.Assert)2