Examples with UkrainianWordTokenizer org.languagetool.tokenizers.uk.UkrainianWordTokenizer used on opensource projects

Example 1 with UkrainianWordTokenizer

use of org.languagetool.tokenizers.uk.UkrainianWordTokenizer in project languagetool by languagetool-org.

the class UkrainianDisambiguationRuleTest method setUp.

@Before
public void setUp() {
    tagger = new UkrainianTagger();
    tokenizer = new UkrainianWordTokenizer();
    sentenceTokenizer = new SRXSentenceTokenizer(new Ukrainian());
    disambiguator = new UkrainianHybridDisambiguator();
    demoDisambiguator = new DemoDisambiguator();
    chunker = new MultiWordChunker("/uk/multiwords.txt", true);
}

Example 2 with UkrainianWordTokenizer

use of org.languagetool.tokenizers.uk.UkrainianWordTokenizer in project languagetool by languagetool-org.

the class UkrainianTaggerTest method setUp.

@Before
public void setUp() {
    tagger = new UkrainianTagger();
    tokenizer = new UkrainianWordTokenizer();
}