Examples with SRXSentenceTokenizer org.languagetool.tokenizers.SRXSentenceTokenizer used on opensource projects

Example 1 with SRXSentenceTokenizer

use of org.languagetool.tokenizers.SRXSentenceTokenizer in project languagetool by languagetool-org.

the class EnglishDisambiguationRuleTest method setUp.

@Before
public void setUp() {
    tagger = new EnglishTagger();
    tokenizer = new WordTokenizer();
    sentenceTokenizer = new SRXSentenceTokenizer(new English());
    disambiguator = new XmlRuleDisambiguator(new English());
    disamb2 = new DemoDisambiguator();
}

Example 2 with SRXSentenceTokenizer

use of org.languagetool.tokenizers.SRXSentenceTokenizer in project languagetool by languagetool-org.

the class UkrainianDisambiguationRuleTest method setUp.

@Before
public void setUp() {
    tagger = new UkrainianTagger();
    tokenizer = new UkrainianWordTokenizer();
    sentenceTokenizer = new SRXSentenceTokenizer(new Ukrainian());
    disambiguator = new UkrainianHybridDisambiguator();
    demoDisambiguator = new DemoDisambiguator();
    chunker = new MultiWordChunker("/uk/multiwords.txt", true);
}

Example 3 with SRXSentenceTokenizer

use of org.languagetool.tokenizers.SRXSentenceTokenizer in project languagetool by languagetool-org.

the class FrenchRuleDisambiguatorTest method setUp.

@Before
public void setUp() throws IOException {
    tagger = new FrenchTagger();
    tokenizer = new WordTokenizer();
    French language = new French();
    sentenceTokenizer = new SRXSentenceTokenizer(language);
    disambiguator = new XmlRuleDisambiguator(language);
    disamb2 = new DemoDisambiguator();
}

Example 4 with SRXSentenceTokenizer

use of org.languagetool.tokenizers.SRXSentenceTokenizer in project languagetool by languagetool-org.

the class CatalanDisambiguationRuleTest method setUp.

@Before
public void setUp() {
    tagger = new CatalanTagger();
    tokenizer = new CatalanWordTokenizer();
    sentenceTokenizer = new SRXSentenceTokenizer(new Catalan());
    disambiguator = new MultiWordChunker("/ca/multiwords.txt", true);
}

Example 5 with SRXSentenceTokenizer

use of org.languagetool.tokenizers.SRXSentenceTokenizer in project languagetool by languagetool-org.

the class RomanianRuleDisambiguatorTest method setUp.

@Before
public void setUp() {
    tagger = new RomanianTagger();
    tokenizer = new RomanianWordTokenizer();
    Romanian language = new Romanian();
    sentenceTokenizer = new SRXSentenceTokenizer(language);
    disambiguator = new XmlRuleDisambiguator(language);
    disamb2 = new DemoDisambiguator();
}