Examples with MmsegTokenizer com.jstarcraft.nlp.lucene.mmseg.MmsegTokenizer used on opensource projects

Example 1 with MmsegTokenizer

use of com.jstarcraft.nlp.lucene.mmseg.MmsegTokenizer in project jstarcraft-nlp by HongZhaoHua.

the class MmsegSegmenterTestCase method getSegmenter.

@Override
protected Tokenizer getSegmenter() {
    Dictionary dictionary = Dictionary.getInstance();
    ComplexSeg complex = new ComplexSeg(dictionary);
    // MMSeg mmSeg = new MMSeg(new StringReader(""), complex);
    MmsegTokenizer tokenizer = new MmsegTokenizer(complex);
    return tokenizer;
}