Examples with UimaTokenizer org.deeplearning4j.text.tokenization.tokenizer.UimaTokenizer used on opensource projects

Example 1 with UimaTokenizer

use of org.deeplearning4j.text.tokenization.tokenizer.UimaTokenizer in project deeplearning4j by deeplearning4j.

the class UimaTokenizerFactory method create.

@Override
public Tokenizer create(String toTokenize) {
    if (toTokenize == null)
        throw new IllegalArgumentException("Unable to proceed; on sentence to tokenize");
    Tokenizer ret = new UimaTokenizer(toTokenize, uimaResource, checkForLabel);
    ret.setTokenPreProcessor(preProcess);
    return ret;
}