Examples with LinguisticsAnnotator com.yahoo.vespa.indexinglanguage.linguistics.LinguisticsAnnotator used on opensource projects

Example 1 with LinguisticsAnnotator

use of com.yahoo.vespa.indexinglanguage.linguistics.LinguisticsAnnotator in project vespa by vespa-engine.

the class TokenizeExpression method doExecute.

@Override
protected void doExecute(ExecutionContext context) {
    StringFieldValue input = (StringFieldValue) context.getValue();
    StringFieldValue output = input.clone();
    context.setValue(output);
    AnnotatorConfig cfg = new AnnotatorConfig(config);
    Language lang = context.resolveLanguage(linguistics);
    if (lang != null) {
        cfg.setLanguage(lang);
    }
    LinguisticsAnnotator annotator = new LinguisticsAnnotator(linguistics, cfg);
    annotator.annotate(output);
}