Examples with Linguistics com.yahoo.language.Linguistics used on opensource projects

Example 6 with Linguistics

use of com.yahoo.language.Linguistics in project vespa by vespa-engine.

the class NormalizeTestCase method requireThatHashCodeAndEqualsAreImplemented.

@Test
public void requireThatHashCodeAndEqualsAreImplemented() {
    Linguistics linguistics = new SimpleLinguistics();
    NormalizeExpression exp = new NormalizeExpression(linguistics);
    assertFalse(exp.equals(new Object()));
    assertFalse(exp.equals(new NormalizeExpression(Mockito.mock(Linguistics.class))));
    assertEquals(exp, new NormalizeExpression(linguistics));
    assertEquals(exp.hashCode(), new NormalizeExpression(linguistics).hashCode());
}

Example 7 with Linguistics

use of com.yahoo.language.Linguistics in project vespa by vespa-engine.

the class NormalizeTestCase method requireThatAccessorsWork.

@Test
public void requireThatAccessorsWork() {
    Linguistics linguistics = new SimpleLinguistics();
    NormalizeExpression exp = new NormalizeExpression(linguistics);
    assertSame(linguistics, exp.getLinguistics());
}

Example 8 with Linguistics

use of com.yahoo.language.Linguistics in project vespa by vespa-engine.

the class TokenizeTestCase method requireThatAccessorsWork.

@Test
public void requireThatAccessorsWork() {
    Linguistics linguistics = new SimpleLinguistics();
    AnnotatorConfig config = new AnnotatorConfig();
    TokenizeExpression exp = new TokenizeExpression(linguistics, config);
    assertSame(linguistics, exp.getLinguistics());
    assertSame(config, exp.getConfig());
}

Example 9 with Linguistics

use of com.yahoo.language.Linguistics in project vespa by vespa-engine.

the class LinguisticsAnnotatorTestCase method requireThatTokenizeCappingWorks.

@Test
public void requireThatTokenizeCappingWorks() {
    String shortString = "short string";
    SpanTree spanTree = new SpanTree(SpanTrees.LINGUISTICS);
    spanTree.setStringFieldValue(new StringFieldValue(shortString));
    spanTree.spanList().span(0, 5).annotate(new Annotation(AnnotationTypes.TERM));
    spanTree.spanList().span(6, 6).annotate(new Annotation(AnnotationTypes.TERM));
    StringFieldValue shortValue = new StringFieldValue(shortString);
    Linguistics linguistics = new SimpleLinguistics();
    LinguisticsAnnotator annotator = new LinguisticsAnnotator(linguistics, new AnnotatorConfig().setMaxTokenLength(12));
    assertTrue(annotator.annotate(shortValue));
    assertEquals(spanTree, shortValue.getSpanTree(SpanTrees.LINGUISTICS));
    assertEquals(shortString, shortValue.getSpanTree(SpanTrees.LINGUISTICS).getStringFieldValue().getString());
    StringFieldValue cappedValue = new StringFieldValue(shortString + " a longer string");
    assertTrue(annotator.annotate(cappedValue));
    assertEquals((shortString + " a longer string"), cappedValue.getSpanTree(SpanTrees.LINGUISTICS).getStringFieldValue().getString());
}

Example 10 with Linguistics

use of com.yahoo.language.Linguistics in project vespa by vespa-engine.

the class ExpressionTestCase method assertExpression.

private static void assertExpression(Class expectedClass, String str) throws ParseException {
    Linguistics linguistics = new SimpleLinguistics();
    Expression foo = Expression.fromString(str, linguistics);
    assertEquals(expectedClass, foo.getClass());
    Expression bar = Expression.fromString(foo.toString(), linguistics);
    assertEquals(foo.hashCode(), bar.hashCode());
    assertEquals(foo, bar);
}