Search in sources :

Example 6 with SubstringItem

use of com.yahoo.prelude.query.SubstringItem in project vespa by vespa-engine.

the class ParseTestCase method testNumberAsSubstring.

@Test
public void testNumberAsSubstring() {
    Item root = tester.assertParsed("*89*", "*89*", Query.Type.ANY);
    assertTrue(root instanceof SubstringItem);
}
Also used : CompositeItem(com.yahoo.prelude.query.CompositeItem) PhraseSegmentItem(com.yahoo.prelude.query.PhraseSegmentItem) PrefixItem(com.yahoo.prelude.query.PrefixItem) Item(com.yahoo.prelude.query.Item) SuffixItem(com.yahoo.prelude.query.SuffixItem) IntItem(com.yahoo.prelude.query.IntItem) OrItem(com.yahoo.prelude.query.OrItem) PhraseItem(com.yahoo.prelude.query.PhraseItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) AndItem(com.yahoo.prelude.query.AndItem) RankItem(com.yahoo.prelude.query.RankItem) WordItem(com.yahoo.prelude.query.WordItem) NotItem(com.yahoo.prelude.query.NotItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) Test(org.junit.Test)

Example 7 with SubstringItem

use of com.yahoo.prelude.query.SubstringItem in project vespa by vespa-engine.

the class YqlParser method instantiateWordItem.

@NonNull
private Item instantiateWordItem(String field, String rawWord, OperatorNode<ExpressionOperator> ast, Class<?> parent, SegmentWhen segmentPolicy, boolean exactMatch, Language language) {
    String wordData = rawWord;
    if (getAnnotation(ast, NFKC, Boolean.class, Boolean.FALSE, "setting for whether to NFKC normalize input data")) {
        // NOTE: If this is set to FALSE (default), we will still NFKC normalize text data
        // during tokenization/segmentation, as that is always turned on also on the indexing side.
        wordData = normalizer.normalize(wordData);
    }
    boolean fromQuery = getAnnotation(ast, IMPLICIT_TRANSFORMS, Boolean.class, Boolean.TRUE, IMPLICIT_TRANSFORMS_DESCRIPTION);
    boolean prefixMatch = getAnnotation(ast, PREFIX, Boolean.class, Boolean.FALSE, "setting for whether to use prefix match of input data");
    boolean suffixMatch = getAnnotation(ast, SUFFIX, Boolean.class, Boolean.FALSE, "setting for whether to use suffix match of input data");
    boolean substrMatch = getAnnotation(ast, SUBSTRING, Boolean.class, Boolean.FALSE, "setting for whether to use substring match of input data");
    Preconditions.checkArgument((prefixMatch ? 1 : 0) + (substrMatch ? 1 : 0) + (suffixMatch ? 1 : 0) < 2, "Only one of prefix, substring and suffix can be set.");
    @NonNull final TaggableItem wordItem;
    if (exactMatch) {
        wordItem = new ExactStringItem(wordData, fromQuery);
    } else if (prefixMatch) {
        wordItem = new PrefixItem(wordData, fromQuery);
    } else if (suffixMatch) {
        wordItem = new SuffixItem(wordData, fromQuery);
    } else if (substrMatch) {
        wordItem = new SubstringItem(wordData, fromQuery);
    } else {
        switch(segmentPolicy) {
            case NEVER:
                wordItem = new WordItem(wordData, fromQuery);
                break;
            case POSSIBLY:
                if (shouldResegmentWord(field, fromQuery)) {
                    wordItem = resegment(field, ast, wordData, fromQuery, parent, language);
                } else {
                    wordItem = new WordItem(wordData, fromQuery);
                }
                break;
            case ALWAYS:
                wordItem = resegment(field, ast, wordData, fromQuery, parent, language);
                break;
            default:
                throw new IllegalArgumentException("Unexpected segmenting rule: " + segmentPolicy);
        }
    }
    if (wordItem instanceof WordItem) {
        prepareWord(field, ast, fromQuery, (WordItem) wordItem);
    }
    if (// mark the language used, unless it's the default
    language != Language.ENGLISH)
        ((Item) wordItem).setLanguage(language);
    return (Item) leafStyleSettings(ast, wordItem);
}
Also used : SuffixItem(com.yahoo.prelude.query.SuffixItem) CompositeItem(com.yahoo.prelude.query.CompositeItem) WordAlternativesItem(com.yahoo.prelude.query.WordAlternativesItem) NullItem(com.yahoo.prelude.query.NullItem) PrefixItem(com.yahoo.prelude.query.PrefixItem) OrItem(com.yahoo.prelude.query.OrItem) PhraseItem(com.yahoo.prelude.query.PhraseItem) TaggableItem(com.yahoo.prelude.query.TaggableItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) AndItem(com.yahoo.prelude.query.AndItem) RankItem(com.yahoo.prelude.query.RankItem) EquivItem(com.yahoo.prelude.query.EquivItem) WeightedSetItem(com.yahoo.prelude.query.WeightedSetItem) PhraseSegmentItem(com.yahoo.prelude.query.PhraseSegmentItem) ExactStringItem(com.yahoo.prelude.query.ExactStringItem) PredicateQueryItem(com.yahoo.prelude.query.PredicateQueryItem) WeakAndItem(com.yahoo.prelude.query.WeakAndItem) ONearItem(com.yahoo.prelude.query.ONearItem) DotProductItem(com.yahoo.prelude.query.DotProductItem) Item(com.yahoo.prelude.query.Item) SuffixItem(com.yahoo.prelude.query.SuffixItem) AndSegmentItem(com.yahoo.prelude.query.AndSegmentItem) SegmentItem(com.yahoo.prelude.query.SegmentItem) IntItem(com.yahoo.prelude.query.IntItem) WandItem(com.yahoo.prelude.query.WandItem) RegExpItem(com.yahoo.prelude.query.RegExpItem) RangeItem(com.yahoo.prelude.query.RangeItem) WordItem(com.yahoo.prelude.query.WordItem) NotItem(com.yahoo.prelude.query.NotItem) NearItem(com.yahoo.prelude.query.NearItem) TaggableItem(com.yahoo.prelude.query.TaggableItem) NonNull(edu.umd.cs.findbugs.annotations.NonNull) SubstringItem(com.yahoo.prelude.query.SubstringItem) WordItem(com.yahoo.prelude.query.WordItem) PrefixItem(com.yahoo.prelude.query.PrefixItem) ExactStringItem(com.yahoo.prelude.query.ExactStringItem) NonNull(edu.umd.cs.findbugs.annotations.NonNull)

Example 8 with SubstringItem

use of com.yahoo.prelude.query.SubstringItem in project vespa by vespa-engine.

the class ParseTestCase method testIndexedSubstring.

@Test
public void testIndexedSubstring() {
    Item root = tester.assertParsed("foo.bar:*substring*", "foo.bar:*substring*", Query.Type.ANY);
    assertTrue(root instanceof SubstringItem);
}
Also used : CompositeItem(com.yahoo.prelude.query.CompositeItem) PhraseSegmentItem(com.yahoo.prelude.query.PhraseSegmentItem) PrefixItem(com.yahoo.prelude.query.PrefixItem) Item(com.yahoo.prelude.query.Item) SuffixItem(com.yahoo.prelude.query.SuffixItem) IntItem(com.yahoo.prelude.query.IntItem) OrItem(com.yahoo.prelude.query.OrItem) PhraseItem(com.yahoo.prelude.query.PhraseItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) AndItem(com.yahoo.prelude.query.AndItem) RankItem(com.yahoo.prelude.query.RankItem) WordItem(com.yahoo.prelude.query.WordItem) NotItem(com.yahoo.prelude.query.NotItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) Test(org.junit.Test)

Example 9 with SubstringItem

use of com.yahoo.prelude.query.SubstringItem in project vespa by vespa-engine.

the class ParseTestCase method testSubstringAndWordTerms.

@Test
public void testSubstringAndWordTerms() {
    Item root = tester.assertParsed("OR foo *substring* bar", "foo *substring* bar", Query.Type.ANY);
    assertTrue(((OrItem) root).getItem(1) instanceof SubstringItem);
}
Also used : CompositeItem(com.yahoo.prelude.query.CompositeItem) PhraseSegmentItem(com.yahoo.prelude.query.PhraseSegmentItem) PrefixItem(com.yahoo.prelude.query.PrefixItem) Item(com.yahoo.prelude.query.Item) SuffixItem(com.yahoo.prelude.query.SuffixItem) IntItem(com.yahoo.prelude.query.IntItem) OrItem(com.yahoo.prelude.query.OrItem) PhraseItem(com.yahoo.prelude.query.PhraseItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) AndItem(com.yahoo.prelude.query.AndItem) RankItem(com.yahoo.prelude.query.RankItem) WordItem(com.yahoo.prelude.query.WordItem) NotItem(com.yahoo.prelude.query.NotItem) SubstringItem(com.yahoo.prelude.query.SubstringItem) OrItem(com.yahoo.prelude.query.OrItem) Test(org.junit.Test)

Aggregations

AndItem (com.yahoo.prelude.query.AndItem)9 CompositeItem (com.yahoo.prelude.query.CompositeItem)9 Item (com.yahoo.prelude.query.Item)9 NotItem (com.yahoo.prelude.query.NotItem)9 OrItem (com.yahoo.prelude.query.OrItem)9 RankItem (com.yahoo.prelude.query.RankItem)9 SubstringItem (com.yahoo.prelude.query.SubstringItem)9 IntItem (com.yahoo.prelude.query.IntItem)8 PhraseItem (com.yahoo.prelude.query.PhraseItem)8 PhraseSegmentItem (com.yahoo.prelude.query.PhraseSegmentItem)8 PrefixItem (com.yahoo.prelude.query.PrefixItem)8 SuffixItem (com.yahoo.prelude.query.SuffixItem)8 WordItem (com.yahoo.prelude.query.WordItem)8 Test (org.junit.Test)7 EquivItem (com.yahoo.prelude.query.EquivItem)2 NearItem (com.yahoo.prelude.query.NearItem)2 NullItem (com.yahoo.prelude.query.NullItem)2 AndSegmentItem (com.yahoo.prelude.query.AndSegmentItem)1 DotProductItem (com.yahoo.prelude.query.DotProductItem)1 ExactStringItem (com.yahoo.prelude.query.ExactStringItem)1