Search in sources :

Example 41 with IntsRef

use of org.apache.lucene.util.IntsRef in project lucene-solr by apache.

the class TestAutomaton method assertMatches.

private void assertMatches(Automaton a, String... strings) {
    Set<IntsRef> expected = new HashSet<>();
    for (String s : strings) {
        IntsRefBuilder ints = new IntsRefBuilder();
        expected.add(Util.toUTF32(s, ints));
    }
    assertEquals(expected, TestOperations.getFiniteStrings(Operations.determinize(a, DEFAULT_MAX_DETERMINIZED_STATES)));
}
Also used : IntsRef(org.apache.lucene.util.IntsRef) IntsRefBuilder(org.apache.lucene.util.IntsRefBuilder) HashSet(java.util.HashSet)

Example 42 with IntsRef

use of org.apache.lucene.util.IntsRef in project lucene-solr by apache.

the class FiniteStringsIteratorTest method testSingleString.

public void testSingleString() {
    Automaton a = new Automaton();
    int start = a.createState();
    int end = a.createState();
    a.setAccept(end, true);
    a.addTransition(start, end, 'a', 'a');
    a.finishState();
    Set<IntsRef> accepted = TestOperations.getFiniteStrings(a);
    assertEquals(1, accepted.size());
    IntsRefBuilder intsRef = new IntsRefBuilder();
    intsRef.append('a');
    assertTrue(accepted.contains(intsRef.toIntsRef()));
}
Also used : IntsRef(org.apache.lucene.util.IntsRef) IntsRefBuilder(org.apache.lucene.util.IntsRefBuilder)

Example 43 with IntsRef

use of org.apache.lucene.util.IntsRef in project lucene-solr by apache.

the class FiniteStringsIteratorTest method testFiniteStringsBasic.

/**
   * Basic test for getFiniteStrings
   */
public void testFiniteStringsBasic() {
    Automaton a = Operations.union(Automata.makeString("dog"), Automata.makeString("duck"));
    a = MinimizationOperations.minimize(a, DEFAULT_MAX_DETERMINIZED_STATES);
    FiniteStringsIterator iterator = new FiniteStringsIterator(a);
    List<IntsRef> actual = getFiniteStrings(iterator);
    assertFiniteStringsRecursive(a, actual);
    assertEquals(2, actual.size());
    IntsRefBuilder dog = new IntsRefBuilder();
    Util.toIntsRef(new BytesRef("dog"), dog);
    assertTrue(actual.contains(dog.get()));
    IntsRefBuilder duck = new IntsRefBuilder();
    Util.toIntsRef(new BytesRef("duck"), duck);
    assertTrue(actual.contains(duck.get()));
}
Also used : IntsRef(org.apache.lucene.util.IntsRef) IntsRefBuilder(org.apache.lucene.util.IntsRefBuilder) BytesRef(org.apache.lucene.util.BytesRef)

Example 44 with IntsRef

use of org.apache.lucene.util.IntsRef in project lucene-solr by apache.

the class LimitedFiniteStringsIteratorTest method testSingleton.

public void testSingleton() {
    Automaton a = Automata.makeString("foobar");
    List<IntsRef> actual = getFiniteStrings(new LimitedFiniteStringsIterator(a, 1));
    assertEquals(1, actual.size());
    IntsRefBuilder scratch = new IntsRefBuilder();
    Util.toUTF32("foobar".toCharArray(), 0, 6, scratch);
    assertTrue(actual.contains(scratch.get()));
}
Also used : IntsRef(org.apache.lucene.util.IntsRef) IntsRefBuilder(org.apache.lucene.util.IntsRefBuilder)

Example 45 with IntsRef

use of org.apache.lucene.util.IntsRef in project lucene-solr by apache.

the class TestFSTsMisc method testRandomWords.

private void testRandomWords(int maxNumWords, int numIter) throws IOException {
    Random random = new Random(random().nextLong());
    for (int iter = 0; iter < numIter; iter++) {
        if (VERBOSE) {
            System.out.println("\nTEST: iter " + iter);
        }
        for (int inputMode = 0; inputMode < 2; inputMode++) {
            final int numWords = random.nextInt(maxNumWords + 1);
            Set<IntsRef> termsSet = new HashSet<>();
            IntsRef[] terms = new IntsRef[numWords];
            while (termsSet.size() < numWords) {
                final String term = getRandomString(random);
                termsSet.add(toIntsRef(term, inputMode));
            }
            doTest(inputMode, termsSet.toArray(new IntsRef[termsSet.size()]));
        }
    }
}
Also used : Random(java.util.Random) IntsRef(org.apache.lucene.util.IntsRef) FSTTester.toIntsRef(org.apache.lucene.util.fst.FSTTester.toIntsRef) FSTTester.getRandomString(org.apache.lucene.util.fst.FSTTester.getRandomString) HashSet(java.util.HashSet)

Aggregations

IntsRef (org.apache.lucene.util.IntsRef)63 BytesRef (org.apache.lucene.util.BytesRef)19 IntsRefBuilder (org.apache.lucene.util.IntsRefBuilder)19 HashSet (java.util.HashSet)16 ArrayList (java.util.ArrayList)13 Automaton (org.apache.lucene.util.automaton.Automaton)13 BytesRefBuilder (org.apache.lucene.util.BytesRefBuilder)12 TokenStreamToAutomaton (org.apache.lucene.analysis.TokenStreamToAutomaton)9 IOException (java.io.IOException)7 Directory (org.apache.lucene.store.Directory)7 HashMap (java.util.HashMap)5 FiniteStringsIterator (org.apache.lucene.util.automaton.FiniteStringsIterator)5 BytesReader (org.apache.lucene.util.fst.FST.BytesReader)5 Pair (org.apache.lucene.util.fst.PairOutputs.Pair)5 ByteArrayInputStream (java.io.ByteArrayInputStream)4 FilterInputStream (java.io.FilterInputStream)4 InputStream (java.io.InputStream)4 Map (java.util.Map)4 Random (java.util.Random)4 TokenStream (org.apache.lucene.analysis.TokenStream)4