Search in sources :

Example 11 with Trie

use of org.trie4j.Trie in project trie4j by takawitter.

the class BitVectorExp method main.

public static void main(String[] args) throws Exception {
    Trie trie = new PatriciaTrie();
    int c = 0;
    // You can download archive from http://dumps.wikimedia.org/jawiki/latest/
    LapTimer t = new LapTimer();
    for (String word : new WikipediaTitles()) {
        trie.insert(word);
        c++;
        if (c == maxCount)
            break;
    }
    t.lapMillis("trie building done. %d words.", c);
    final BytesSuccinctBitVector bv = new BytesSuccinctBitVector(5000000);
    final AtomicInteger nodeCount = new AtomicInteger();
    Algorithms.traverseByDepth(trie.getRoot(), new NodeVisitor() {

        @Override
        public boolean visit(Node node, int nest) {
            Node[] children = node.getChildren();
            if (children != null) {
                int n = node.getChildren().length;
                for (int i = 0; i < n; i++) {
                    bv.append(true);
                }
            }
            bv.append(false);
            nodeCount.incrementAndGet();
            return true;
        }
    });
    trie = null;
    t.lapMillis("done. %d nodes inserted. do rank and select", nodeCount.intValue());
    for (int i = 0; i < c; i += 100) {
        int count = bv.rank(i, true);
        bv.select(count, true);
    }
    t.lapMillis("done.");
    Thread.sleep(10000);
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) PatriciaTrie(org.trie4j.patricia.PatriciaTrie) Node(org.trie4j.Node) BytesSuccinctBitVector(org.trie4j.bv.BytesSuccinctBitVector) WikipediaTitles(org.trie4j.test.WikipediaTitles) Trie(org.trie4j.Trie) PatriciaTrie(org.trie4j.patricia.PatriciaTrie) LapTimer(org.trie4j.test.LapTimer) NodeVisitor(org.trie4j.NodeVisitor)

Example 12 with Trie

use of org.trie4j.Trie in project trie4j by takawitter.

the class Issue_031 method test_UnsafeDoubleArray.

@Test
public void test_UnsafeDoubleArray() throws Throwable {
    Trie trie = new PatriciaTrie();
    insertLines(trie, FILE_NAME);
    @SuppressWarnings("deprecation") Trie da = new org.trie4j.doublearray.UnsafeDoubleArray(trie);
    Assert.assertFalse(trie.contains("you"));
    Assert.assertFalse(da.contains("you"));
}
Also used : PatriciaTrie(org.trie4j.patricia.PatriciaTrie) MapPatriciaTrie(org.trie4j.patricia.MapPatriciaTrie) MapTrie(org.trie4j.MapTrie) PatriciaTrie(org.trie4j.patricia.PatriciaTrie) Trie(org.trie4j.Trie) MapPatriciaTrie(org.trie4j.patricia.MapPatriciaTrie) Test(org.junit.Test)

Example 13 with Trie

use of org.trie4j.Trie in project trie4j by takawitter.

the class LOUDSTrieTest method test_save_load.

@Test
public void test_save_load() throws Exception {
    String[] words = { "こんにちは", "さようなら", "おはよう", "おおきなかぶ", "おおやまざき" };
    Trie lt = trieWithWords(words);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(baos);
    try {
        oos.writeObject(lt);
    } finally {
        oos.flush();
    }
    lt = (Trie) new ObjectInputStream(new ByteArrayInputStream(baos.toByteArray())).readObject();
    for (String w : words) {
        Assert.assertTrue(lt.contains(w));
    }
    Assert.assertFalse(lt.contains("おやすみなさい"));
    StringBuilder b = new StringBuilder();
    Node[] children = lt.getRoot().getChildren();
    for (Node n : children) {
        char[] letters = n.getLetters();
        b.append(letters[0]);
    }
    Assert.assertEquals("おこさ", b.toString());
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) Node(org.trie4j.Node) ByteArrayOutputStream(java.io.ByteArrayOutputStream) ObjectOutputStream(java.io.ObjectOutputStream) Trie(org.trie4j.Trie) ObjectInputStream(java.io.ObjectInputStream) AbstractImmutableTrieTest(org.trie4j.AbstractImmutableTrieTest) Test(org.junit.Test)

Example 14 with Trie

use of org.trie4j.Trie in project trie4j by takawitter.

the class LOUDSTrieTest method test.

@Test
public void test() throws Exception {
    String[] words = { "こんにちは", "さようなら", "おはよう", "おおきなかぶ", "おおやまざき" };
    Trie lt = trieWithWords(words);
    for (String w : words) {
        Assert.assertTrue(w, lt.contains(w));
    }
    Assert.assertFalse(lt.contains("おやすみなさい"));
    StringBuilder b = new StringBuilder();
    Node[] children = lt.getRoot().getChildren();
    for (Node n : children) {
        char[] letters = n.getLetters();
        b.append(letters[0]);
    }
    Assert.assertEquals("おこさ", b.toString());
}
Also used : Node(org.trie4j.Node) Trie(org.trie4j.Trie) AbstractImmutableTrieTest(org.trie4j.AbstractImmutableTrieTest) Test(org.junit.Test)

Example 15 with Trie

use of org.trie4j.Trie in project trie4j by takawitter.

the class LongsConstantTimeSelect0TailLOUDSPPTrieWithSuffixTrieDenseTailArrayWikipediaSerializeTest method secondTrie.

@Override
protected Trie secondTrie(Trie firstTrie) {
    Trie t = new TailLOUDSTrie(firstTrie, new LOUDSPPBvTree(new LongsRank0OnlySuccinctBitVector(), new LongsConstantTimeSelect0SuccinctBitVector()), new SuffixTrieDenseTailArrayBuilder());
    t.trimToSize();
    return t;
}
Also used : LOUDSPPBvTree(org.trie4j.louds.bvtree.LOUDSPPBvTree) SuffixTrieDenseTailArrayBuilder(org.trie4j.tail.SuffixTrieDenseTailArrayBuilder) Trie(org.trie4j.Trie) LongsConstantTimeSelect0SuccinctBitVector(org.trie4j.bv.LongsConstantTimeSelect0SuccinctBitVector) LongsRank0OnlySuccinctBitVector(org.trie4j.bv.LongsRank0OnlySuccinctBitVector)

Aggregations

Trie (org.trie4j.Trie)21 Test (org.junit.Test)15 PatriciaTrie (org.trie4j.patricia.PatriciaTrie)11 Node (org.trie4j.Node)10 AbstractTermIdTrieTest (org.trie4j.AbstractTermIdTrieTest)7 ByteArrayInputStream (java.io.ByteArrayInputStream)5 ByteArrayOutputStream (java.io.ByteArrayOutputStream)5 LOUDSPPBvTree (org.trie4j.louds.bvtree.LOUDSPPBvTree)5 WikipediaTitles (org.trie4j.test.WikipediaTitles)5 ObjectInputStream (java.io.ObjectInputStream)4 ObjectOutputStream (java.io.ObjectOutputStream)4 SuffixTrieDenseTailArrayBuilder (org.trie4j.tail.SuffixTrieDenseTailArrayBuilder)4 LapTimer (org.trie4j.test.LapTimer)4 PrintWriter (java.io.PrintWriter)3 MapTrie (org.trie4j.MapTrie)3 MapPatriciaTrie (org.trie4j.patricia.MapPatriciaTrie)3 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)2 AbstractImmutableTrieTest (org.trie4j.AbstractImmutableTrieTest)2 NodeVisitor (org.trie4j.NodeVisitor)2 LongsRank0OnlySuccinctBitVector (org.trie4j.bv.LongsRank0OnlySuccinctBitVector)2