Search in sources :

Example 21 with ByteBuffersDirectory

use of org.apache.lucene.store.ByteBuffersDirectory in project jena by apache.

the class TestTextTxnTDB method create.

private Dataset create() {
    Dataset ds1 = factory.create();
    Directory dir = new ByteBuffersDirectory();
    EntityDefinition eDef = new EntityDefinition("iri", "text");
    eDef.setPrimaryPredicate(RDFS.label);
    TextIndex tidx = new TextIndexLucene(dir, new TextIndexConfig(eDef));
    Dataset ds = TextDatasetFactory.create(ds1, tidx);
    return ds;
}
Also used : ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) Directory(org.apache.lucene.store.Directory)

Example 22 with ByteBuffersDirectory

use of org.apache.lucene.store.ByteBuffersDirectory in project jena by apache.

the class TestBuildTextDataset method createCode.

public static Dataset createCode() {
    // Base data
    Dataset ds1 = DatasetFactory.create();
    // Define the index mapping
    EntityDefinition entDef = new EntityDefinition("uri", "text");
    entDef.setPrimaryPredicate(RDFS.label);
    // Lucene, in memory.
    Directory dir = new ByteBuffersDirectory();
    // Join together into a dataset
    Dataset ds = TextDatasetFactory.createLucene(ds1, dir, new TextIndexConfig(entDef));
    return ds;
}
Also used : ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) Directory(org.apache.lucene.store.Directory)

Example 23 with ByteBuffersDirectory

use of org.apache.lucene.store.ByteBuffersDirectory in project OpenGrok by OpenGrok.

the class SuggesterSearcherTest method setUpClass.

@BeforeAll
public static void setUpClass() throws IOException {
    dir = new ByteBuffersDirectory();
    try (IndexWriter iw = new IndexWriter(dir, new IndexWriterConfig())) {
        Document doc1 = new Document();
        Document doc2 = new Document();
        doc1.add(new TextField("test", "opengrok opengrok2", Field.Store.NO));
        doc2.add(new TextField("test", "opengrok test", Field.Store.NO));
        iw.addDocument(doc1);
        iw.addDocument(doc2);
    }
    IndexReader ir = DirectoryReader.open(dir);
    searcher = new SuggesterSearcher(ir, 10);
}
Also used : IndexWriter(org.apache.lucene.index.IndexWriter) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) IndexReader(org.apache.lucene.index.IndexReader) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) BeforeAll(org.junit.jupiter.api.BeforeAll)

Example 24 with ByteBuffersDirectory

use of org.apache.lucene.store.ByteBuffersDirectory in project OpenGrok by OpenGrok.

the class CustomSloppyPhraseScorerTest method test.

public static void test(final int slop, final int offset, final String[] terms, final Integer[] expectedPositions) throws IOException {
    Directory dir = new ByteBuffersDirectory();
    try (IndexWriter iw = new IndexWriter(dir, new IndexWriterConfig())) {
        Document doc = new Document();
        doc.add(new TextField("test", "zero one two three four five six seven eight nine ten", Field.Store.NO));
        iw.addDocument(doc);
    }
    CustomPhraseQuery query = new CustomPhraseQuery(slop, "test", terms);
    query.setOffset(offset);
    try (IndexReader ir = DirectoryReader.open(dir)) {
        IndexSearcher is = new IndexSearcher(ir);
        Weight w = query.createWeight(is, ScoreMode.COMPLETE_NO_SCORES, 1);
        LeafReaderContext context = ir.getContext().leaves().get(0);
        Scorer scorer = w.scorer(context);
        TwoPhaseIterator it = scorer.twoPhaseIterator();
        int correctDoc = -1;
        int docId;
        while ((docId = it.approximation().nextDoc()) != DocIdSetIterator.NO_MORE_DOCS) {
            if (it.matches()) {
                correctDoc = docId;
            }
        }
        BitIntsHolder bs = (BitIntsHolder) ((PhraseScorer) scorer).getPositions(correctDoc);
        assertThat(toSet(bs), contains(expectedPositions));
    }
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) BitIntsHolder(org.opengrok.suggest.query.data.BitIntsHolder) TwoPhaseIterator(org.apache.lucene.search.TwoPhaseIterator) Scorer(org.apache.lucene.search.Scorer) PhraseScorer(org.opengrok.suggest.query.PhraseScorer) Document(org.apache.lucene.document.Document) Weight(org.apache.lucene.search.Weight) IndexWriter(org.apache.lucene.index.IndexWriter) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) IndexReader(org.apache.lucene.index.IndexReader) TextField(org.apache.lucene.document.TextField) LeafReaderContext(org.apache.lucene.index.LeafReaderContext) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) Directory(org.apache.lucene.store.Directory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Example 25 with ByteBuffersDirectory

use of org.apache.lucene.store.ByteBuffersDirectory in project OpenGrok by OpenGrok.

the class SuggesterProjectDataTest method testRemove.

@Test
public void testRemove() throws IOException {
    Directory dir = new ByteBuffersDirectory();
    Path tempDir = Files.createTempDirectory("test");
    try (IndexWriter iw = new IndexWriter(dir, new IndexWriterConfig())) {
        Document doc = new Document();
        doc.add(new TextField("test", "text", Field.Store.NO));
        iw.addDocument(doc);
    }
    SuggesterProjectData data = new SuggesterProjectData(dir, tempDir, false, Collections.singleton("test"));
    data.init();
    data.remove();
    assertFalse(tempDir.toFile().exists());
}
Also used : Path(java.nio.file.Path) IndexWriter(org.apache.lucene.index.IndexWriter) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) TextField(org.apache.lucene.document.TextField) Document(org.apache.lucene.document.Document) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) Directory(org.apache.lucene.store.Directory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) Test(org.junit.jupiter.api.Test)

Aggregations

ByteBuffersDirectory (org.apache.lucene.store.ByteBuffersDirectory)32 IndexWriter (org.apache.lucene.index.IndexWriter)18 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)18 Document (org.apache.lucene.document.Document)15 IndexSearcher (org.apache.lucene.search.IndexSearcher)13 Directory (org.apache.lucene.store.Directory)13 Test (org.junit.Test)11 StandardAnalyzer (org.apache.lucene.analysis.standard.StandardAnalyzer)8 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)6 TextField (org.apache.lucene.document.TextField)6 Before (org.junit.Before)6 BytesRef (org.apache.lucene.util.BytesRef)5 SortedSetDocValuesField (org.apache.lucene.document.SortedSetDocValuesField)4 RandomizedTest (com.carrotsearch.randomizedtesting.RandomizedTest)3 Row (io.crate.data.Row)3 LuceneCollectorExpression (io.crate.expression.reference.doc.lucene.LuceneCollectorExpression)3 ScoreCollectorExpression (io.crate.expression.reference.doc.lucene.ScoreCollectorExpression)3 KeywordAnalyzer (org.apache.lucene.analysis.core.KeywordAnalyzer)3 DirectoryReader (org.apache.lucene.index.DirectoryReader)3 Term (org.apache.lucene.index.Term)3