Search in sources :

Example 1 with SearcherReference

use of org.neo4j.kernel.api.impl.index.SearcherReference in project neo4j by neo4j.

the class FulltextIndexReader method searchLucene.

private ValuesIterator searchLucene(Query query, IndexQueryConstraints constraints, QueryContext context, CursorContext cursorContext, MemoryTracker memoryTracker) {
    try {
        // We are replicating the behaviour of IndexSearcher.search(Query, Collector), which starts out by re-writing the query,
        // then creates a weight based on the query and index reader context, and then we finally search the leaf contexts with
        // the weight we created.
        // The query rewrite does not really depend on any data in the index searcher (we don't produce such queries), so it's fine
        // that we only rewrite the query once with the first searcher in our partition list.
        query = searchers.get(0).getIndexSearcher().rewrite(query);
        boolean includeTransactionState = context.getTransactionStateOrNull() != null && !isEventuallyConsistent(index);
        // If we have transaction state, then we need to make our result collector filter out all results touched by the transaction state.
        // The reason we filter them out entirely, is that we will query the transaction state separately.
        LongPredicate filter = includeTransactionState ? transactionState.isModifiedInTransactionPredicate() : ALWAYS_FALSE;
        List<PreparedSearch> searches = new ArrayList<>(searchers.size() + 1);
        for (SearcherReference searcher : searchers) {
            Neo4jIndexSearcher indexSearcher = searcher.getIndexSearcher();
            searches.add(new PreparedSearch(indexSearcher, filter));
        }
        if (includeTransactionState) {
            SearcherReference reference = transactionState.maybeUpdate(context, cursorContext, memoryTracker);
            searches.add(new PreparedSearch(reference.getIndexSearcher(), ALWAYS_FALSE));
        }
        // The StatsCollector aggregates index statistics across all our partitions.
        // Weights created based on these statistics will produce scores that are comparable across partitions.
        StatsCollector statsCollector = new StatsCollector(searches);
        List<ValuesIterator> results = new ArrayList<>(searches.size());
        for (PreparedSearch search : searches) {
            // Weights are bonded with the top IndexReaderContext of the index searcher that they are created for.
            // That's why we have to create a new StatsCachingIndexSearcher, and a new weight, for every index partition.
            // However, the important thing is that we re-use the statsCollector.
            StatsCachingIndexSearcher statsCachingIndexSearcher = new StatsCachingIndexSearcher(search, statsCollector);
            Weight weight = statsCachingIndexSearcher.createWeight(query, ScoreMode.COMPLETE, 1);
            results.add(search.search(weight, constraints));
        }
        return ScoreEntityIterator.mergeIterators(results);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}
Also used : ArrayList(java.util.ArrayList) IOException(java.io.IOException) LongPredicate(java.util.function.LongPredicate) Weight(org.apache.lucene.search.Weight) SearcherReference(org.neo4j.kernel.api.impl.index.SearcherReference) Neo4jIndexSearcher(org.neo4j.kernel.api.impl.index.partition.Neo4jIndexSearcher) ValuesIterator(org.neo4j.kernel.api.impl.index.collector.ValuesIterator)

Example 2 with SearcherReference

use of org.neo4j.kernel.api.impl.index.SearcherReference in project neo4j by neo4j.

the class LuceneSchemaIndex method createSimpleUniquenessVerifier.

private static UniquenessVerifier createSimpleUniquenessVerifier(List<AbstractIndexPartition> partitions) throws IOException {
    AbstractIndexPartition singlePartition = getFirstPartition(partitions);
    SearcherReference partitionSearcher = singlePartition.acquireSearcher();
    return new SimpleUniquenessVerifier(partitionSearcher);
}
Also used : SimpleUniquenessVerifier(org.neo4j.kernel.api.impl.schema.verification.SimpleUniquenessVerifier) AbstractIndexPartition(org.neo4j.kernel.api.impl.index.partition.AbstractIndexPartition) SearcherReference(org.neo4j.kernel.api.impl.index.SearcherReference)

Example 3 with SearcherReference

use of org.neo4j.kernel.api.impl.index.SearcherReference in project neo4j by neo4j.

the class FulltextIndexReader method countIndexedEntities.

/**
 * When matching entities in the fulltext index there are some special cases that makes it hard to check that entities
 * actually have the expected property values. To match we use the entityId and only take entries that doesn't contain any
 * unexpected properties. But we don't check that expected properties are present, see
 * {@link LuceneFulltextDocumentStructure#newCountEntityEntriesQuery} for more details.
 */
@Override
public long countIndexedEntities(long entityId, CursorContext cursorContext, int[] propertyKeyIds, Value... propertyValues) {
    long count = 0;
    for (SearcherReference searcher : searchers) {
        try {
            String[] propertyKeys = new String[propertyKeyIds.length];
            for (int i = 0; i < propertyKeyIds.length; i++) {
                propertyKeys[i] = getPropertyKeyName(propertyKeyIds[i]);
            }
            Query query = LuceneFulltextDocumentStructure.newCountEntityEntriesQuery(entityId, propertyKeys, propertyValues);
            TotalHitCountCollector collector = new TotalHitCountCollector();
            searcher.getIndexSearcher().search(query, collector);
            count += collector.getTotalHits();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
    return count;
}
Also used : Query(org.apache.lucene.search.Query) ConstantScoreQuery(org.apache.lucene.search.ConstantScoreQuery) WildcardQuery(org.apache.lucene.search.WildcardQuery) TermQuery(org.apache.lucene.search.TermQuery) BooleanQuery(org.apache.lucene.search.BooleanQuery) TermRangeQuery(org.apache.lucene.search.TermRangeQuery) PropertyIndexQuery(org.neo4j.internal.kernel.api.PropertyIndexQuery) TotalHitCountCollector(org.apache.lucene.search.TotalHitCountCollector) ParseException(org.apache.lucene.queryparser.classic.ParseException) IOException(java.io.IOException) IndexReaderCloseException(org.neo4j.kernel.api.impl.schema.reader.IndexReaderCloseException) TokenNotFoundException(org.neo4j.token.api.TokenNotFoundException) IndexNotApplicableKernelException(org.neo4j.internal.kernel.api.exceptions.schema.IndexNotApplicableKernelException) SearcherReference(org.neo4j.kernel.api.impl.index.SearcherReference)

Example 4 with SearcherReference

use of org.neo4j.kernel.api.impl.index.SearcherReference in project neo4j by neo4j.

the class IndexPartitionFactoryTest method createWritablePartition.

@Test
void createWritablePartition() throws Exception {
    try (AbstractIndexPartition indexPartition = new WritableIndexPartitionFactory(() -> IndexWriterConfigs.standard(Config.defaults())).createPartition(testDirectory.homePath(), directory)) {
        try (IndexWriter indexWriter = indexPartition.getIndexWriter()) {
            indexWriter.addDocument(new Document());
            indexWriter.commit();
            indexPartition.maybeRefreshBlocking();
            try (SearcherReference searcher = indexPartition.acquireSearcher()) {
                assertEquals(1, searcher.getIndexSearcher().getIndexReader().numDocs(), "We should be able to see newly added document ");
            }
        }
    }
}
Also used : IndexWriter(org.apache.lucene.index.IndexWriter) Document(org.apache.lucene.document.Document) SearcherReference(org.neo4j.kernel.api.impl.index.SearcherReference) Test(org.junit.jupiter.api.Test)

Aggregations

SearcherReference (org.neo4j.kernel.api.impl.index.SearcherReference)4 IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)1 LongPredicate (java.util.function.LongPredicate)1 Document (org.apache.lucene.document.Document)1 IndexWriter (org.apache.lucene.index.IndexWriter)1 ParseException (org.apache.lucene.queryparser.classic.ParseException)1 BooleanQuery (org.apache.lucene.search.BooleanQuery)1 ConstantScoreQuery (org.apache.lucene.search.ConstantScoreQuery)1 Query (org.apache.lucene.search.Query)1 TermQuery (org.apache.lucene.search.TermQuery)1 TermRangeQuery (org.apache.lucene.search.TermRangeQuery)1 TotalHitCountCollector (org.apache.lucene.search.TotalHitCountCollector)1 Weight (org.apache.lucene.search.Weight)1 WildcardQuery (org.apache.lucene.search.WildcardQuery)1 Test (org.junit.jupiter.api.Test)1 PropertyIndexQuery (org.neo4j.internal.kernel.api.PropertyIndexQuery)1 IndexNotApplicableKernelException (org.neo4j.internal.kernel.api.exceptions.schema.IndexNotApplicableKernelException)1 ValuesIterator (org.neo4j.kernel.api.impl.index.collector.ValuesIterator)1 AbstractIndexPartition (org.neo4j.kernel.api.impl.index.partition.AbstractIndexPartition)1