Search in sources :

Example 1 with ScoreMode

use of org.apache.lucene.search.ScoreMode in project crate by crate.

the class ShardSplittingQuery method createWeight.

@Override
public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) {
    return new ConstantScoreWeight(this, boost) {

        @Override
        public String toString() {
            return "weight(delete docs query)";
        }

        @Override
        public Scorer scorer(LeafReaderContext context) throws IOException {
            LeafReader leafReader = context.reader();
            FixedBitSet bitSet = new FixedBitSet(leafReader.maxDoc());
            Terms terms = leafReader.terms(RoutingFieldMapper.NAME);
            Predicate<BytesRef> includeInShard = ref -> {
                int targetShardId = OperationRouting.generateShardId(indexMetadata, Uid.decodeId(ref.bytes, ref.offset, ref.length), null);
                return shardId == targetShardId;
            };
            if (terms == null) {
                // by ID and parent and nested all have the same id.
                assert indexMetadata.isRoutingPartitionedIndex() == false;
                findSplitDocs(IdFieldMapper.NAME, includeInShard, leafReader, bitSet::set);
            } else {
                if (indexMetadata.isRoutingPartitionedIndex()) {
                    // this is the heaviest invariant. Here we have to visit all docs stored fields do extract _id and _routing
                    // this this index is routing partitioned.
                    Visitor visitor = new Visitor(leafReader);
                    TwoPhaseIterator twoPhaseIterator = new RoutingPartitionedDocIdSetIterator(visitor);
                    return new ConstantScoreScorer(this, score(), scoreMode, twoPhaseIterator);
                } else {
                    // in the _routing case we first go and find all docs that have a routing value and mark the ones we have to delete
                    findSplitDocs(RoutingFieldMapper.NAME, ref -> {
                        int targetShardId = OperationRouting.generateShardId(indexMetadata, null, ref.utf8ToString());
                        return shardId == targetShardId;
                    }, leafReader, bitSet::set);
                    // with a routing value from the next iteration an delete / select based on the ID.
                    if (terms.getDocCount() != leafReader.maxDoc()) {
                        // this is a special case where some of the docs have no routing values this sucks but it's possible today
                        FixedBitSet hasRoutingValue = new FixedBitSet(leafReader.maxDoc());
                        findSplitDocs(RoutingFieldMapper.NAME, ref -> false, leafReader, hasRoutingValue::set);
                        IntConsumer bitSetConsumer = bitSet::set;
                        findSplitDocs(IdFieldMapper.NAME, includeInShard, leafReader, docId -> {
                            if (hasRoutingValue.get(docId) == false) {
                                bitSetConsumer.accept(docId);
                            }
                        });
                    }
                }
            }
            return new ConstantScoreScorer(this, score(), scoreMode, new BitSetIterator(bitSet, bitSet.length()));
        }

        @Override
        public boolean isCacheable(LeafReaderContext ctx) {
            // anyway.
            return false;
        }
    };
}
Also used : Query(org.apache.lucene.search.Query) ConstantScoreScorer(org.apache.lucene.search.ConstantScoreScorer) BitSetIterator(org.apache.lucene.util.BitSetIterator) IndexMetadata(org.elasticsearch.cluster.metadata.IndexMetadata) Weight(org.apache.lucene.search.Weight) IntConsumer(java.util.function.IntConsumer) StoredFieldVisitor(org.apache.lucene.index.StoredFieldVisitor) FixedBitSet(org.apache.lucene.util.FixedBitSet) RoutingFieldMapper(org.elasticsearch.index.mapper.RoutingFieldMapper) TermsEnum(org.apache.lucene.index.TermsEnum) LeafReaderContext(org.apache.lucene.index.LeafReaderContext) DocIdSetIterator(org.apache.lucene.search.DocIdSetIterator) PostingsEnum(org.apache.lucene.index.PostingsEnum) Terms(org.apache.lucene.index.Terms) Scorer(org.apache.lucene.search.Scorer) OperationRouting(org.elasticsearch.cluster.routing.OperationRouting) Uid(org.elasticsearch.index.mapper.Uid) BytesRef(org.apache.lucene.util.BytesRef) IdFieldMapper(org.elasticsearch.index.mapper.IdFieldMapper) Predicate(java.util.function.Predicate) IOException(java.io.IOException) ScoreMode(org.apache.lucene.search.ScoreMode) LeafReader(org.apache.lucene.index.LeafReader) FieldInfo(org.apache.lucene.index.FieldInfo) ConstantScoreWeight(org.apache.lucene.search.ConstantScoreWeight) TwoPhaseIterator(org.apache.lucene.search.TwoPhaseIterator) IndexSearcher(org.apache.lucene.search.IndexSearcher) BitSetIterator(org.apache.lucene.util.BitSetIterator) TwoPhaseIterator(org.apache.lucene.search.TwoPhaseIterator) LeafReader(org.apache.lucene.index.LeafReader) StoredFieldVisitor(org.apache.lucene.index.StoredFieldVisitor) Terms(org.apache.lucene.index.Terms) ConstantScoreWeight(org.apache.lucene.search.ConstantScoreWeight) IntConsumer(java.util.function.IntConsumer) ConstantScoreScorer(org.apache.lucene.search.ConstantScoreScorer) FixedBitSet(org.apache.lucene.util.FixedBitSet) LeafReaderContext(org.apache.lucene.index.LeafReaderContext) BytesRef(org.apache.lucene.util.BytesRef)

Example 2 with ScoreMode

use of org.apache.lucene.search.ScoreMode in project OpenGrok by OpenGrok.

the class SuggesterSearcher method getComplexQueryData.

private ComplexQueryData getComplexQueryData(final Query query, final LeafReaderContext leafReaderContext) {
    ComplexQueryData data = new ComplexQueryData();
    if (query == null || query instanceof SuggesterQuery) {
        data.documentIds = new BitIntsHolder(0);
        return data;
    }
    BitIntsHolder documentIds = new BitIntsHolder();
    try {
        search(query, new Collector() {

            @Override
            public LeafCollector getLeafCollector(final LeafReaderContext context) {
                return new LeafCollector() {

                    final int docBase = context.docBase;

                    @Override
                    public void setScorer(final Scorable scorer) {
                        if (leafReaderContext == context) {
                            if (scorer instanceof PhraseScorer) {
                                data.scorer = (PhraseScorer) scorer;
                            } else {
                                try {
                                    // in #setScorer but no better way was found
                                    for (Scorer.ChildScorable childScorer : scorer.getChildren()) {
                                        if (childScorer.child instanceof PhraseScorer) {
                                            data.scorer = (PhraseScorer) childScorer.child;
                                        }
                                    }
                                } catch (Exception e) {
                                // ignore
                                }
                            }
                        }
                    }

                    @Override
                    public void collect(int doc) {
                        if (leafReaderContext == context) {
                            documentIds.set(docBase + doc);
                        }
                    }
                };
            }

            @Override
            public ScoreMode scoreMode() {
                return ScoreMode.COMPLETE_NO_SCORES;
            }
        });
    } catch (IOException e) {
        if (Thread.currentThread().isInterrupted()) {
            interrupted = true;
            return null;
        } else {
            logger.log(Level.WARNING, "Could not get document ids for " + query, e);
        }
    } catch (Exception e) {
        logger.log(Level.WARNING, "Could not get document ids for " + query, e);
    }
    data.documentIds = documentIds;
    return data;
}
Also used : ScoreMode(org.apache.lucene.search.ScoreMode) BitIntsHolder(org.opengrok.suggest.query.data.BitIntsHolder) Scorable(org.apache.lucene.search.Scorable) SuggesterQuery(org.opengrok.suggest.query.SuggesterQuery) IOException(java.io.IOException) PhraseScorer(org.opengrok.suggest.query.PhraseScorer) IOException(java.io.IOException) LeafCollector(org.apache.lucene.search.LeafCollector) LeafCollector(org.apache.lucene.search.LeafCollector) Collector(org.apache.lucene.search.Collector) LeafReaderContext(org.apache.lucene.index.LeafReaderContext)

Aggregations

IOException (java.io.IOException)2 LeafReaderContext (org.apache.lucene.index.LeafReaderContext)2 ScoreMode (org.apache.lucene.search.ScoreMode)2 IntConsumer (java.util.function.IntConsumer)1 Predicate (java.util.function.Predicate)1 FieldInfo (org.apache.lucene.index.FieldInfo)1 LeafReader (org.apache.lucene.index.LeafReader)1 PostingsEnum (org.apache.lucene.index.PostingsEnum)1 StoredFieldVisitor (org.apache.lucene.index.StoredFieldVisitor)1 Terms (org.apache.lucene.index.Terms)1 TermsEnum (org.apache.lucene.index.TermsEnum)1 Collector (org.apache.lucene.search.Collector)1 ConstantScoreScorer (org.apache.lucene.search.ConstantScoreScorer)1 ConstantScoreWeight (org.apache.lucene.search.ConstantScoreWeight)1 DocIdSetIterator (org.apache.lucene.search.DocIdSetIterator)1 IndexSearcher (org.apache.lucene.search.IndexSearcher)1 LeafCollector (org.apache.lucene.search.LeafCollector)1 Query (org.apache.lucene.search.Query)1 Scorable (org.apache.lucene.search.Scorable)1 Scorer (org.apache.lucene.search.Scorer)1