Search in sources :

Example 16 with SortedNumericSortField

use of org.apache.lucene.search.SortedNumericSortField in project crate by crate.

the class SortSymbolVisitor method mappedSortField.

private static SortField mappedSortField(Reference symbol, MappedFieldType fieldType, boolean reverse, NullValueOrder nullValueOrder) {
    String fieldName = symbol.column().fqn();
    MultiValueMode sortMode = reverse ? MultiValueMode.MAX : MultiValueMode.MIN;
    switch(symbol.valueType().id()) {
        case StringType.ID:
            {
                SortField sortField = new SortedSetSortField(fieldName, reverse, sortMode == MultiValueMode.MAX ? SortedSetSelector.Type.MAX : SortedSetSelector.Type.MIN);
                sortField.setMissingValue(nullValueOrder == NullValueOrder.LAST ^ reverse ? SortedSetSortField.STRING_LAST : SortedSetSortField.STRING_FIRST);
                return sortField;
            }
        case BooleanType.ID:
        case ByteType.ID:
        case ShortType.ID:
        case IntegerType.ID:
        case LongType.ID:
        case TimestampType.ID_WITHOUT_TZ:
        case TimestampType.ID_WITH_TZ:
            {
                SortedNumericSelector.Type selectorType = sortMode == MultiValueMode.MAX ? SortedNumericSelector.Type.MAX : SortedNumericSelector.Type.MIN;
                var reducedType = SortField.Type.LONG;
                var sortField = new SortedNumericSortField(fieldName, SortField.Type.LONG, reverse, selectorType);
                sortField.setMissingValue(NullSentinelValues.nullSentinelForReducedType(reducedType, nullValueOrder, reverse));
                return sortField;
            }
        case FloatType.ID:
            {
                SortedNumericSelector.Type selectorType = sortMode == MultiValueMode.MAX ? SortedNumericSelector.Type.MAX : SortedNumericSelector.Type.MIN;
                var reducedType = SortField.Type.FLOAT;
                var sortField = new SortedNumericSortField(fieldName, SortField.Type.FLOAT, reverse, selectorType);
                sortField.setMissingValue(NullSentinelValues.nullSentinelForReducedType(reducedType, nullValueOrder, reverse));
                return sortField;
            }
        case DoubleType.ID:
            {
                SortedNumericSelector.Type selectorType = sortMode == MultiValueMode.MAX ? SortedNumericSelector.Type.MAX : SortedNumericSelector.Type.MIN;
                var reducedType = SortField.Type.DOUBLE;
                var sortField = new SortedNumericSortField(fieldName, SortField.Type.DOUBLE, reverse, selectorType);
                sortField.setMissingValue(NullSentinelValues.nullSentinelForReducedType(reducedType, nullValueOrder, reverse));
                return sortField;
            }
        case GeoPointType.ID:
            throw new IllegalArgumentException("can't sort on geo_point field without using specific sorting feature, like geo_distance");
        default:
            throw new UnsupportedOperationException("Cannot order on " + symbol + "::" + symbol.valueType());
    }
}
Also used : SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) LongType(io.crate.types.LongType) DoubleType(io.crate.types.DoubleType) FloatType(io.crate.types.FloatType) MappedFieldType(org.elasticsearch.index.mapper.MappedFieldType) IntegerType(io.crate.types.IntegerType) ByteType(io.crate.types.ByteType) StringType(io.crate.types.StringType) GeoPointType(io.crate.types.GeoPointType) DataType(io.crate.types.DataType) TimestampType(io.crate.types.TimestampType) BitStringType(io.crate.types.BitStringType) BooleanType(io.crate.types.BooleanType) ShortType(io.crate.types.ShortType) SortedSetSortField(org.apache.lucene.search.SortedSetSortField) SortField(org.apache.lucene.search.SortField) SortedSetSortField(org.apache.lucene.search.SortedSetSortField) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) MultiValueMode(org.elasticsearch.search.MultiValueMode)

Example 17 with SortedNumericSortField

use of org.apache.lucene.search.SortedNumericSortField in project crate by crate.

the class OrderedLuceneBatchIteratorFactoryTest method createOrderedCollector.

private LuceneOrderedDocCollector createOrderedCollector(IndexSearcher searcher, int shardId) {
    CollectorContext collectorContext = new CollectorContext();
    List<LuceneCollectorExpression<?>> expressions = Collections.singletonList(new OrderByCollectorExpression(reference, orderBy, o -> o));
    return new LuceneOrderedDocCollector(new ShardId("dummy", UUIDs.randomBase64UUID(), shardId), searcher, new MatchAllDocsQuery(), null, false, // batchSize < 10 to have at least one searchMore call.
    5, RamAccounting.NO_ACCOUNTING, collectorContext, f -> null, new Sort(new SortedNumericSortField(columnName, SortField.Type.LONG, reverseFlags[0])), expressions, expressions);
}
Also used : ShardId(org.elasticsearch.index.shard.ShardId) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Arrays(java.util.Arrays) RowAccounting(io.crate.breaker.RowAccounting) LongType(io.crate.types.LongType) CollectorContext(io.crate.expression.reference.doc.lucene.CollectorContext) TestingHelpers.createReference(io.crate.testing.TestingHelpers.createReference) RowAccountingWithEstimators(io.crate.breaker.RowAccountingWithEstimators) BatchIterator(io.crate.data.BatchIterator) BatchIteratorTester(io.crate.testing.BatchIteratorTester) AtomicReference(java.util.concurrent.atomic.AtomicReference) Mockito.doThrow(org.mockito.Mockito.doThrow) Document(org.apache.lucene.document.Document) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) ESTestCase(org.elasticsearch.test.ESTestCase) SortField(org.apache.lucene.search.SortField) OrderingByPosition(io.crate.execution.engine.sort.OrderingByPosition) Before(org.junit.Before) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) LongStream(java.util.stream.LongStream) Sort(org.apache.lucene.search.Sort) DirectoryReader(org.apache.lucene.index.DirectoryReader) Reference(io.crate.metadata.Reference) UUIDs(org.elasticsearch.common.UUIDs) Test(org.junit.Test) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) RamAccounting(io.crate.breaker.RamAccounting) LuceneCollectorExpression(io.crate.expression.reference.doc.lucene.LuceneCollectorExpression) Collectors(java.util.stream.Collectors) TimeUnit(java.util.concurrent.TimeUnit) CountDownLatch(java.util.concurrent.CountDownLatch) IndexWriter(org.apache.lucene.index.IndexWriter) List(java.util.List) OrderBy(io.crate.analyze.OrderBy) StandardAnalyzer(org.apache.lucene.analysis.standard.StandardAnalyzer) Row(io.crate.data.Row) CircuitBreakingException(org.elasticsearch.common.breaker.CircuitBreakingException) OrderByCollectorExpression(io.crate.expression.reference.doc.lucene.OrderByCollectorExpression) DataTypes(io.crate.types.DataTypes) Matchers.is(org.hamcrest.Matchers.is) TestingRowConsumer(io.crate.testing.TestingRowConsumer) Comparator(java.util.Comparator) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) Collections(java.util.Collections) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) IndexSearcher(org.apache.lucene.search.IndexSearcher) Mockito.mock(org.mockito.Mockito.mock) ShardId(org.elasticsearch.index.shard.ShardId) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) OrderByCollectorExpression(io.crate.expression.reference.doc.lucene.OrderByCollectorExpression) Sort(org.apache.lucene.search.Sort) CollectorContext(io.crate.expression.reference.doc.lucene.CollectorContext) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) LuceneCollectorExpression(io.crate.expression.reference.doc.lucene.LuceneCollectorExpression)

Example 18 with SortedNumericSortField

use of org.apache.lucene.search.SortedNumericSortField in project elasticsearch by elastic.

the class CollapsingTopDocsCollectorTests method testCollapseLong.

public void testCollapseLong() throws Exception {
    CollapsingDocValuesProducer producer = new CollapsingDocValuesProducer<Long>() {

        @Override
        public Long randomGroup(int maxGroup) {
            return randomNonNegativeLong() % maxGroup;
        }

        @Override
        public void add(Document doc, Long value, boolean multivalued) {
            if (multivalued) {
                doc.add(new SortedNumericDocValuesField("field", value));
            } else {
                doc.add(new NumericDocValuesField("field", value));
            }
        }

        @Override
        public SortField sortField(boolean multivalued) {
            if (multivalued) {
                return new SortedNumericSortField("field", SortField.Type.LONG);
            } else {
                return new SortField("field", SortField.Type.LONG);
            }
        }
    };
    assertSearchCollapse(producer, true);
}
Also used : SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) SortField(org.apache.lucene.search.SortField) SortedSetSortField(org.apache.lucene.search.SortedSetSortField) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) Document(org.apache.lucene.document.Document)

Example 19 with SortedNumericSortField

use of org.apache.lucene.search.SortedNumericSortField in project elasticsearch by elastic.

the class CollapsingTopDocsCollectorTests method testCollapseFloat.

public void testCollapseFloat() throws Exception {
    CollapsingDocValuesProducer producer = new CollapsingDocValuesProducer<Float>() {

        @Override
        public Float randomGroup(int maxGroup) {
            return new Float(randomIntBetween(0, maxGroup - 1));
        }

        @Override
        public void add(Document doc, Float value, boolean multivalued) {
            if (multivalued) {
                doc.add(new SortedNumericDocValuesField("field", NumericUtils.floatToSortableInt(value)));
            } else {
                doc.add(new NumericDocValuesField("field", Float.floatToIntBits(value)));
            }
        }

        @Override
        public SortField sortField(boolean multivalued) {
            if (multivalued) {
                return new SortedNumericSortField("field", SortField.Type.FLOAT);
            } else {
                return new SortField("field", SortField.Type.FLOAT);
            }
        }
    };
    assertSearchCollapse(producer, true);
}
Also used : SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) SortField(org.apache.lucene.search.SortField) SortedSetSortField(org.apache.lucene.search.SortedSetSortField) SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) Document(org.apache.lucene.document.Document)

Example 20 with SortedNumericSortField

use of org.apache.lucene.search.SortedNumericSortField in project neo4j by neo4j.

the class TestLuceneIndex method queryAndSortNodesByNumericProperty.

private void queryAndSortNodesByNumericProperty(Index<Node> index, String numericProperty, Function<Integer, ? extends Number> expectedValueProvider) {
    try (Transaction transaction = graphDb.beginTx()) {
        QueryContext queryContext = new QueryContext(numericProperty + ":**");
        queryContext.sort(new Sort(new SortedNumericSortField(numericProperty, SortField.Type.INT, false)));
        IndexHits<Node> nodes = index.query(queryContext);
        int nodeIndex = 0;
        for (Node node : nodes) {
            assertEquals("Nodes should be sorted by numeric property", expectedValueProvider.apply(nodeIndex++), node.getProperty(numericProperty));
        }
        transaction.success();
    }
}
Also used : SortedNumericSortField(org.apache.lucene.search.SortedNumericSortField) Transaction(org.neo4j.graphdb.Transaction) Node(org.neo4j.graphdb.Node) Sort(org.apache.lucene.search.Sort) QueryContext(org.neo4j.index.lucene.QueryContext)

Aggregations

SortedNumericSortField (org.apache.lucene.search.SortedNumericSortField)33 SortField (org.apache.lucene.search.SortField)25 Sort (org.apache.lucene.search.Sort)24 SortedSetSortField (org.apache.lucene.search.SortedSetSortField)22 Document (org.apache.lucene.document.Document)18 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)18 SortedNumericDocValuesField (org.apache.lucene.document.SortedNumericDocValuesField)18 MockAnalyzer (org.apache.lucene.analysis.MockAnalyzer)13 Directory (org.apache.lucene.store.Directory)13 Version (org.apache.lucene.util.Version)5 CorruptIndexException (org.apache.lucene.index.CorruptIndexException)3 SegmentInfo (org.apache.lucene.index.SegmentInfo)3 SortedNumericSelector (org.apache.lucene.search.SortedNumericSelector)3 SortedSetSelector (org.apache.lucene.search.SortedSetSelector)3 ChecksumIndexInput (org.apache.lucene.store.ChecksumIndexInput)3 OrderBy (io.crate.analyze.OrderBy)2 LongType (io.crate.types.LongType)2 IndexSearcher (org.apache.lucene.search.IndexSearcher)2 IndexOutput (org.apache.lucene.store.IndexOutput)2 Path (com.querydsl.core.types.Path)1