Search in sources :

Example 1 with SortedNumericDoubleValues

use of org.opensearch.index.fielddata.SortedNumericDoubleValues in project OpenSearch by opensearch-project.

the class ExpressionNumberSortScriptTests method setUp.

@Override
public void setUp() throws Exception {
    super.setUp();
    NumberFieldType fieldType = new NumberFieldType("field", NumberType.DOUBLE);
    MapperService mapperService = mock(MapperService.class);
    when(mapperService.fieldType("field")).thenReturn(fieldType);
    when(mapperService.fieldType("alias")).thenReturn(fieldType);
    SortedNumericDoubleValues doubleValues = mock(SortedNumericDoubleValues.class);
    when(doubleValues.advanceExact(anyInt())).thenReturn(true);
    when(doubleValues.nextValue()).thenReturn(2.718);
    LeafNumericFieldData atomicFieldData = mock(LeafNumericFieldData.class);
    when(atomicFieldData.getDoubleValues()).thenReturn(doubleValues);
    IndexNumericFieldData fieldData = mock(IndexNumericFieldData.class);
    when(fieldData.getFieldName()).thenReturn("field");
    when(fieldData.load(any())).thenReturn(atomicFieldData);
    service = new ExpressionScriptEngine();
    lookup = new SearchLookup(mapperService, (ignored, lookup) -> fieldData);
}
Also used : NumberSortScript(org.opensearch.script.NumberSortScript) IndexNumericFieldData(org.opensearch.index.fielddata.IndexNumericFieldData) LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) OpenSearchTestCase(org.opensearch.test.OpenSearchTestCase) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) SearchLookup(org.opensearch.search.lookup.SearchLookup) NumberFieldType(org.opensearch.index.mapper.NumberFieldMapper.NumberFieldType) ScriptException(org.opensearch.script.ScriptException) MapperService(org.opensearch.index.mapper.MapperService) NumberType(org.opensearch.index.mapper.NumberFieldMapper.NumberType) Mockito.anyInt(org.mockito.Mockito.anyInt) ParseException(java.text.ParseException) Mockito.any(org.mockito.Mockito.any) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) NumberFieldType(org.opensearch.index.mapper.NumberFieldMapper.NumberFieldType) IndexNumericFieldData(org.opensearch.index.fielddata.IndexNumericFieldData) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) MapperService(org.opensearch.index.mapper.MapperService) SearchLookup(org.opensearch.search.lookup.SearchLookup)

Example 2 with SortedNumericDoubleValues

use of org.opensearch.index.fielddata.SortedNumericDoubleValues in project OpenSearch by opensearch-project.

the class ExpressionFieldScriptTests method setUp.

@Override
public void setUp() throws Exception {
    super.setUp();
    NumberFieldMapper.NumberFieldType fieldType = new NumberFieldMapper.NumberFieldType("field", NumberFieldMapper.NumberType.DOUBLE);
    MapperService mapperService = mock(MapperService.class);
    when(mapperService.fieldType("field")).thenReturn(fieldType);
    when(mapperService.fieldType("alias")).thenReturn(fieldType);
    SortedNumericDoubleValues doubleValues = mock(SortedNumericDoubleValues.class);
    when(doubleValues.advanceExact(anyInt())).thenReturn(true);
    when(doubleValues.nextValue()).thenReturn(2.718);
    LeafNumericFieldData atomicFieldData = mock(LeafNumericFieldData.class);
    when(atomicFieldData.getDoubleValues()).thenReturn(doubleValues);
    IndexNumericFieldData fieldData = mock(IndexNumericFieldData.class);
    when(fieldData.getFieldName()).thenReturn("field");
    when(fieldData.load(any())).thenReturn(atomicFieldData);
    service = new ExpressionScriptEngine();
    lookup = new SearchLookup(mapperService, (ignored, lookup) -> fieldData);
}
Also used : IndexNumericFieldData(org.opensearch.index.fielddata.IndexNumericFieldData) LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) FieldScript(org.opensearch.script.FieldScript) OpenSearchTestCase(org.opensearch.test.OpenSearchTestCase) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) NumberFieldMapper(org.opensearch.index.mapper.NumberFieldMapper) SearchLookup(org.opensearch.search.lookup.SearchLookup) ScriptException(org.opensearch.script.ScriptException) MapperService(org.opensearch.index.mapper.MapperService) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Mockito.anyInt(org.mockito.Mockito.anyInt) ParseException(java.text.ParseException) Mockito.any(org.mockito.Mockito.any) Collections(java.util.Collections) Mockito.mock(org.mockito.Mockito.mock) NumberFieldMapper(org.opensearch.index.mapper.NumberFieldMapper) LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) IndexNumericFieldData(org.opensearch.index.fielddata.IndexNumericFieldData) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) MapperService(org.opensearch.index.mapper.MapperService) SearchLookup(org.opensearch.search.lookup.SearchLookup)

Example 3 with SortedNumericDoubleValues

use of org.opensearch.index.fielddata.SortedNumericDoubleValues in project OpenSearch by opensearch-project.

the class GeoUtils method distanceValues.

/**
 * Return a {@link SortedNumericDoubleValues} instance that returns the distances to a list of geo-points
 * for each document.
 */
public static SortedNumericDoubleValues distanceValues(final GeoDistance distance, final DistanceUnit unit, final MultiGeoPointValues geoPointValues, final GeoPoint... fromPoints) {
    final GeoPointValues singleValues = FieldData.unwrapSingleton(geoPointValues);
    if (singleValues != null && fromPoints.length == 1) {
        return FieldData.singleton(new NumericDoubleValues() {

            @Override
            public boolean advanceExact(int doc) throws IOException {
                return singleValues.advanceExact(doc);
            }

            @Override
            public double doubleValue() throws IOException {
                final GeoPoint from = fromPoints[0];
                final GeoPoint to = singleValues.geoPointValue();
                return distance.calculate(from.lat(), from.lon(), to.lat(), to.lon(), unit);
            }
        });
    } else {
        return new SortingNumericDoubleValues() {

            @Override
            public boolean advanceExact(int target) throws IOException {
                if (geoPointValues.advanceExact(target)) {
                    resize(geoPointValues.docValueCount() * fromPoints.length);
                    int v = 0;
                    for (int i = 0; i < geoPointValues.docValueCount(); ++i) {
                        final GeoPoint point = geoPointValues.nextValue();
                        for (GeoPoint from : fromPoints) {
                            values[v] = distance.calculate(from.lat(), from.lon(), point.lat(), point.lon(), unit);
                            v++;
                        }
                    }
                    sort();
                    return true;
                } else {
                    return false;
                }
            }
        };
    }
}
Also used : SortingNumericDoubleValues(org.opensearch.index.fielddata.SortingNumericDoubleValues) MultiGeoPointValues(org.opensearch.index.fielddata.MultiGeoPointValues) GeoPointValues(org.opensearch.index.fielddata.GeoPointValues) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) SortingNumericDoubleValues(org.opensearch.index.fielddata.SortingNumericDoubleValues) NumericDoubleValues(org.opensearch.index.fielddata.NumericDoubleValues) IOException(java.io.IOException)

Example 4 with SortedNumericDoubleValues

use of org.opensearch.index.fielddata.SortedNumericDoubleValues in project OpenSearch by opensearch-project.

the class ScaledFloatFieldTypeTests method testFieldData.

public void testFieldData() throws IOException {
    double scalingFactor = 0.1 + randomDouble() * 100;
    Directory dir = newDirectory();
    IndexWriter w = new IndexWriter(dir, new IndexWriterConfig(null));
    Document doc = new Document();
    doc.add(new SortedNumericDocValuesField("scaled_float1", 10));
    doc.add(new SortedNumericDocValuesField("scaled_float2", 5));
    doc.add(new SortedNumericDocValuesField("scaled_float2", 12));
    w.addDocument(doc);
    try (DirectoryReader reader = DirectoryReader.open(w)) {
        // single-valued
        ScaledFloatFieldMapper.ScaledFloatFieldType f1 = new ScaledFloatFieldMapper.ScaledFloatFieldType("scaled_float1", scalingFactor);
        IndexNumericFieldData fielddata = (IndexNumericFieldData) f1.fielddataBuilder("index", () -> {
            throw new UnsupportedOperationException();
        }).build(null, null);
        assertEquals(fielddata.getNumericType(), IndexNumericFieldData.NumericType.DOUBLE);
        LeafNumericFieldData leafFieldData = fielddata.load(reader.leaves().get(0));
        SortedNumericDoubleValues values = leafFieldData.getDoubleValues();
        assertTrue(values.advanceExact(0));
        assertEquals(1, values.docValueCount());
        assertEquals(10 / f1.getScalingFactor(), values.nextValue(), 10e-5);
        // multi-valued
        ScaledFloatFieldMapper.ScaledFloatFieldType f2 = new ScaledFloatFieldMapper.ScaledFloatFieldType("scaled_float2", scalingFactor);
        fielddata = (IndexNumericFieldData) f2.fielddataBuilder("index", () -> {
            throw new UnsupportedOperationException();
        }).build(null, null);
        leafFieldData = fielddata.load(reader.leaves().get(0));
        values = leafFieldData.getDoubleValues();
        assertTrue(values.advanceExact(0));
        assertEquals(2, values.docValueCount());
        assertEquals(5 / f2.getScalingFactor(), values.nextValue(), 10e-5);
        assertEquals(12 / f2.getScalingFactor(), values.nextValue(), 10e-5);
    }
    IOUtils.close(w, dir);
}
Also used : LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) DirectoryReader(org.apache.lucene.index.DirectoryReader) IndexNumericFieldData(org.opensearch.index.fielddata.IndexNumericFieldData) Document(org.apache.lucene.document.Document) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) IndexWriter(org.apache.lucene.index.IndexWriter) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) Directory(org.apache.lucene.store.Directory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Example 5 with SortedNumericDoubleValues

use of org.opensearch.index.fielddata.SortedNumericDoubleValues in project OpenSearch by opensearch-project.

the class FloatValuesComparatorSource method getNumericDocValues.

private NumericDoubleValues getNumericDocValues(LeafReaderContext context, float missingValue) throws IOException {
    final SortedNumericDoubleValues values = indexFieldData.load(context).getDoubleValues();
    if (nested == null) {
        return FieldData.replaceMissing(sortMode.select(values), missingValue);
    } else {
        final BitSet rootDocs = nested.rootDocs(context);
        final DocIdSetIterator innerDocs = nested.innerDocs(context);
        final int maxChildren = nested.getNestedSort() != null ? nested.getNestedSort().getMaxChildren() : Integer.MAX_VALUE;
        return sortMode.select(values, missingValue, rootDocs, innerDocs, context.reader().maxDoc(), maxChildren);
    }
}
Also used : BitSet(org.apache.lucene.util.BitSet) SortedNumericDoubleValues(org.opensearch.index.fielddata.SortedNumericDoubleValues) DocIdSetIterator(org.apache.lucene.search.DocIdSetIterator)

Aggregations

SortedNumericDoubleValues (org.opensearch.index.fielddata.SortedNumericDoubleValues)33 LeafBucketCollectorBase (org.opensearch.search.aggregations.LeafBucketCollectorBase)13 BigArrays (org.opensearch.common.util.BigArrays)12 NumericDoubleValues (org.opensearch.index.fielddata.NumericDoubleValues)8 IOException (java.io.IOException)6 LeafNumericFieldData (org.opensearch.index.fielddata.LeafNumericFieldData)6 IndexNumericFieldData (org.opensearch.index.fielddata.IndexNumericFieldData)4 ParseException (java.text.ParseException)3 Collections (java.util.Collections)3 Document (org.apache.lucene.document.Document)3 DirectoryReader (org.apache.lucene.index.DirectoryReader)3 IndexWriter (org.apache.lucene.index.IndexWriter)3 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)3 DocIdSetIterator (org.apache.lucene.search.DocIdSetIterator)3 Directory (org.apache.lucene.store.Directory)3 BitSet (org.apache.lucene.util.BitSet)3 Mockito.any (org.mockito.Mockito.any)3 Mockito.anyInt (org.mockito.Mockito.anyInt)3 Mockito.mock (org.mockito.Mockito.mock)3 Mockito.when (org.mockito.Mockito.when)3