Search in sources :

Example 1 with SortedNumericIndexFieldData

use of org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData in project OpenSearch by opensearch-project.

the class IndexFieldDataServiceTests method testGetForFieldDefaults.

public void testGetForFieldDefaults() {
    final IndexService indexService = createIndex("test");
    final IndicesService indicesService = getInstanceFromNode(IndicesService.class);
    final IndexFieldDataService ifdService = new IndexFieldDataService(indexService.getIndexSettings(), indicesService.getIndicesFieldDataCache(), indicesService.getCircuitBreakerService(), indexService.mapperService());
    final BuilderContext ctx = new BuilderContext(indexService.getIndexSettings().getSettings(), new ContentPath(1));
    final MappedFieldType stringMapper = new KeywordFieldMapper.Builder("string").build(ctx).fieldType();
    ifdService.clear();
    IndexFieldData<?> fd = ifdService.getForField(stringMapper, "test", () -> {
        throw new UnsupportedOperationException();
    });
    assertTrue(fd instanceof SortedSetOrdinalsIndexFieldData);
    for (MappedFieldType mapper : Arrays.asList(new NumberFieldMapper.Builder("int", NumberFieldMapper.NumberType.BYTE, false, true).build(ctx).fieldType(), new NumberFieldMapper.Builder("int", NumberFieldMapper.NumberType.SHORT, false, true).build(ctx).fieldType(), new NumberFieldMapper.Builder("int", NumberFieldMapper.NumberType.INTEGER, false, true).build(ctx).fieldType(), new NumberFieldMapper.Builder("long", NumberFieldMapper.NumberType.LONG, false, true).build(ctx).fieldType())) {
        ifdService.clear();
        fd = ifdService.getForField(mapper, "test", () -> {
            throw new UnsupportedOperationException();
        });
        assertTrue(fd instanceof SortedNumericIndexFieldData);
    }
    final MappedFieldType floatMapper = new NumberFieldMapper.Builder("float", NumberFieldMapper.NumberType.FLOAT, false, true).build(ctx).fieldType();
    ifdService.clear();
    fd = ifdService.getForField(floatMapper, "test", () -> {
        throw new UnsupportedOperationException();
    });
    assertTrue(fd instanceof SortedNumericIndexFieldData);
    final MappedFieldType doubleMapper = new NumberFieldMapper.Builder("double", NumberFieldMapper.NumberType.DOUBLE, false, true).build(ctx).fieldType();
    ifdService.clear();
    fd = ifdService.getForField(doubleMapper, "test", () -> {
        throw new UnsupportedOperationException();
    });
    assertTrue(fd instanceof SortedNumericIndexFieldData);
}
Also used : NumberFieldMapper(org.opensearch.index.mapper.NumberFieldMapper) IndexService(org.opensearch.index.IndexService) IndicesService(org.opensearch.indices.IndicesService) SortedSetOrdinalsIndexFieldData(org.opensearch.index.fielddata.plain.SortedSetOrdinalsIndexFieldData) ContentPath(org.opensearch.index.mapper.ContentPath) SortedNumericIndexFieldData(org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) BuilderContext(org.opensearch.index.mapper.Mapper.BuilderContext)

Example 2 with SortedNumericIndexFieldData

use of org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData in project OpenSearch by opensearch-project.

the class DateFieldTypeTests method testDateNanoDocValues.

public void testDateNanoDocValues() throws IOException {
    // Create an index with some docValues
    Directory dir = newDirectory();
    IndexWriter w = new IndexWriter(dir, new IndexWriterConfig(null));
    Document doc = new Document();
    NumericDocValuesField docValuesField = new NumericDocValuesField("my_date", 1444608000000L);
    doc.add(docValuesField);
    w.addDocument(doc);
    docValuesField.setLongValue(1459641600000L);
    w.addDocument(doc);
    // Create the doc values reader
    SortedNumericIndexFieldData fieldData = new SortedNumericIndexFieldData("my_date", IndexNumericFieldData.NumericType.DATE_NANOSECONDS);
    // Read index and check the doc values
    DirectoryReader reader = DirectoryReader.open(w);
    assertTrue(reader.leaves().size() > 0);
    LeafNumericFieldData a = fieldData.load(reader.leaves().get(0).reader().getContext());
    SortedNumericDocValues docValues = a.getLongValues();
    assertEquals(0, docValues.nextDoc());
    assertEquals(1, docValues.nextDoc());
    assertEquals(DocIdSetIterator.NO_MORE_DOCS, docValues.nextDoc());
    reader.close();
    w.close();
    dir.close();
}
Also used : SortedNumericIndexFieldData(org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData) LeafNumericFieldData(org.opensearch.index.fielddata.LeafNumericFieldData) SortedNumericDocValues(org.apache.lucene.index.SortedNumericDocValues) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) IndexWriter(org.apache.lucene.index.IndexWriter) DirectoryReader(org.apache.lucene.index.DirectoryReader) Document(org.opensearch.index.mapper.ParseContext.Document) Directory(org.apache.lucene.store.Directory) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig)

Example 3 with SortedNumericIndexFieldData

use of org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData in project OpenSearch by opensearch-project.

the class DiversifiedSamplerTests method testCase.

private void testCase(IndexSearcher indexSearcher, MappedFieldType genreFieldType, String executionHint, Consumer<InternalSampler> verify, int shardSize, int maxDocsPerValue) throws IOException {
    MappedFieldType idFieldType = new KeywordFieldMapper.KeywordFieldType("id");
    SortedNumericIndexFieldData fieldData = new SortedNumericIndexFieldData("price", IndexNumericFieldData.NumericType.DOUBLE);
    FunctionScoreQuery query = new FunctionScoreQuery(new MatchAllDocsQuery(), new FieldValueFactorFunction("price", 1, FieldValueFactorFunction.Modifier.RECIPROCAL, null, fieldData));
    DiversifiedAggregationBuilder builder = new DiversifiedAggregationBuilder("_name").field(genreFieldType.name()).executionHint(executionHint).maxDocsPerValue(maxDocsPerValue).shardSize(shardSize).subAggregation(new TermsAggregationBuilder("terms").field("id"));
    InternalSampler result = searchAndReduce(indexSearcher, query, builder, genreFieldType, idFieldType);
    verify.accept(result);
}
Also used : SortedNumericIndexFieldData(org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData) TermsAggregationBuilder(org.opensearch.search.aggregations.bucket.terms.TermsAggregationBuilder) FunctionScoreQuery(org.opensearch.common.lucene.search.function.FunctionScoreQuery) FieldValueFactorFunction(org.opensearch.common.lucene.search.function.FieldValueFactorFunction) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery)

Aggregations

SortedNumericIndexFieldData (org.opensearch.index.fielddata.plain.SortedNumericIndexFieldData)3 MappedFieldType (org.opensearch.index.mapper.MappedFieldType)2 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)1 SortedNumericDocValuesField (org.apache.lucene.document.SortedNumericDocValuesField)1 DirectoryReader (org.apache.lucene.index.DirectoryReader)1 IndexWriter (org.apache.lucene.index.IndexWriter)1 IndexWriterConfig (org.apache.lucene.index.IndexWriterConfig)1 SortedNumericDocValues (org.apache.lucene.index.SortedNumericDocValues)1 MatchAllDocsQuery (org.apache.lucene.search.MatchAllDocsQuery)1 Directory (org.apache.lucene.store.Directory)1 FieldValueFactorFunction (org.opensearch.common.lucene.search.function.FieldValueFactorFunction)1 FunctionScoreQuery (org.opensearch.common.lucene.search.function.FunctionScoreQuery)1 IndexService (org.opensearch.index.IndexService)1 LeafNumericFieldData (org.opensearch.index.fielddata.LeafNumericFieldData)1 SortedSetOrdinalsIndexFieldData (org.opensearch.index.fielddata.plain.SortedSetOrdinalsIndexFieldData)1 ContentPath (org.opensearch.index.mapper.ContentPath)1 BuilderContext (org.opensearch.index.mapper.Mapper.BuilderContext)1 NumberFieldMapper (org.opensearch.index.mapper.NumberFieldMapper)1 Document (org.opensearch.index.mapper.ParseContext.Document)1 IndicesService (org.opensearch.indices.IndicesService)1