Search in sources :

Example 1 with TermVectorsService

use of org.opensearch.index.termvectors.TermVectorsService in project OpenSearch by opensearch-project.

the class KeywordFieldMapperTests method testDefaults.

public void testDefaults() throws Exception {
    XContentBuilder mapping = fieldMapping(this::minimalMapping);
    DocumentMapper mapper = createDocumentMapper(mapping);
    assertEquals(Strings.toString(mapping), mapper.mappingSource().toString());
    ParsedDocument doc = mapper.parse(source(b -> b.field("field", "1234")));
    IndexableField[] fields = doc.rootDoc().getFields("field");
    assertEquals(2, fields.length);
    assertEquals(new BytesRef("1234"), fields[0].binaryValue());
    IndexableFieldType fieldType = fields[0].fieldType();
    assertThat(fieldType.omitNorms(), equalTo(true));
    assertFalse(fieldType.tokenized());
    assertFalse(fieldType.stored());
    assertThat(fieldType.indexOptions(), equalTo(IndexOptions.DOCS));
    assertThat(fieldType.storeTermVectors(), equalTo(false));
    assertThat(fieldType.storeTermVectorOffsets(), equalTo(false));
    assertThat(fieldType.storeTermVectorPositions(), equalTo(false));
    assertThat(fieldType.storeTermVectorPayloads(), equalTo(false));
    assertEquals(DocValuesType.NONE, fieldType.docValuesType());
    assertEquals(new BytesRef("1234"), fields[1].binaryValue());
    fieldType = fields[1].fieldType();
    assertThat(fieldType.indexOptions(), equalTo(IndexOptions.NONE));
    assertEquals(DocValuesType.SORTED_SET, fieldType.docValuesType());
    // used by TermVectorsService
    assertArrayEquals(new String[] { "1234" }, TermVectorsService.getValues(doc.rootDoc().getFields("field")));
}
Also used : TokenizerFactory(org.opensearch.index.analysis.TokenizerFactory) Arrays(java.util.Arrays) IndexableField(org.apache.lucene.index.IndexableField) MockTokenizer(org.apache.lucene.analysis.MockTokenizer) TokenFilterFactory(org.opensearch.index.analysis.TokenFilterFactory) MockLowerCaseFilter(org.apache.lucene.analysis.MockLowerCaseFilter) AnalyzerScope(org.opensearch.index.analysis.AnalyzerScope) LowerCaseFilter(org.apache.lucene.analysis.core.LowerCaseFilter) Strings(org.opensearch.common.Strings) Collections.singletonList(java.util.Collections.singletonList) IndexableFieldType(org.apache.lucene.index.IndexableFieldType) CharFilterFactory(org.opensearch.index.analysis.CharFilterFactory) Map(java.util.Map) Collections.singletonMap(java.util.Collections.singletonMap) CustomAnalyzer(org.opensearch.index.analysis.CustomAnalyzer) WhitespaceTokenizer(org.apache.lucene.analysis.core.WhitespaceTokenizer) NamedAnalyzer(org.opensearch.index.analysis.NamedAnalyzer) TokenStream(org.apache.lucene.analysis.TokenStream) PreConfiguredTokenFilter(org.opensearch.index.analysis.PreConfiguredTokenFilter) BytesRef(org.apache.lucene.util.BytesRef) Analyzer(org.apache.lucene.analysis.Analyzer) Collection(java.util.Collection) IOException(java.io.IOException) Plugin(org.opensearch.plugins.Plugin) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) Matchers.instanceOf(org.hamcrest.Matchers.instanceOf) AnalysisPlugin(org.opensearch.plugins.AnalysisPlugin) List(java.util.List) StandardAnalyzer(org.apache.lucene.analysis.standard.StandardAnalyzer) AnalysisModule(org.opensearch.indices.analysis.AnalysisModule) DocValuesType(org.apache.lucene.index.DocValuesType) Matchers.equalTo(org.hamcrest.Matchers.equalTo) IndexSettings(org.opensearch.index.IndexSettings) BaseTokenStreamTestCase.assertTokenStreamContents(org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents) IndexOptions(org.apache.lucene.index.IndexOptions) Matchers.containsString(org.hamcrest.Matchers.containsString) IndexAnalyzers(org.opensearch.index.analysis.IndexAnalyzers) TermVectorsService(org.opensearch.index.termvectors.TermVectorsService) LowercaseNormalizer(org.opensearch.index.analysis.LowercaseNormalizer) IndexableField(org.apache.lucene.index.IndexableField) IndexableFieldType(org.apache.lucene.index.IndexableFieldType) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) BytesRef(org.apache.lucene.util.BytesRef)

Aggregations

IOException (java.io.IOException)1 Arrays (java.util.Arrays)1 Collection (java.util.Collection)1 Collections.singletonList (java.util.Collections.singletonList)1 Collections.singletonMap (java.util.Collections.singletonMap)1 List (java.util.List)1 Map (java.util.Map)1 Analyzer (org.apache.lucene.analysis.Analyzer)1 BaseTokenStreamTestCase.assertTokenStreamContents (org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents)1 MockLowerCaseFilter (org.apache.lucene.analysis.MockLowerCaseFilter)1 MockTokenizer (org.apache.lucene.analysis.MockTokenizer)1 TokenStream (org.apache.lucene.analysis.TokenStream)1 LowerCaseFilter (org.apache.lucene.analysis.core.LowerCaseFilter)1 WhitespaceTokenizer (org.apache.lucene.analysis.core.WhitespaceTokenizer)1 StandardAnalyzer (org.apache.lucene.analysis.standard.StandardAnalyzer)1 DocValuesType (org.apache.lucene.index.DocValuesType)1 IndexOptions (org.apache.lucene.index.IndexOptions)1 IndexableField (org.apache.lucene.index.IndexableField)1 IndexableFieldType (org.apache.lucene.index.IndexableFieldType)1 BytesRef (org.apache.lucene.util.BytesRef)1