Search in sources :

Example 1 with MappingLookup

use of org.opensearch.index.mapper.MappingLookup in project OpenSearch by opensearch-project.

the class TransportGetFieldMappingsIndexAction method findFieldMappingsByType.

private static Map<String, FieldMappingMetadata> findFieldMappingsByType(Predicate<String> fieldPredicate, DocumentMapper documentMapper, GetFieldMappingsIndexRequest request) {
    Map<String, FieldMappingMetadata> fieldMappings = new HashMap<>();
    final MappingLookup allFieldMappers = documentMapper.mappers();
    for (String field : request.fields()) {
        if (Regex.isMatchAllPattern(field)) {
            for (Mapper fieldMapper : allFieldMappers) {
                addFieldMapper(fieldPredicate, fieldMapper.name(), fieldMapper, fieldMappings, request.includeDefaults());
            }
        } else if (Regex.isSimpleMatchPattern(field)) {
            for (Mapper fieldMapper : allFieldMappers) {
                if (Regex.simpleMatch(field, fieldMapper.name())) {
                    addFieldMapper(fieldPredicate, fieldMapper.name(), fieldMapper, fieldMappings, request.includeDefaults());
                }
            }
        } else {
            // not a pattern
            Mapper fieldMapper = allFieldMappers.getMapper(field);
            if (fieldMapper != null) {
                addFieldMapper(fieldPredicate, field, fieldMapper, fieldMappings, request.includeDefaults());
            } else if (request.probablySingleFieldRequest()) {
                fieldMappings.put(field, FieldMappingMetadata.NULL);
            }
        }
    }
    return Collections.unmodifiableMap(fieldMappings);
}
Also used : Mapper(org.opensearch.index.mapper.Mapper) DocumentMapper(org.opensearch.index.mapper.DocumentMapper) HashMap(java.util.HashMap) MappingLookup(org.opensearch.index.mapper.MappingLookup) FieldMappingMetadata(org.opensearch.action.admin.indices.mapping.get.GetFieldMappingsResponse.FieldMappingMetadata)

Example 2 with MappingLookup

use of org.opensearch.index.mapper.MappingLookup in project OpenSearch by opensearch-project.

the class ParentToChildrenAggregatorTests method mapperServiceMock.

@Override
protected MapperService mapperServiceMock() {
    ParentJoinFieldMapper joinFieldMapper = createJoinFieldMapper();
    MapperService mapperService = mock(MapperService.class);
    MetaJoinFieldMapper.MetaJoinFieldType metaJoinFieldType = mock(MetaJoinFieldMapper.MetaJoinFieldType.class);
    when(metaJoinFieldType.getJoinField()).thenReturn("join_field");
    when(mapperService.fieldType("_parent_join")).thenReturn(metaJoinFieldType);
    MappingLookup fieldMappers = new MappingLookup(Collections.singleton(joinFieldMapper), Collections.emptyList(), Collections.emptyList(), 0, null);
    DocumentMapper mockMapper = mock(DocumentMapper.class);
    when(mockMapper.mappers()).thenReturn(fieldMappers);
    when(mapperService.documentMapper()).thenReturn(mockMapper);
    return mapperService;
}
Also used : ParentJoinFieldMapper(org.opensearch.join.mapper.ParentJoinFieldMapper) MappingLookup(org.opensearch.index.mapper.MappingLookup) MetaJoinFieldMapper(org.opensearch.join.mapper.MetaJoinFieldMapper) DocumentMapper(org.opensearch.index.mapper.DocumentMapper) MapperService(org.opensearch.index.mapper.MapperService)

Example 3 with MappingLookup

use of org.opensearch.index.mapper.MappingLookup in project OpenSearch by opensearch-project.

the class ChildrenToParentAggregatorTests method mapperServiceMock.

@Override
protected MapperService mapperServiceMock() {
    ParentJoinFieldMapper joinFieldMapper = createJoinFieldMapper();
    MapperService mapperService = mock(MapperService.class);
    MetaJoinFieldMapper.MetaJoinFieldType metaJoinFieldType = mock(MetaJoinFieldMapper.MetaJoinFieldType.class);
    when(metaJoinFieldType.getJoinField()).thenReturn("join_field");
    when(mapperService.fieldType("_parent_join")).thenReturn(metaJoinFieldType);
    MappingLookup fieldMappers = new MappingLookup(Collections.singleton(joinFieldMapper), Collections.emptyList(), Collections.emptyList(), 0, null);
    DocumentMapper mockMapper = mock(DocumentMapper.class);
    when(mockMapper.mappers()).thenReturn(fieldMappers);
    when(mapperService.documentMapper()).thenReturn(mockMapper);
    return mapperService;
}
Also used : ParentJoinFieldMapper(org.opensearch.join.mapper.ParentJoinFieldMapper) MappingLookup(org.opensearch.index.mapper.MappingLookup) MetaJoinFieldMapper(org.opensearch.join.mapper.MetaJoinFieldMapper) DocumentMapper(org.opensearch.index.mapper.DocumentMapper) MapperService(org.opensearch.index.mapper.MapperService)

Example 4 with MappingLookup

use of org.opensearch.index.mapper.MappingLookup in project OpenSearch by opensearch-project.

the class ParentJoinFieldMapper method getMapper.

/**
 * Returns the {@link ParentJoinFieldMapper} associated with the <code>service</code> or null
 * if there is no parent-join field in this mapping.
 */
public static ParentJoinFieldMapper getMapper(MapperService service) {
    MetaJoinFieldMapper.MetaJoinFieldType fieldType = (MetaJoinFieldMapper.MetaJoinFieldType) service.fieldType(MetaJoinFieldMapper.NAME);
    if (fieldType == null) {
        return null;
    }
    DocumentMapper mapper = service.documentMapper();
    String joinField = fieldType.getJoinField();
    MappingLookup fieldMappers = mapper.mappers();
    return (ParentJoinFieldMapper) fieldMappers.getMapper(joinField);
}
Also used : MappingLookup(org.opensearch.index.mapper.MappingLookup) DocumentMapper(org.opensearch.index.mapper.DocumentMapper)

Example 5 with MappingLookup

use of org.opensearch.index.mapper.MappingLookup in project OpenSearch by opensearch-project.

the class MetadataRolloverServiceTests method testRolloverClusterStateForDataStream.

public void testRolloverClusterStateForDataStream() throws Exception {
    final DataStream dataStream = DataStreamTests.randomInstance();
    ComposableIndexTemplate template = new ComposableIndexTemplate(Collections.singletonList(dataStream.getName() + "*"), null, null, null, null, null, new ComposableIndexTemplate.DataStreamTemplate());
    Metadata.Builder builder = Metadata.builder();
    builder.put("template", template);
    for (Index index : dataStream.getIndices()) {
        builder.put(DataStreamTestHelper.getIndexMetadataBuilderForIndex(index));
    }
    builder.put(dataStream);
    final ClusterState clusterState = ClusterState.builder(new ClusterName("test")).metadata(builder).build();
    ThreadPool testThreadPool = new TestThreadPool(getTestName());
    try {
        Mapper.BuilderContext builderContext = new Mapper.BuilderContext(Settings.EMPTY, new ContentPath(0));
        DateFieldMapper dateFieldMapper = new DateFieldMapper.Builder("@timestamp", DateFieldMapper.Resolution.MILLISECONDS, null, false, Version.CURRENT).build(builderContext);
        MetadataFieldMapper mockedTimestampField = mock(MetadataFieldMapper.class);
        when(mockedTimestampField.name()).thenReturn("_data_stream_timestamp");
        MappedFieldType mockedTimestampFieldType = mock(MappedFieldType.class);
        when(mockedTimestampFieldType.name()).thenReturn("_data_stream_timestamp");
        when(mockedTimestampField.fieldType()).thenReturn(mockedTimestampFieldType);
        when(mockedTimestampField.copyTo()).thenReturn(FieldMapper.CopyTo.empty());
        when(mockedTimestampField.multiFields()).thenReturn(FieldMapper.MultiFields.empty());
        MappingLookup mappingLookup = new MappingLookup(Arrays.asList(mockedTimestampField, dateFieldMapper), Collections.emptyList(), Collections.emptyList(), 0, new StandardAnalyzer());
        ClusterService clusterService = ClusterServiceUtils.createClusterService(testThreadPool);
        Environment env = mock(Environment.class);
        when(env.sharedDataFile()).thenReturn(null);
        AllocationService allocationService = mock(AllocationService.class);
        when(allocationService.reroute(any(ClusterState.class), any(String.class))).then(i -> i.getArguments()[0]);
        DocumentMapper documentMapper = mock(DocumentMapper.class);
        when(documentMapper.mappers()).thenReturn(mappingLookup);
        when(documentMapper.type()).thenReturn("_doc");
        CompressedXContent mapping = new CompressedXContent("{\"_doc\":" + generateMapping(dataStream.getTimeStampField().getName(), "date") + "}");
        when(documentMapper.mappingSource()).thenReturn(mapping);
        RoutingFieldMapper routingFieldMapper = mock(RoutingFieldMapper.class);
        when(routingFieldMapper.required()).thenReturn(false);
        when(documentMapper.routingFieldMapper()).thenReturn(routingFieldMapper);
        IndicesService indicesService = mockIndicesServices(documentMapper);
        IndexNameExpressionResolver mockIndexNameExpressionResolver = mock(IndexNameExpressionResolver.class);
        when(mockIndexNameExpressionResolver.resolveDateMathExpression(any())).then(returnsFirstArg());
        ShardLimitValidator shardLimitValidator = new ShardLimitValidator(Settings.EMPTY, clusterService);
        MetadataCreateIndexService createIndexService = new MetadataCreateIndexService(Settings.EMPTY, clusterService, indicesService, allocationService, null, shardLimitValidator, env, IndexScopedSettings.DEFAULT_SCOPED_SETTINGS, testThreadPool, null, new SystemIndices(emptyMap()), false);
        MetadataIndexAliasesService indexAliasesService = new MetadataIndexAliasesService(clusterService, indicesService, new AliasValidator(), null, xContentRegistry());
        MetadataRolloverService rolloverService = new MetadataRolloverService(testThreadPool, createIndexService, indexAliasesService, mockIndexNameExpressionResolver);
        MaxDocsCondition condition = new MaxDocsCondition(randomNonNegativeLong());
        List<Condition<?>> metConditions = Collections.singletonList(condition);
        CreateIndexRequest createIndexRequest = new CreateIndexRequest("_na_");
        long before = testThreadPool.absoluteTimeInMillis();
        MetadataRolloverService.RolloverResult rolloverResult = rolloverService.rolloverClusterState(clusterState, dataStream.getName(), null, createIndexRequest, metConditions, randomBoolean(), false);
        long after = testThreadPool.absoluteTimeInMillis();
        String sourceIndexName = DataStream.getDefaultBackingIndexName(dataStream.getName(), dataStream.getGeneration());
        String newIndexName = DataStream.getDefaultBackingIndexName(dataStream.getName(), dataStream.getGeneration() + 1);
        assertEquals(sourceIndexName, rolloverResult.sourceIndexName);
        assertEquals(newIndexName, rolloverResult.rolloverIndexName);
        Metadata rolloverMetadata = rolloverResult.clusterState.metadata();
        assertEquals(dataStream.getIndices().size() + 1, rolloverMetadata.indices().size());
        IndexMetadata rolloverIndexMetadata = rolloverMetadata.index(newIndexName);
        IndexAbstraction ds = rolloverMetadata.getIndicesLookup().get(dataStream.getName());
        assertThat(ds.getType(), equalTo(IndexAbstraction.Type.DATA_STREAM));
        assertThat(ds.getIndices(), hasSize(dataStream.getIndices().size() + 1));
        assertThat(ds.getIndices(), hasItem(rolloverMetadata.index(sourceIndexName)));
        assertThat(ds.getIndices(), hasItem(rolloverIndexMetadata));
        assertThat(ds.getWriteIndex(), equalTo(rolloverIndexMetadata));
        RolloverInfo info = rolloverMetadata.index(sourceIndexName).getRolloverInfos().get(dataStream.getName());
        assertThat(info.getTime(), lessThanOrEqualTo(after));
        assertThat(info.getTime(), greaterThanOrEqualTo(before));
        assertThat(info.getMetConditions(), hasSize(1));
        assertThat(info.getMetConditions().get(0).value(), equalTo(condition.value()));
    } finally {
        testThreadPool.shutdown();
    }
}
Also used : ComposableIndexTemplate(org.opensearch.cluster.metadata.ComposableIndexTemplate) DataStream(org.opensearch.cluster.metadata.DataStream) Metadata(org.opensearch.cluster.metadata.Metadata) IndexMetadata(org.opensearch.cluster.metadata.IndexMetadata) AliasMetadata(org.opensearch.cluster.metadata.AliasMetadata) IndexTemplateMetadata(org.opensearch.cluster.metadata.IndexTemplateMetadata) TestThreadPool(org.opensearch.threadpool.TestThreadPool) ThreadPool(org.opensearch.threadpool.ThreadPool) Index(org.opensearch.index.Index) Matchers.containsString(org.hamcrest.Matchers.containsString) TestThreadPool(org.opensearch.threadpool.TestThreadPool) MetadataFieldMapper(org.opensearch.index.mapper.MetadataFieldMapper) RoutingFieldMapper(org.opensearch.index.mapper.RoutingFieldMapper) FieldMapper(org.opensearch.index.mapper.FieldMapper) Mapper(org.opensearch.index.mapper.Mapper) DocumentMapper(org.opensearch.index.mapper.DocumentMapper) DateFieldMapper(org.opensearch.index.mapper.DateFieldMapper) MetadataCreateIndexService(org.opensearch.cluster.metadata.MetadataCreateIndexService) MappingLookup(org.opensearch.index.mapper.MappingLookup) ShardLimitValidator(org.opensearch.indices.ShardLimitValidator) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) CompressedXContent(org.opensearch.common.compress.CompressedXContent) ClusterName(org.opensearch.cluster.ClusterName) IndexMetadata(org.opensearch.cluster.metadata.IndexMetadata) AllocationService(org.opensearch.cluster.routing.allocation.AllocationService) DateFieldMapper(org.opensearch.index.mapper.DateFieldMapper) ClusterState(org.opensearch.cluster.ClusterState) DocumentMapper(org.opensearch.index.mapper.DocumentMapper) AliasValidator(org.opensearch.cluster.metadata.AliasValidator) RoutingFieldMapper(org.opensearch.index.mapper.RoutingFieldMapper) IndicesService(org.opensearch.indices.IndicesService) ContentPath(org.opensearch.index.mapper.ContentPath) IndexAbstraction(org.opensearch.cluster.metadata.IndexAbstraction) MetadataFieldMapper(org.opensearch.index.mapper.MetadataFieldMapper) ClusterService(org.opensearch.cluster.service.ClusterService) MetadataIndexAliasesService(org.opensearch.cluster.metadata.MetadataIndexAliasesService) StandardAnalyzer(org.apache.lucene.analysis.standard.StandardAnalyzer) Environment(org.opensearch.env.Environment) IndexNameExpressionResolver(org.opensearch.cluster.metadata.IndexNameExpressionResolver) SystemIndices(org.opensearch.indices.SystemIndices) CreateIndexRequest(org.opensearch.action.admin.indices.create.CreateIndexRequest)

Aggregations

DocumentMapper (org.opensearch.index.mapper.DocumentMapper)5 MappingLookup (org.opensearch.index.mapper.MappingLookup)5 Mapper (org.opensearch.index.mapper.Mapper)2 MapperService (org.opensearch.index.mapper.MapperService)2 MetaJoinFieldMapper (org.opensearch.join.mapper.MetaJoinFieldMapper)2 ParentJoinFieldMapper (org.opensearch.join.mapper.ParentJoinFieldMapper)2 HashMap (java.util.HashMap)1 StandardAnalyzer (org.apache.lucene.analysis.standard.StandardAnalyzer)1 Matchers.containsString (org.hamcrest.Matchers.containsString)1 CreateIndexRequest (org.opensearch.action.admin.indices.create.CreateIndexRequest)1 FieldMappingMetadata (org.opensearch.action.admin.indices.mapping.get.GetFieldMappingsResponse.FieldMappingMetadata)1 ClusterName (org.opensearch.cluster.ClusterName)1 ClusterState (org.opensearch.cluster.ClusterState)1 AliasMetadata (org.opensearch.cluster.metadata.AliasMetadata)1 AliasValidator (org.opensearch.cluster.metadata.AliasValidator)1 ComposableIndexTemplate (org.opensearch.cluster.metadata.ComposableIndexTemplate)1 DataStream (org.opensearch.cluster.metadata.DataStream)1 IndexAbstraction (org.opensearch.cluster.metadata.IndexAbstraction)1 IndexMetadata (org.opensearch.cluster.metadata.IndexMetadata)1 IndexNameExpressionResolver (org.opensearch.cluster.metadata.IndexNameExpressionResolver)1