Search in sources :

Example 1 with InvalidIndexTemplateException

use of org.opensearch.indices.InvalidIndexTemplateException in project OpenSearch by opensearch-project.

the class MetadataIndexTemplateService method validateV2TemplateRequest.

public static void validateV2TemplateRequest(Metadata metadata, String name, ComposableIndexTemplate template) {
    if (template.indexPatterns().stream().anyMatch(Regex::isMatchAllPattern)) {
        Settings mergedSettings = resolveSettings(metadata, template);
        if (IndexMetadata.INDEX_HIDDEN_SETTING.exists(mergedSettings)) {
            throw new InvalidIndexTemplateException(name, "global composable templates may not specify the setting " + IndexMetadata.INDEX_HIDDEN_SETTING.getKey());
        }
    }
    final Map<String, ComponentTemplate> componentTemplates = metadata.componentTemplates();
    final List<String> missingComponentTemplates = template.composedOf().stream().filter(componentTemplate -> componentTemplates.containsKey(componentTemplate) == false).collect(Collectors.toList());
    if (missingComponentTemplates.size() > 0) {
        throw new InvalidIndexTemplateException(name, "index template [" + name + "] specifies component templates " + missingComponentTemplates + " that do not exist");
    }
}
Also used : NO_LONGER_ASSIGNED(org.opensearch.indices.cluster.IndicesClusterStateService.AllocatedIndices.IndexRemovalReason.NO_LONGER_ASSIGNED) IndexScopedSettings(org.opensearch.common.settings.IndexScopedSettings) Version(org.opensearch.Version) Regex(org.opensearch.common.regex.Regex) Strings(org.opensearch.common.Strings) ObjectObjectCursor(com.carrotsearch.hppc.cursors.ObjectObjectCursor) MasterNodeRequest(org.opensearch.action.support.master.MasterNodeRequest) Operations(org.apache.lucene.util.automaton.Operations) MapperService(org.opensearch.index.mapper.MapperService) Alias(org.opensearch.action.admin.indices.alias.Alias) Locale(java.util.Locale) Map(java.util.Map) XContentFactory(org.opensearch.common.xcontent.XContentFactory) Inject(org.opensearch.common.inject.Inject) ActionListener(org.opensearch.action.ActionListener) HeaderWarning(org.opensearch.common.logging.HeaderWarning) TimeValue(org.opensearch.common.unit.TimeValue) Automaton(org.apache.lucene.util.automaton.Automaton) Index(org.opensearch.index.Index) Predicate(java.util.function.Predicate) MetadataCreateDataStreamService.validateTimestampFieldMapping(org.opensearch.cluster.metadata.MetadataCreateDataStreamService.validateTimestampFieldMapping) IndicesService(org.opensearch.indices.IndicesService) Set(java.util.Set) Settings(org.opensearch.common.settings.Settings) ObjectCursor(com.carrotsearch.hppc.cursors.ObjectCursor) Collectors(java.util.stream.Collectors) Nullable(org.opensearch.common.Nullable) ValidationException(org.opensearch.common.ValidationException) UncheckedIOException(java.io.UncheckedIOException) Objects(java.util.Objects) List(java.util.List) Logger(org.apache.logging.log4j.Logger) ClusterStateUpdateTask(org.opensearch.cluster.ClusterStateUpdateTask) Optional(java.util.Optional) XContentType(org.opensearch.common.xcontent.XContentType) IndexTemplateMissingException(org.opensearch.indices.IndexTemplateMissingException) BytesReference(org.opensearch.common.bytes.BytesReference) MapperParsingException(org.opensearch.index.mapper.MapperParsingException) CompressedXContent(org.opensearch.common.compress.CompressedXContent) Priority(org.opensearch.common.Priority) HashMap(java.util.HashMap) Function(java.util.function.Function) TreeSet(java.util.TreeSet) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) ClusterState(org.opensearch.cluster.ClusterState) UUIDs(org.opensearch.common.UUIDs) LinkedList(java.util.LinkedList) IOException(java.io.IOException) IndexService(org.opensearch.index.IndexService) CollectionUtil(org.apache.lucene.util.CollectionUtil) AcknowledgedResponse(org.opensearch.action.support.master.AcknowledgedResponse) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) Sets(org.opensearch.common.util.set.Sets) MergeReason(org.opensearch.index.mapper.MapperService.MergeReason) NamedXContentRegistry(org.opensearch.common.xcontent.NamedXContentRegistry) ClusterService(org.opensearch.cluster.service.ClusterService) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) Comparator(java.util.Comparator) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) Regex(org.opensearch.common.regex.Regex) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) IndexScopedSettings(org.opensearch.common.settings.IndexScopedSettings) Settings(org.opensearch.common.settings.Settings)

Example 2 with InvalidIndexTemplateException

use of org.opensearch.indices.InvalidIndexTemplateException in project OpenSearch by opensearch-project.

the class MetadataIndexTemplateService method validate.

private void validate(String name, @Nullable Settings settings, List<String> indexPatterns, List<Alias> aliases) {
    List<String> validationErrors = new ArrayList<>();
    if (name.contains(" ")) {
        validationErrors.add("name must not contain a space");
    }
    if (name.contains(",")) {
        validationErrors.add("name must not contain a ','");
    }
    if (name.contains("#")) {
        validationErrors.add("name must not contain a '#'");
    }
    if (name.contains("*")) {
        validationErrors.add("name must not contain a '*'");
    }
    if (name.startsWith("_")) {
        validationErrors.add("name must not start with '_'");
    }
    if (name.toLowerCase(Locale.ROOT).equals(name) == false) {
        validationErrors.add("name must be lower cased");
    }
    for (String indexPattern : indexPatterns) {
        if (indexPattern.contains(" ")) {
            validationErrors.add("index_patterns [" + indexPattern + "] must not contain a space");
        }
        if (indexPattern.contains(",")) {
            validationErrors.add("index_pattern [" + indexPattern + "] must not contain a ','");
        }
        if (indexPattern.contains("#")) {
            validationErrors.add("index_pattern [" + indexPattern + "] must not contain a '#'");
        }
        if (indexPattern.contains(":")) {
            validationErrors.add("index_pattern [" + indexPattern + "] must not contain a ':'");
        }
        if (indexPattern.startsWith("_")) {
            validationErrors.add("index_pattern [" + indexPattern + "] must not start with '_'");
        }
        if (Strings.validFileNameExcludingAstrix(indexPattern) == false) {
            validationErrors.add("index_pattern [" + indexPattern + "] must not contain the following characters " + Strings.INVALID_FILENAME_CHARS);
        }
    }
    if (settings != null) {
        try {
            // templates must be consistent with regards to dependencies
            indexScopedSettings.validate(settings, true);
        } catch (IllegalArgumentException iae) {
            validationErrors.add(iae.getMessage());
            for (Throwable t : iae.getSuppressed()) {
                validationErrors.add(t.getMessage());
            }
        }
        List<String> indexSettingsValidation = metadataCreateIndexService.getIndexSettingsValidationErrors(settings, true);
        validationErrors.addAll(indexSettingsValidation);
    }
    if (indexPatterns.stream().anyMatch(Regex::isMatchAllPattern)) {
        if (settings != null && IndexMetadata.INDEX_HIDDEN_SETTING.exists(settings)) {
            validationErrors.add("global templates may not specify the setting " + IndexMetadata.INDEX_HIDDEN_SETTING.getKey());
        }
    }
    if (validationErrors.size() > 0) {
        ValidationException validationException = new ValidationException();
        validationException.addValidationErrors(validationErrors);
        throw new InvalidIndexTemplateException(name, validationException.getMessage());
    }
    for (Alias alias : aliases) {
        // we validate the alias only partially, as we don't know yet to which index it'll get applied to
        aliasValidator.validateAliasStandalone(alias);
        if (indexPatterns.contains(alias.name())) {
            throw new IllegalArgumentException("alias [" + alias.name() + "] cannot be the same as any pattern in [" + String.join(", ", indexPatterns) + "]");
        }
    }
}
Also used : ValidationException(org.opensearch.common.ValidationException) Regex(org.opensearch.common.regex.Regex) Alias(org.opensearch.action.admin.indices.alias.Alias) ArrayList(java.util.ArrayList) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException)

Example 3 with InvalidIndexTemplateException

use of org.opensearch.indices.InvalidIndexTemplateException in project OpenSearch by opensearch-project.

the class ExceptionSerializationTests method testInvalidIndexTemplateException.

public void testInvalidIndexTemplateException() throws IOException {
    InvalidIndexTemplateException ex = serialize(new InvalidIndexTemplateException("foo", "bar"));
    assertEquals(ex.getMessage(), "index_template [foo] invalid, cause [bar]");
    assertEquals(ex.name(), "foo");
    ex = serialize(new InvalidIndexTemplateException(null, "bar"));
    assertEquals(ex.getMessage(), "index_template [null] invalid, cause [bar]");
    assertEquals(ex.name(), null);
}
Also used : InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException)

Example 4 with InvalidIndexTemplateException

use of org.opensearch.indices.InvalidIndexTemplateException in project OpenSearch by opensearch-project.

the class SearchPhaseExecutionExceptionTests method testToAndFromXContent.

public void testToAndFromXContent() throws IOException {
    final XContent xContent = randomFrom(XContentType.values()).xContent();
    ShardSearchFailure[] shardSearchFailures = new ShardSearchFailure[randomIntBetween(1, 5)];
    for (int i = 0; i < shardSearchFailures.length; i++) {
        Exception cause = randomFrom(new ParsingException(1, 2, "foobar", null), new InvalidIndexTemplateException("foo", "bar"), new TimestampParsingException("foo", null), new NullPointerException());
        shardSearchFailures[i] = new ShardSearchFailure(cause, new SearchShardTarget("node_" + i, new ShardId("test", "_na_", i), null, OriginalIndices.NONE));
    }
    final String phase = randomFrom("query", "search", "other");
    SearchPhaseExecutionException actual = new SearchPhaseExecutionException(phase, "unexpected failures", shardSearchFailures);
    BytesReference exceptionBytes = toShuffledXContent(actual, xContent.type(), ToXContent.EMPTY_PARAMS, randomBoolean());
    OpenSearchException parsedException;
    try (XContentParser parser = createParser(xContent, exceptionBytes)) {
        assertEquals(XContentParser.Token.START_OBJECT, parser.nextToken());
        parsedException = OpenSearchException.fromXContent(parser);
        assertEquals(XContentParser.Token.END_OBJECT, parser.currentToken());
        assertNull(parser.nextToken());
    }
    assertNotNull(parsedException);
    assertThat(parsedException.getHeaderKeys(), hasSize(0));
    assertThat(parsedException.getMetadataKeys(), hasSize(1));
    assertThat(parsedException.getMetadata("opensearch.phase"), hasItem(phase));
    // SearchPhaseExecutionException has no cause field
    assertNull(parsedException.getCause());
}
Also used : BytesReference(org.opensearch.common.bytes.BytesReference) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) OpenSearchRejectedExecutionException(org.opensearch.common.util.concurrent.OpenSearchRejectedExecutionException) IndexShardClosedException(org.opensearch.index.shard.IndexShardClosedException) IOException(java.io.IOException) OpenSearchException(org.opensearch.OpenSearchException) TimestampParsingException(org.opensearch.action.TimestampParsingException) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) ParsingException(org.opensearch.common.ParsingException) ShardId(org.opensearch.index.shard.ShardId) ToXContent(org.opensearch.common.xcontent.ToXContent) XContent(org.opensearch.common.xcontent.XContent) TimestampParsingException(org.opensearch.action.TimestampParsingException) ParsingException(org.opensearch.common.ParsingException) SearchShardTarget(org.opensearch.search.SearchShardTarget) TimestampParsingException(org.opensearch.action.TimestampParsingException) OpenSearchException(org.opensearch.OpenSearchException) XContentParser(org.opensearch.common.xcontent.XContentParser)

Example 5 with InvalidIndexTemplateException

use of org.opensearch.indices.InvalidIndexTemplateException in project OpenSearch by opensearch-project.

the class MetadataIndexTemplateServiceTests method testAddInvalidTemplate.

public void testAddInvalidTemplate() throws Exception {
    ComposableIndexTemplate template = new ComposableIndexTemplate(Collections.singletonList("a"), null, Arrays.asList("good", "bad"), null, null, null);
    ComponentTemplate ct = new ComponentTemplate(new Template(Settings.EMPTY, null, null), null, null);
    final MetadataIndexTemplateService service = getMetadataIndexTemplateService();
    CountDownLatch ctLatch = new CountDownLatch(1);
    service.putComponentTemplate("api", randomBoolean(), "good", TimeValue.timeValueSeconds(5), ct, ActionListener.wrap(r -> ctLatch.countDown(), e -> {
        logger.error("unexpected error", e);
        fail("unexpected error");
    }));
    ctLatch.await(5, TimeUnit.SECONDS);
    InvalidIndexTemplateException e = expectThrows(InvalidIndexTemplateException.class, () -> {
        CountDownLatch latch = new CountDownLatch(1);
        AtomicReference<Exception> err = new AtomicReference<>();
        service.putIndexTemplateV2("api", randomBoolean(), "template", TimeValue.timeValueSeconds(30), template, ActionListener.wrap(r -> fail("should have failed!"), exception -> {
            err.set(exception);
            latch.countDown();
        }));
        latch.await(5, TimeUnit.SECONDS);
        if (err.get() != null) {
            throw err.get();
        }
    });
    assertThat(e.name(), equalTo("template"));
    assertThat(e.getMessage(), containsString("index template [template] specifies " + "component templates [bad] that do not exist"));
}
Also used : Arrays(java.util.Arrays) OpenSearchSingleNodeTestCase(org.opensearch.test.OpenSearchSingleNodeTestCase) IndexScopedSettings(org.opensearch.common.settings.IndexScopedSettings) TIMESTAMP_FIELD(org.opensearch.index.mapper.DataStreamFieldMapper.Defaults.TIMESTAMP_FIELD) Version(org.opensearch.Version) Strings(org.opensearch.common.Strings) Collections.singletonList(java.util.Collections.singletonList) XContentParser(org.opensearch.common.xcontent.XContentParser) CoreMatchers.instanceOf(org.hamcrest.CoreMatchers.instanceOf) MapperService(org.opensearch.index.mapper.MapperService) Alias(org.opensearch.action.admin.indices.alias.Alias) Settings.builder(org.opensearch.common.settings.Settings.builder) Map(java.util.Map) XContentFactory(org.opensearch.common.xcontent.XContentFactory) ActionListener(org.opensearch.action.ActionListener) TimeValue(org.opensearch.common.unit.TimeValue) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) Index(org.opensearch.index.Index) LoggingDeprecationHandler(org.opensearch.common.xcontent.LoggingDeprecationHandler) IndicesService(org.opensearch.indices.IndicesService) Set(java.util.Set) Settings(org.opensearch.common.settings.Settings) PutRequest(org.opensearch.cluster.metadata.MetadataIndexTemplateService.PutRequest) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) CountDownLatch(java.util.concurrent.CountDownLatch) List(java.util.List) ShardLimitValidatorTests.createTestShardLimitService(org.opensearch.indices.ShardLimitValidatorTests.createTestShardLimitService) Matchers.contains(org.hamcrest.Matchers.contains) Matchers.containsInAnyOrder(org.hamcrest.Matchers.containsInAnyOrder) XContentType(org.opensearch.common.xcontent.XContentType) Matchers.is(org.hamcrest.Matchers.is) IndexTemplateMissingException(org.opensearch.indices.IndexTemplateMissingException) CoreMatchers.equalTo(org.hamcrest.CoreMatchers.equalTo) MapperParsingException(org.opensearch.index.mapper.MapperParsingException) CompressedXContent(org.opensearch.common.compress.CompressedXContent) CoreMatchers.not(org.hamcrest.CoreMatchers.not) HashMap(java.util.HashMap) AtomicReference(java.util.concurrent.atomic.AtomicReference) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Matchers.matchesRegex(org.hamcrest.Matchers.matchesRegex) ClusterState(org.opensearch.cluster.ClusterState) Environment(org.opensearch.env.Environment) Matchers.empty(org.hamcrest.Matchers.empty) IOException(java.io.IOException) AcknowledgedResponse(org.opensearch.action.support.master.AcknowledgedResponse) TimeUnit(java.util.concurrent.TimeUnit) SystemIndices(org.opensearch.indices.SystemIndices) NamedXContentRegistry(org.opensearch.common.xcontent.NamedXContentRegistry) ClusterService(org.opensearch.cluster.service.ClusterService) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) CoreMatchers.containsStringIgnoringCase(org.hamcrest.CoreMatchers.containsStringIgnoringCase) Collections(java.util.Collections) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException) AtomicReference(java.util.concurrent.atomic.AtomicReference) CountDownLatch(java.util.concurrent.CountDownLatch) IndexTemplateMissingException(org.opensearch.indices.IndexTemplateMissingException) MapperParsingException(org.opensearch.index.mapper.MapperParsingException) IOException(java.io.IOException) InvalidIndexTemplateException(org.opensearch.indices.InvalidIndexTemplateException)

Aggregations

InvalidIndexTemplateException (org.opensearch.indices.InvalidIndexTemplateException)7 IOException (java.io.IOException)4 ArrayList (java.util.ArrayList)3 Alias (org.opensearch.action.admin.indices.alias.Alias)3 Collections (java.util.Collections)2 HashMap (java.util.HashMap)2 HashSet (java.util.HashSet)2 List (java.util.List)2 Map (java.util.Map)2 Objects (java.util.Objects)2 Set (java.util.Set)2 Collectors (java.util.stream.Collectors)2 Version (org.opensearch.Version)2 ActionListener (org.opensearch.action.ActionListener)2 AcknowledgedResponse (org.opensearch.action.support.master.AcknowledgedResponse)2 ClusterState (org.opensearch.cluster.ClusterState)2 ClusterService (org.opensearch.cluster.service.ClusterService)2 Strings (org.opensearch.common.Strings)2 ValidationException (org.opensearch.common.ValidationException)2 BytesReference (org.opensearch.common.bytes.BytesReference)2