Search in sources :

Example 11 with AliasMetadata

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata in project elasticsearch-skywalker by jprante.

the class MetaDataToXContent method toXContent.

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
    builder.startObject();
    builder.field("version", metadata.version());
    builder.field("numberOfShards", metadata.numberOfShards());
    builder.startArray("concreteIndices");
    for (String index : metadata.concreteAllIndices()) {
        builder.value(index);
    }
    builder.endArray();
    ImmutableOpenMap<String, ImmutableOpenMap<String, AliasMetaData>> aliases = metadata.getAliases();
    builder.startArray("aliases");
    for (ObjectCursor<String> alias : aliases.keys()) {
        builder.startObject(alias.value);
        builder.startArray("aliasMetadata");
        for (ObjectCursor<String> s : aliases.get(alias.value).keys()) {
            builder.startObject();
            AliasMetaData amd = aliases.get(alias.value).get(s.value);
            builder.field("alias", amd.getAlias());
            builder.field("filter", amd.getFilter().string());
            builder.field("indexRouting", amd.getIndexRouting());
            builder.field("searchRouting", amd.getSearchRouting());
            builder.endObject();
        }
        builder.endArray();
        builder.endObject();
    }
    builder.endArray();
    builder.startArray("indexes");
    ImmutableOpenMap<String, IndexMetaData> indices = metadata.getIndices();
    for (ObjectCursor<String> s : indices.keys()) {
        IndexMetaData imd = indices.get(s.value);
        builder.startObject();
        builder.field("index", imd.getIndex());
        builder.field("state", imd.getState().name());
        builder.field("numberOfReplicas", imd.getNumberOfReplicas());
        builder.field("numberOfShards", imd.getNumberOfShards());
        builder.field("totalNumberOfShards", imd.getTotalNumberOfShards());
        builder.field("version", imd.getVersion());
        builder.field("settings", imd.getSettings().getAsMap());
        ImmutableOpenMap<String, MappingMetaData> m = imd.getMappings();
        // skip mappings here
        builder.endObject();
    }
    builder.endArray();
    builder.startArray("templates");
    ImmutableOpenMap<String, IndexTemplateMetaData> templates = metadata.getTemplates();
    for (ObjectCursor<String> s : templates.keys()) {
        IndexTemplateMetaData itmd = templates.get(s.value);
        itmd.getName();
        itmd.getOrder();
        itmd.getTemplate();
        itmd.getSettings();
        itmd.getMappings();
    }
    builder.endArray();
    builder.endObject();
    return builder;
}
Also used : AliasMetaData(org.elasticsearch.cluster.metadata.AliasMetaData) IndexTemplateMetaData(org.elasticsearch.cluster.metadata.IndexTemplateMetaData) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) MappingMetaData(org.elasticsearch.cluster.metadata.MappingMetaData) IndexMetaData(org.elasticsearch.cluster.metadata.IndexMetaData)

Example 12 with AliasMetadata

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata in project crate by crate.

the class TransportCreatePartitionsAction method applyTemplates.

private void applyTemplates(Map<String, Map<String, Object>> mappings, Map<String, AliasMetadata> templatesAliases, List<String> templateNames, List<IndexTemplateMetadata> templates) throws Exception {
    for (IndexTemplateMetadata template : templates) {
        templateNames.add(template.getName());
        for (ObjectObjectCursor<String, CompressedXContent> cursor : template.mappings()) {
            if (mappings.containsKey(cursor.key)) {
                XContentHelper.mergeDefaults(mappings.get(cursor.key), parseMapping(cursor.value.string()));
            } else {
                mappings.put(cursor.key, parseMapping(cursor.value.string()));
            }
        }
        // handle aliases
        for (ObjectObjectCursor<String, AliasMetadata> cursor : template.aliases()) {
            AliasMetadata aliasMetadata = cursor.value;
            templatesAliases.put(aliasMetadata.alias(), aliasMetadata);
        }
    }
}
Also used : AliasMetadata(org.elasticsearch.cluster.metadata.AliasMetadata) IndexTemplateMetadata(org.elasticsearch.cluster.metadata.IndexTemplateMetadata) CompressedXContent(org.elasticsearch.common.compress.CompressedXContent)

Example 13 with AliasMetadata

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method aliases.

@Override
public Map<String, Set<String>> aliases(String indexPattern) {
    final GetAliasesRequest request = new GetAliasesRequest().indices(indexPattern).indicesOptions(IndicesOptions.fromOptions(false, false, true, false));
    final GetAliasesResponse result = client.execute((c, requestOptions) -> c.indices().getAlias(request, requestOptions), "Couldn't collect aliases for index pattern " + indexPattern);
    return result.getAliases().entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey, entry -> entry.getValue().stream().map(AliasMetadata::alias).collect(Collectors.toSet())));
}
Also used : Terms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.Terms) DateTimeZone(org.joda.time.DateTimeZone) Arrays(java.util.Arrays) MultiBucketsAggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation) LoggerFactory(org.slf4j.LoggerFactory) OpenIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.open.OpenIndexRequest) IndicesAdapter(org.graylog2.indexer.indices.IndicesAdapter) HealthStatus(org.graylog2.indexer.indices.HealthStatus) TimeValue(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.unit.TimeValue) Indices(org.graylog2.indexer.indices.Indices) Locale(java.util.Locale) Map(java.util.Map) GetAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) IndicesOptions(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.IndicesOptions) JsonNode(com.fasterxml.jackson.databind.JsonNode) IndexRangeStats(org.graylog2.indexer.searches.IndexRangeStats) SearchType(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchType) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) StatsApi(org.graylog.storage.elasticsearch7.stats.StatsApi) Collection(java.util.Collection) DeleteIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) Set(java.util.Set) Max(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Max) IndexTemplatesExistRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.IndexTemplatesExistRequest) ClusterStateApi(org.graylog.storage.elasticsearch7.cluster.ClusterStateApi) Collectors(java.util.stream.Collectors) UpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) List(java.util.List) AliasMetadata(org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata) GetSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest) GetAliasesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.GetAliasesResponse) IndexMoveResult(org.graylog2.indexer.indices.IndexMoveResult) GetSettingsResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.get.GetSettingsResponse) Optional(java.util.Optional) BulkByScrollResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.reindex.BulkByScrollResponse) Min(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Min) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) ClusterHealthResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse) FlushRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.flush.FlushRequest) HashMap(java.util.HashMap) CloseIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CloseIndexRequest) AggregationBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilders) Inject(javax.inject.Inject) PutIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest) PutMappingRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutMappingRequest) IndexStatistics(org.graylog2.indexer.indices.stats.IndexStatistics) Duration(com.github.joschi.jadconfig.util.Duration) Filter(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.Filter) Nonnull(javax.annotation.Nonnull) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) FilterAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) IndexSettings(org.graylog2.indexer.indices.IndexSettings) ElasticsearchClient.withTimeout(org.graylog.storage.elasticsearch7.ElasticsearchClient.withTimeout) CatApi(org.graylog.storage.elasticsearch7.cat.CatApi) IndexNotFoundException(org.graylog2.indexer.IndexNotFoundException) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) DateTime(org.joda.time.DateTime) AcknowledgedResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.master.AcknowledgedResponse) ReindexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.reindex.ReindexRequest) ForceMergeRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequest) ClusterHealthRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) DeleteIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest) Consumer(java.util.function.Consumer) DeleteAliasRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.DeleteAliasRequest) Collectors.toList(java.util.stream.Collectors.toList) IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) QueryBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.query.QueryBuilders) Message(org.graylog2.plugin.Message) Collections(java.util.Collections) GetAliasesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.GetAliasesResponse) GetAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.get.GetAliasesRequest) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) HashMap(java.util.HashMap)

Example 14 with AliasMetadata

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata in project crate by crate.

the class DocIndexMetaDataTest method testMergePartitionWithDifferentShardsAndReplicas.

@Test
public void testMergePartitionWithDifferentShardsAndReplicas() throws Exception {
    XContentBuilder builder = XContentFactory.jsonBuilder().startObject().startObject("_meta").field("primary_keys", "id").startArray("partitioned_by").startArray().value("datum").value("date").endArray().endArray().endObject().startObject("properties").startObject("id").field("type", "integer").field("index", "not_analyzed").endObject().endObject();
    Settings templateSettings = Settings.builder().put(IndexMetaData.SETTING_NUMBER_OF_REPLICAS, 2).put(IndexMetaData.SETTING_NUMBER_OF_SHARDS, 6).build();
    IndexMetaData metaData = getIndexMetaData("test1", builder, templateSettings, null);
    DocIndexMetaData md = newMeta(metaData, "test1");
    PartitionName partitionName = new PartitionName("test1", Arrays.asList(new BytesRef("0")));
    Settings partitionSettings = Settings.builder().put(IndexMetaData.SETTING_NUMBER_OF_REPLICAS, 10).put(IndexMetaData.SETTING_NUMBER_OF_SHARDS, 50).build();
    AliasMetaData aliasMetaData = AliasMetaData.newAliasMetaDataBuilder("test1").build();
    IndexMetaData partitionMetaData = getIndexMetaData(partitionName.asIndexName(), builder, partitionSettings, aliasMetaData);
    DocIndexMetaData partitionMD = newMeta(partitionMetaData, partitionName.asIndexName());
    assertThat(partitionMD.aliases().size(), is(1));
    assertThat(partitionMD.aliases(), hasItems("test1"));
    assertThat(partitionMD.isAlias(), is(false));
    assertThat(partitionMD.concreteIndexName(), is(partitionName.asIndexName()));
    DocIndexMetaData merged = md.merge(partitionMD, mock(TransportPutIndexTemplateAction.class), true);
    assertThat(merged.numberOfReplicas(), is(BytesRefs.toBytesRef(2)));
    assertThat(merged.numberOfShards(), is(6));
}
Also used : AliasMetaData(org.elasticsearch.cluster.metadata.AliasMetaData) TransportPutIndexTemplateAction(org.elasticsearch.action.admin.indices.template.put.TransportPutIndexTemplateAction) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder) Settings(org.elasticsearch.common.settings.Settings) BytesRef(org.apache.lucene.util.BytesRef) IndexMetaData(org.elasticsearch.cluster.metadata.IndexMetaData) Test(org.junit.Test) CrateUnitTest(io.crate.test.integration.CrateUnitTest)

Example 15 with AliasMetadata

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.metadata.AliasMetadata in project elasticsearch by elastic.

the class RestGetIndicesAction method prepareRequest.

@Override
public RestChannelConsumer prepareRequest(final RestRequest request, final NodeClient client) throws IOException {
    String[] indices = Strings.splitStringByCommaToArray(request.param("index"));
    String[] featureParams = request.paramAsStringArray("type", null);
    // Work out if the indices is a list of features
    if (featureParams == null && indices.length > 0 && indices[0] != null && indices[0].startsWith("_") && !"_all".equals(indices[0])) {
        featureParams = indices;
        indices = new String[] { "_all" };
    }
    final GetIndexRequest getIndexRequest = new GetIndexRequest();
    getIndexRequest.indices(indices);
    if (featureParams != null) {
        Feature[] features = Feature.convertToFeatures(featureParams);
        getIndexRequest.features(features);
    }
    getIndexRequest.indicesOptions(IndicesOptions.fromRequest(request, getIndexRequest.indicesOptions()));
    getIndexRequest.local(request.paramAsBoolean("local", getIndexRequest.local()));
    getIndexRequest.humanReadable(request.paramAsBoolean("human", false));
    final boolean defaults = request.paramAsBoolean("include_defaults", false);
    return channel -> client.admin().indices().getIndex(getIndexRequest, new RestBuilderListener<GetIndexResponse>(channel) {

        @Override
        public RestResponse buildResponse(final GetIndexResponse response, final XContentBuilder builder) throws Exception {
            builder.startObject();
            {
                for (final String index : response.indices()) {
                    builder.startObject(index);
                    {
                        for (final Feature feature : getIndexRequest.features()) {
                            switch(feature) {
                                case ALIASES:
                                    writeAliases(response.aliases().get(index), builder, request);
                                    break;
                                case MAPPINGS:
                                    writeMappings(response.mappings().get(index), builder);
                                    break;
                                case SETTINGS:
                                    writeSettings(response.settings().get(index), builder, request, defaults);
                                    break;
                                default:
                                    throw new IllegalStateException("feature [" + feature + "] is not valid");
                            }
                        }
                    }
                    builder.endObject();
                }
            }
            builder.endObject();
            return new BytesRestResponse(OK, builder);
        }

        private void writeAliases(final List<AliasMetaData> aliases, final XContentBuilder builder, final Params params) throws IOException {
            builder.startObject("aliases");
            {
                if (aliases != null) {
                    for (final AliasMetaData alias : aliases) {
                        AliasMetaData.Builder.toXContent(alias, builder, params);
                    }
                }
            }
            builder.endObject();
        }

        private void writeMappings(final ImmutableOpenMap<String, MappingMetaData> mappings, final XContentBuilder builder) throws IOException {
            builder.startObject("mappings");
            {
                if (mappings != null) {
                    for (final ObjectObjectCursor<String, MappingMetaData> typeEntry : mappings) {
                        builder.field(typeEntry.key);
                        builder.map(typeEntry.value.sourceAsMap());
                    }
                }
            }
            builder.endObject();
        }

        private void writeSettings(final Settings settings, final XContentBuilder builder, final Params params, final boolean defaults) throws IOException {
            builder.startObject("settings");
            {
                settings.toXContent(builder, params);
            }
            builder.endObject();
            if (defaults) {
                builder.startObject("defaults");
                {
                    settingsFilter.filter(indexScopedSettings.diff(settings, RestGetIndicesAction.this.settings)).toXContent(builder, request);
                }
                builder.endObject();
            }
        }
    });
}
Also used : BaseRestHandler(org.elasticsearch.rest.BaseRestHandler) GET(org.elasticsearch.rest.RestRequest.Method.GET) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) MappingMetaData(org.elasticsearch.cluster.metadata.MappingMetaData) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder) Strings(org.elasticsearch.common.Strings) ObjectObjectCursor(com.carrotsearch.hppc.cursors.ObjectObjectCursor) Feature(org.elasticsearch.action.admin.indices.get.GetIndexRequest.Feature) Settings(org.elasticsearch.common.settings.Settings) GetIndexResponse(org.elasticsearch.action.admin.indices.get.GetIndexResponse) IndicesOptions(org.elasticsearch.action.support.IndicesOptions) RestRequest(org.elasticsearch.rest.RestRequest) OK(org.elasticsearch.rest.RestStatus.OK) NodeClient(org.elasticsearch.client.node.NodeClient) HEAD(org.elasticsearch.rest.RestRequest.Method.HEAD) SettingsFilter(org.elasticsearch.common.settings.SettingsFilter) GetIndexRequest(org.elasticsearch.action.admin.indices.get.GetIndexRequest) RestResponse(org.elasticsearch.rest.RestResponse) Params(org.elasticsearch.common.xcontent.ToXContent.Params) AliasMetaData(org.elasticsearch.cluster.metadata.AliasMetaData) RestBuilderListener(org.elasticsearch.rest.action.RestBuilderListener) Set(java.util.Set) IOException(java.io.IOException) RestController(org.elasticsearch.rest.RestController) IndexScopedSettings(org.elasticsearch.common.settings.IndexScopedSettings) BytesRestResponse(org.elasticsearch.rest.BytesRestResponse) List(java.util.List) RestResponse(org.elasticsearch.rest.RestResponse) BytesRestResponse(org.elasticsearch.rest.BytesRestResponse) Params(org.elasticsearch.common.xcontent.ToXContent.Params) IOException(java.io.IOException) MappingMetaData(org.elasticsearch.cluster.metadata.MappingMetaData) Feature(org.elasticsearch.action.admin.indices.get.GetIndexRequest.Feature) IOException(java.io.IOException) AliasMetaData(org.elasticsearch.cluster.metadata.AliasMetaData) GetIndexResponse(org.elasticsearch.action.admin.indices.get.GetIndexResponse) GetIndexRequest(org.elasticsearch.action.admin.indices.get.GetIndexRequest) BytesRestResponse(org.elasticsearch.rest.BytesRestResponse) XContentBuilder(org.elasticsearch.common.xcontent.XContentBuilder) Settings(org.elasticsearch.common.settings.Settings) IndexScopedSettings(org.elasticsearch.common.settings.IndexScopedSettings)

Aggregations

AliasMetaData (org.elasticsearch.cluster.metadata.AliasMetaData)20 List (java.util.List)13 Settings (org.elasticsearch.common.settings.Settings)10 ArrayList (java.util.ArrayList)9 ImmutableOpenMap (org.elasticsearch.common.collect.ImmutableOpenMap)8 ObjectObjectCursor (com.carrotsearch.hppc.cursors.ObjectObjectCursor)5 IOException (java.io.IOException)5 HashMap (java.util.HashMap)5 Map (java.util.Map)5 Set (java.util.Set)5 IndicesOptions (org.elasticsearch.action.support.IndicesOptions)4 IndexTemplateMetaData (org.elasticsearch.cluster.metadata.IndexTemplateMetaData)4 Index (org.elasticsearch.index.Index)4 Arrays (java.util.Arrays)3 Collections (java.util.Collections)3 Optional (java.util.Optional)3 Collectors (java.util.stream.Collectors)3 GetAliasesResponse (org.elasticsearch.action.admin.indices.alias.get.GetAliasesResponse)3 AliasMetadata (org.elasticsearch.cluster.metadata.AliasMetadata)3 IndexMetaData (org.elasticsearch.cluster.metadata.IndexMetaData)3