Search in sources :

Example 1 with NewIndex

use of org.sonar.server.es.newindex.NewIndex in project sonarqube by SonarSource.

the class EsTester method createIndices.

private static List<BuiltIndex> createIndices(IndexDefinition... definitions) {
    IndexDefinitionContext context = new IndexDefinitionContext();
    Stream.of(definitions).forEach(d -> d.define(context));
    List<BuiltIndex> result = new ArrayList<>();
    for (NewIndex newIndex : context.getIndices().values()) {
        BuiltIndex index = newIndex.build();
        String indexName = index.getMainType().getIndex().getName();
        deleteIndexIfExists(indexName);
        // create index
        Settings.Builder settings = Settings.builder();
        settings.put(index.getSettings());
        CreateIndexResponse indexResponse = createIndex(indexName, settings);
        if (!indexResponse.isAcknowledged()) {
            throw new IllegalStateException("Failed to create index " + indexName);
        }
        waitForClusterYellowStatus(indexName);
        // create types
        String typeName = index.getMainType().getType();
        putIndexMapping(index, indexName, typeName);
        waitForClusterYellowStatus(indexName);
        result.add(index);
    }
    return result;
}
Also used : BuiltIndex(org.sonar.server.es.newindex.BuiltIndex) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) ArrayList(java.util.ArrayList) NewIndex(org.sonar.server.es.newindex.NewIndex) CreateIndexResponse(org.elasticsearch.client.indices.CreateIndexResponse) IndexDefinitionContext(org.sonar.server.es.IndexDefinition.IndexDefinitionContext) RecoverySettings(org.elasticsearch.indices.recovery.RecoverySettings) Settings(org.elasticsearch.common.settings.Settings) HttpTransportSettings(org.elasticsearch.http.HttpTransportSettings) DiskThresholdSettings(org.elasticsearch.cluster.routing.allocation.DiskThresholdSettings)

Example 2 with NewIndex

use of org.sonar.server.es.newindex.NewIndex in project sonarqube by SonarSource.

the class ViewIndexDefinitionTest method define.

@Test
public void define() {
    ViewIndexDefinition def = new ViewIndexDefinition(new MapSettings().asConfig());
    def.define(underTest);
    assertThat(underTest.getIndices()).hasSize(1);
    NewIndex index = underTest.getIndices().get("views");
    assertThat(index.getMainType()).isEqualTo(IndexType.main(Index.simple("views"), "view"));
    assertThat(index.getRelationsStream()).isEmpty();
    assertThat(index.getSetting("index.number_of_shards")).isEqualTo("5");
    assertThat(index.getSetting("index.number_of_replicas")).isEqualTo("0");
}
Also used : MapSettings(org.sonar.api.config.internal.MapSettings) NewIndex(org.sonar.server.es.newindex.NewIndex) Test(org.junit.Test)

Example 3 with NewIndex

use of org.sonar.server.es.newindex.NewIndex in project sonarqube by SonarSource.

the class UserIndexDefinitionTest method define.

@Test
public void define() {
    UserIndexDefinition def = new UserIndexDefinition(new MapSettings().asConfig());
    def.define(underTest);
    assertThat(underTest.getIndices()).hasSize(1);
    NewIndex index = underTest.getIndices().get("users");
    assertThat(index.getMainType()).isEqualTo(IndexType.main(Index.simple("users"), "user"));
    assertThat(index.getRelationsStream()).isEmpty();
    // no cluster by default
    assertThat(index.getSetting("index.number_of_shards")).isEqualTo("1");
    assertThat(index.getSetting("index.number_of_replicas")).isEqualTo("0");
}
Also used : MapSettings(org.sonar.api.config.internal.MapSettings) NewIndex(org.sonar.server.es.newindex.NewIndex) Test(org.junit.Test)

Example 4 with NewIndex

use of org.sonar.server.es.newindex.NewIndex in project sonarqube by SonarSource.

the class IndexCreator method start.

@Override
public void start() {
    // create the "metadata" index first
    IndexType.IndexMainType metadataMainType = TYPE_METADATA;
    if (!client.indexExists(new GetIndexRequest(metadataMainType.getIndex().getName()))) {
        IndexDefinition.IndexDefinitionContext context = new IndexDefinition.IndexDefinitionContext();
        metadataIndexDefinition.define(context);
        NewIndex index = context.getIndices().values().iterator().next();
        createIndex(index.build(), false);
    } else {
        ensureWritable(metadataMainType);
    }
    checkDbCompatibility(definitions.getIndices().values());
    // create indices that do not exist or that have a new definition (different mapping, cluster enabled, ...)
    definitions.getIndices().values().stream().filter(i -> !i.getMainType().equals(metadataMainType)).forEach(index -> {
        boolean exists = client.indexExists(new GetIndexRequest(index.getMainType().getIndex().getName()));
        if (!exists) {
            createIndex(index, true);
        } else if (hasDefinitionChange(index)) {
            updateIndex(index);
        } else {
            ensureWritable(index.getMainType());
        }
    });
}
Also used : Arrays(java.util.Arrays) StringUtils(org.apache.commons.lang.StringUtils) BuiltIndex(org.sonar.server.es.newindex.BuiltIndex) UpdateSettingsRequest(org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) Loggers(org.sonar.api.utils.log.Loggers) Settings(org.elasticsearch.common.settings.Settings) Configuration(org.sonar.api.config.Configuration) MetadataIndex(org.sonar.server.es.metadata.MetadataIndex) MetadataIndexDefinition(org.sonar.server.es.metadata.MetadataIndexDefinition) DESCRIPTOR(org.sonar.server.es.metadata.MetadataIndexDefinition.DESCRIPTOR) Logger(org.sonar.api.utils.log.Logger) TYPE_METADATA(org.sonar.server.es.metadata.MetadataIndexDefinition.TYPE_METADATA) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.elasticsearch.client.indices.CreateIndexRequest) Startable(org.sonar.api.Startable) Collection(java.util.Collection) EsDbCompatibility(org.sonar.server.es.metadata.EsDbCompatibility) Set(java.util.Set) AcknowledgedResponse(org.elasticsearch.action.support.master.AcknowledgedResponse) Collectors(java.util.stream.Collectors) List(java.util.List) ProcessProperties(org.sonar.process.ProcessProperties) NewIndex(org.sonar.server.es.newindex.NewIndex) PutMappingRequest(org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest) ServerSide(org.sonar.api.server.ServerSide) CreateIndexResponse(org.elasticsearch.client.indices.CreateIndexResponse) MigrationEsClient(org.sonar.server.platform.db.migration.es.MigrationEsClient) ClusterHealthStatus(org.elasticsearch.cluster.health.ClusterHealthStatus) GetSettingsRequest(org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest) GetIndexRequest(org.elasticsearch.client.indices.GetIndexRequest) MetadataIndexDefinition(org.sonar.server.es.metadata.MetadataIndexDefinition) GetIndexRequest(org.elasticsearch.client.indices.GetIndexRequest) NewIndex(org.sonar.server.es.newindex.NewIndex)

Example 5 with NewIndex

use of org.sonar.server.es.newindex.NewIndex in project sonarqube by SonarSource.

the class RuleIndexDefinitionTest method enable_replica_if_clustering_is_enabled.

@Test
public void enable_replica_if_clustering_is_enabled() {
    settings.setProperty(CLUSTER_ENABLED.getKey(), true);
    IndexDefinition.IndexDefinitionContext context = new IndexDefinition.IndexDefinitionContext();
    underTest.define(context);
    NewIndex ruleIndex = context.getIndices().get("rules");
    assertThat(ruleIndex.getSetting("index.number_of_replicas")).isEqualTo("1");
}
Also used : IndexDefinition(org.sonar.server.es.IndexDefinition) NewIndex(org.sonar.server.es.newindex.NewIndex) Test(org.junit.Test)

Aggregations

NewIndex (org.sonar.server.es.newindex.NewIndex)5 Test (org.junit.Test)3 CreateIndexResponse (org.elasticsearch.client.indices.CreateIndexResponse)2 Settings (org.elasticsearch.common.settings.Settings)2 MapSettings (org.sonar.api.config.internal.MapSettings)2 BuiltIndex (org.sonar.server.es.newindex.BuiltIndex)2 Lists.newArrayList (com.google.common.collect.Lists.newArrayList)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Collection (java.util.Collection)1 List (java.util.List)1 Set (java.util.Set)1 Collectors (java.util.stream.Collectors)1 StringUtils (org.apache.commons.lang.StringUtils)1 DeleteIndexRequest (org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)1 PutMappingRequest (org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest)1 GetSettingsRequest (org.elasticsearch.action.admin.indices.settings.get.GetSettingsRequest)1 UpdateSettingsRequest (org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest)1 AcknowledgedResponse (org.elasticsearch.action.support.master.AcknowledgedResponse)1 CreateIndexRequest (org.elasticsearch.client.indices.CreateIndexRequest)1