Search in sources :

Example 81 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project sonarqube by SonarSource.

the class UserIndex method search.

public SearchResult<UserDoc> search(UserQuery userQuery, SearchOptions options) {
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().size(options.getLimit()).from(options.getOffset()).sort(FIELD_NAME, SortOrder.ASC);
    BoolQueryBuilder filter = boolQuery().must(termQuery(FIELD_ACTIVE, true));
    QueryBuilder esQuery = matchAllQuery();
    Optional<String> textQuery = userQuery.getTextQuery();
    if (textQuery.isPresent()) {
        esQuery = QueryBuilders.multiMatchQuery(textQuery.get(), FIELD_LOGIN, USER_SEARCH_GRAMS_ANALYZER.subField(FIELD_LOGIN), FIELD_NAME, USER_SEARCH_GRAMS_ANALYZER.subField(FIELD_NAME), FIELD_EMAIL, USER_SEARCH_GRAMS_ANALYZER.subField(FIELD_EMAIL)).operator(Operator.AND);
    }
    SearchRequest request = EsClient.prepareSearch(UserIndexDefinition.TYPE_USER).source(searchSourceBuilder.query(boolQuery().must(esQuery).filter(filter)));
    return new SearchResult<>(esClient.search(request), UserDoc::new, system2.getDefaultTimeZone().toZoneId());
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) SearchResult(org.sonar.server.es.SearchResult) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Example 82 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project sonarqube by SonarSource.

the class RuleIndex method search.

public SearchIdResult<String> search(RuleQuery query, SearchOptions options) {
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    QueryBuilder qb = buildQuery(query);
    Map<String, QueryBuilder> filters = buildFilters(query);
    if (!options.getFacets().isEmpty()) {
        for (AggregationBuilder aggregation : getFacets(query, options, qb, filters).values()) {
            sourceBuilder.aggregation(aggregation);
        }
    }
    setSorting(query, sourceBuilder);
    setPagination(options, sourceBuilder);
    BoolQueryBuilder fb = boolQuery();
    for (QueryBuilder filterBuilder : filters.values()) {
        fb.must(filterBuilder);
    }
    sourceBuilder.query(boolQuery().must(qb).filter(fb));
    SearchRequest esSearch = EsClient.prepareSearch(TYPE_RULE).source(sourceBuilder);
    return new SearchIdResult<>(client.search(esSearch), input -> input, system2.getDefaultTimeZone().toZoneId());
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) AggregationBuilder(org.elasticsearch.search.aggregations.AggregationBuilder) TermsAggregationBuilder(org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) HasParentQueryBuilder(org.elasticsearch.join.query.HasParentQueryBuilder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) SearchIdResult(org.sonar.server.es.SearchIdResult) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Example 83 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project sonarqube by SonarSource.

the class EsTester method after.

@Override
protected void after() {
    if (isCustom) {
        // delete non-core indices
        String[] existingIndices = getIndicesNames();
        Stream.of(existingIndices).filter(i -> !CORE_INDICES_NAMES.contains(i)).forEach(EsTester::deleteIndexIfExists);
    }
    BulkIndexer.delete(ES_REST_CLIENT, IndexType.main(ALL_INDICES, "dummy"), EsClient.prepareSearch(ALL_INDICES.getName()).source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery())));
}
Also used : IndexRelationType(org.sonar.server.es.IndexType.IndexRelationType) BuiltIndex(org.sonar.server.es.newindex.BuiltIndex) ComponentIndexDefinition(org.sonar.server.component.index.ComponentIndexDefinition) Environment(org.elasticsearch.env.Environment) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) ALL_INDICES(org.sonar.server.es.Index.ALL_INDICES) Random(java.util.Random) Collections2(com.google.common.collect.Collections2) BindHttpException(org.elasticsearch.http.BindHttpException) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) ParentJoinPlugin(org.elasticsearch.join.ParentJoinPlugin) NodeValidationException(org.elasticsearch.node.NodeValidationException) ServerSocket(java.net.ServerSocket) RecoverySettings(org.elasticsearch.indices.recovery.RecoverySettings) Loggers(org.sonar.api.utils.log.Loggers) IndexRequest(org.elasticsearch.action.index.IndexRequest) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Settings(org.elasticsearch.common.settings.Settings) Map(java.util.Map) SearchResponse(org.elasticsearch.action.search.SearchResponse) RequestOptions(org.elasticsearch.client.RequestOptions) HttpTransportSettings(org.elasticsearch.http.HttpTransportSettings) ElasticsearchStatusException(org.elasticsearch.ElasticsearchStatusException) Path(java.nio.file.Path) TimeValue(org.elasticsearch.core.TimeValue) SearchHit(org.elasticsearch.search.SearchHit) ImmutableMap(com.google.common.collect.ImmutableMap) ConstructorUtils(org.apache.commons.lang.reflect.ConstructorUtils) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) REFRESH_IMMEDIATE(org.sonar.server.es.newindex.DefaultIndexSettings.REFRESH_IMMEDIATE) Set(java.util.Set) ProjectMeasuresIndexDefinition(org.sonar.server.measure.index.ProjectMeasuresIndexDefinition) Netty4Plugin(org.elasticsearch.transport.Netty4Plugin) IndexDefinitionContext(org.sonar.server.es.IndexDefinition.IndexDefinitionContext) Collectors(java.util.stream.Collectors) String.format(java.lang.String.format) Preconditions.checkState(com.google.common.base.Preconditions.checkState) List(java.util.List) Stream(java.util.stream.Stream) ExternalResource(org.junit.rules.ExternalResource) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) ClusterHealthRequest(org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) CreateIndexResponse(org.elasticsearch.client.indices.CreateIndexResponse) SortOrder(org.elasticsearch.search.sort.SortOrder) RuleIndexDefinition(org.sonar.server.rule.index.RuleIndexDefinition) IssueIndexDefinition(org.sonar.server.issue.index.IssueIndexDefinition) ViewIndexDefinition(org.sonar.server.view.index.ViewIndexDefinition) GetIndexRequest(org.elasticsearch.client.indices.GetIndexRequest) Iterables(com.google.common.collect.Iterables) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) DiskThresholdSettings(org.elasticsearch.cluster.routing.allocation.DiskThresholdSettings) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) InternalSettingsPreparer(org.elasticsearch.node.InternalSettingsPreparer) SearchRequest(org.elasticsearch.action.search.SearchRequest) ArrayList(java.util.ArrayList) UpdateSettingsRequest(org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) Strings(org.elasticsearch.common.Strings) DatagramSocket(java.net.DatagramSocket) HashSet(java.util.HashSet) ImmutableList(com.google.common.collect.ImmutableList) DiscoveryModule(org.elasticsearch.discovery.DiscoveryModule) CommonAnalysisPlugin(org.elasticsearch.analysis.common.CommonAnalysisPlugin) Node(org.elasticsearch.node.Node) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) FIELD_INDEX_TYPE(org.sonar.server.es.IndexType.FIELD_INDEX_TYPE) NoSuchElementException(java.util.NoSuchElementException) Logger(org.sonar.api.utils.log.Logger) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) DeleteIndexRequest(org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) QueryBuilders.matchAllQuery(org.elasticsearch.index.query.QueryBuilders.matchAllQuery) CreateIndexRequest(org.elasticsearch.client.indices.CreateIndexRequest) Files(java.nio.file.Files) Throwables(com.google.common.base.Throwables) UserIndexDefinition(org.sonar.server.user.index.UserIndexDefinition) IOException(java.io.IOException) AcknowledgedResponse(org.elasticsearch.action.support.master.AcknowledgedResponse) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) TermQueryBuilder(org.elasticsearch.index.query.TermQueryBuilder) NodeEnvironment(org.elasticsearch.env.NodeEnvironment) NewIndex(org.sonar.server.es.newindex.NewIndex) PutMappingRequest(org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) ClusterHealthStatus(org.elasticsearch.cluster.health.ClusterHealthStatus) Collections(java.util.Collections) HttpHost(org.apache.http.HttpHost) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Example 84 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project sonarqube by SonarSource.

the class RuleIndexDefinitionTest method support_long_html_description.

@Test
public void support_long_html_description() {
    String longText = StringUtils.repeat("The quick brown fox jumps over the lazy dog ", 700);
    List<AnalyzeResponse.AnalyzeToken> tokens = analyzeIndexedTokens(longText);
    assertThat(tokens).extracting(AnalyzeResponse.AnalyzeToken::getTerm).containsOnly("quick", "brown", "fox", "jump", "over", "lazi", "dog");
    // the following method fails if PUT fails
    tester.putDocuments(TYPE_RULE, new RuleDoc(ImmutableMap.of(FIELD_RULE_UUID, "123", FIELD_RULE_HTML_DESCRIPTION, longText, FIELD_RULE_REPOSITORY, "squid", FIELD_RULE_KEY, "squid:S001")));
    assertThat(tester.countDocuments(TYPE_RULE)).isOne();
    assertThat(tester.client().search(EsClient.prepareSearch(TYPE_RULE).source(new SearchSourceBuilder().query(matchQuery(ENGLISH_HTML_ANALYZER.subField(FIELD_RULE_HTML_DESCRIPTION), "brown fox jumps lazy")))).getHits().getTotalHits()).isEqualTo(new TotalHits(1, TotalHits.Relation.EQUAL_TO));
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) AnalyzeResponse(org.elasticsearch.client.indices.AnalyzeResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) Test(org.junit.Test)

Example 85 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project sonarqube by SonarSource.

the class IssueIndex method getSonarSourceReport.

public List<SecurityStandardCategoryStatistics> getSonarSourceReport(String projectUuid, boolean isViewOrApp, boolean includeCwe) {
    SearchSourceBuilder request = prepareNonClosedVulnerabilitiesAndHotspotSearch(projectUuid, isViewOrApp);
    Arrays.stream(SQCategory.values()).forEach(sonarsourceCategory -> request.aggregation(newSecurityReportSubAggregations(AggregationBuilders.filter(sonarsourceCategory.getKey(), boolQuery().filter(termQuery(FIELD_ISSUE_SQ_SECURITY_CATEGORY, sonarsourceCategory.getKey()))), includeCwe, SecurityStandards.CWES_BY_SQ_CATEGORY.get(sonarsourceCategory))));
    return processSecurityReportSearchResults(request, includeCwe);
}
Also used : SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Aggregations

SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)152 SearchResponse (org.elasticsearch.action.search.SearchResponse)77 SearchRequest (org.elasticsearch.action.search.SearchRequest)52 ElasticsearchAssertions.assertSearchResponse (org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse)37 Matchers.containsString (org.hamcrest.Matchers.containsString)32 IOException (java.io.IOException)31 QueryBuilder (org.elasticsearch.index.query.QueryBuilder)28 BoolQueryBuilder (org.elasticsearch.index.query.BoolQueryBuilder)26 List (java.util.List)25 SearchSourceBuilder (org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder)24 SearchSourceBuilder (org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder)24 Search (io.searchbox.core.Search)23 Map (java.util.Map)23 Set (java.util.Set)19 ArrayList (java.util.ArrayList)18 Collectors (java.util.stream.Collectors)18 Test (org.junit.Test)18 Collections (java.util.Collections)15 SearchHit (org.elasticsearch.search.SearchHit)15 TermsAggregationBuilder (org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder)15