Search in sources :

Example 81 with BoolQueryBuilder

use of org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder in project molgenis by molgenis.

the class QueryGeneratorTest method generateMultipleQueryRule.

@Test
public void generateMultipleQueryRule() {
    // query: a or (b and c)
    Boolean booleanValue = Boolean.TRUE;
    String stringValue = "str";
    Integer intValue = 1;
    Query<Entity> q = new QueryImpl<>().eq(boolAttrName, booleanValue).or().nest().eq(stringAttrName, stringValue).and().eq(intAttrName, intValue).unnest();
    QueryBuilder query = queryGenerator.createQueryBuilder(q, entityType);
    QueryBuilder booleanQuery = constantScoreQuery(termQuery(boolAttrName, booleanValue));
    QueryBuilder stringQuery = constantScoreQuery(termQuery(stringAttrName + '.' + FIELD_NOT_ANALYZED, stringValue));
    QueryBuilder intQuery = constantScoreQuery(termQuery(intAttrName, intValue));
    BoolQueryBuilder stringIntQuery = boolQuery().must(stringQuery).must(intQuery);
    QueryBuilder expectedQuery = boolQuery().should(booleanQuery).should(stringIntQuery).minimumShouldMatch(1);
    assertQueryBuilderEquals(query, expectedQuery);
}
Also used : DynamicEntity(org.molgenis.data.support.DynamicEntity) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Test(org.testng.annotations.Test)

Example 82 with BoolQueryBuilder

use of org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder in project Anserini by castorini.

the class SearchElastic method searchTweets.

public <K> ScoredDocuments searchTweets(String queryString, long t) {
    SearchHits results = null;
    String specials = "+-=&|><!(){}[]^\"~*?:\\/";
    for (int i = 0; i < specials.length(); i++) {
        char c = specials.charAt(i);
        queryString = queryString.replace(String.valueOf(c), " ");
    }
    // Do not consider the tweets with tweet ids that are beyond the queryTweetTime
    // <querytweettime> tag contains the timestamp of the query in terms of the
    // chronologically nearest tweet id within the corpus
    RangeQueryBuilder queryTweetTime = QueryBuilders.rangeQuery(TweetGenerator.TweetField.ID_LONG.name).from(0L).to(t);
    QueryStringQueryBuilder queryTerms = QueryBuilders.queryStringQuery(queryString).defaultField("contents").analyzer("english");
    BoolQueryBuilder query = QueryBuilders.boolQuery().filter(queryTweetTime).should(queryTerms);
    SearchRequest searchRequest = new SearchRequest(args.esIndex);
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    sourceBuilder.query(query);
    sourceBuilder.size(args.hits);
    sourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC));
    sourceBuilder.sort(new FieldSortBuilder(TweetGenerator.TweetField.ID_LONG.name).order(SortOrder.DESC));
    searchRequest.source(sourceBuilder);
    try {
        SearchResponse searchResponse = client.search(searchRequest, COMMON_OPTIONS);
        results = searchResponse.getHits();
    } catch (Exception e) {
        LOG.error("Exception during ES query: ", e);
    }
    ScoreTiesAdjusterReranker reranker = new ScoreTiesAdjusterReranker();
    return reranker.rerank(ScoredDocuments.fromESDocs(results), null);
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) FieldSortBuilder(org.elasticsearch.search.sort.FieldSortBuilder) RangeQueryBuilder(org.elasticsearch.index.query.RangeQueryBuilder) IOException(java.io.IOException) CmdLineException(org.kohsuke.args4j.CmdLineException) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.elasticsearch.action.search.SearchResponse) ScoreSortBuilder(org.elasticsearch.search.sort.ScoreSortBuilder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) ScoreTiesAdjusterReranker(io.anserini.rerank.lib.ScoreTiesAdjusterReranker) SearchHits(org.elasticsearch.search.SearchHits) QueryStringQueryBuilder(org.elasticsearch.index.query.QueryStringQueryBuilder)

Example 83 with BoolQueryBuilder

use of org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder in project engine by craftercms.

the class ContentTypeBasedDataFetcher method addFieldFilterFromMapEntry.

protected void addFieldFilterFromMapEntry(String path, Map.Entry<String, Object> filter, BoolQueryBuilder query, DataFetchingEnvironment env) {
    if (filter.getValue() instanceof List) {
        List<Map<String, Object>> actualFilters = (List<Map<String, Object>>) filter.getValue();
        switch(filter.getKey()) {
            case ARG_NAME_NOT:
                BoolQueryBuilder notQuery = boolQuery();
                actualFilters.forEach(notFilter -> notFilter.entrySet().forEach(notField -> addFieldFilterFromMapEntry(path, notField, notQuery, env)));
                notQuery.filter().forEach(query::mustNot);
                break;
            case ARG_NAME_AND:
                actualFilters.forEach(andFilter -> andFilter.entrySet().forEach(andField -> addFieldFilterFromMapEntry(path, andField, query, env)));
                break;
            case ARG_NAME_OR:
                BoolQueryBuilder tempQuery = boolQuery();
                BoolQueryBuilder orQuery = boolQuery();
                actualFilters.forEach(orFilter -> orFilter.entrySet().forEach(orField -> addFieldFilterFromMapEntry(path, orField, tempQuery, env)));
                tempQuery.filter().forEach(orQuery::should);
                query.filter(boolQuery().must(orQuery));
                break;
            default:
        }
    } else {
        query.filter(getFilterQueryFromMapEntry(path, filter));
    }
}
Also used : ObjectValue(graphql.language.ObjectValue) DataFetchingEnvironment(graphql.schema.DataFetchingEnvironment) FloatValue(graphql.language.FloatValue) Value(graphql.language.Value) LoggerFactory(org.slf4j.LoggerFactory) FragmentSpread(graphql.language.FragmentSpread) HashMap(java.util.HashMap) SearchRequest(org.elasticsearch.action.search.SearchRequest) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) StringUtils(org.apache.commons.lang3.StringUtils) ElasticsearchWrapper(org.craftercms.search.elasticsearch.ElasticsearchWrapper) LinkedHashMap(java.util.LinkedHashMap) Selection(graphql.language.Selection) VariableReference(graphql.language.VariableReference) Map(java.util.Map) SearchResponse(org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) DataFetcher(graphql.schema.DataFetcher) LinkedList(java.util.LinkedList) SearchHit(org.elasticsearch.search.SearchHit) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) Logger(org.slf4j.Logger) MapUtils(org.apache.commons.collections.MapUtils) ObjectField(graphql.language.ObjectField) StopWatch(org.springframework.util.StopWatch) Field(graphql.language.Field) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) Argument(graphql.language.Argument) List(java.util.List) StringValue(graphql.language.StringValue) ArrayValue(graphql.language.ArrayValue) IntValue(graphql.language.IntValue) SortOrder(org.elasticsearch.search.sort.SortOrder) Optional(java.util.Optional) FragmentDefinition(graphql.language.FragmentDefinition) Required(org.springframework.beans.factory.annotation.Required) InlineFragment(graphql.language.InlineFragment) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Collections(java.util.Collections) SchemaUtils(org.craftercms.engine.graphql.SchemaUtils) BooleanValue(graphql.language.BooleanValue) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) LinkedList(java.util.LinkedList) List(java.util.List) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map)

Example 84 with BoolQueryBuilder

use of org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder in project engine by craftercms.

the class ContentTypeBasedDataFetcher method doGet.

/**
 * {@inheritDoc}
 */
@Override
public Object doGet(final DataFetchingEnvironment env) {
    Field field = env.getMergedField().getSingleField();
    String fieldName = field.getName();
    // Get arguments for pagination & sorting
    int offset = Optional.ofNullable(env.<Integer>getArgument(ARG_NAME_OFFSET)).orElse(0);
    int limit = Optional.ofNullable(env.<Integer>getArgument(ARG_NAME_LIMIT)).orElse(defaultLimit);
    String sortBy = Optional.ofNullable(env.<String>getArgument(ARG_NAME_SORT_BY)).orElse(defaultSortField);
    String sortOrder = Optional.ofNullable(env.<String>getArgument(ARG_NAME_SORT_ORDER)).orElse(defaultSortOrder);
    List<String> queryFieldIncludes = new LinkedList<>();
    // Add content-type to includes, we might need it for a GraphQL TypeResolver
    queryFieldIncludes.add(QUERY_FIELD_NAME_CONTENT_TYPE);
    List<Map<String, Object>> items = new LinkedList<>();
    Map<String, Object> result = new HashMap<>(2);
    result.put(FIELD_NAME_ITEMS, items);
    // Setup the ES query
    SearchSourceBuilder source = new SearchSourceBuilder();
    BoolQueryBuilder query = boolQuery();
    source.query(query).from(offset).size(limit).sort(sortBy, SortOrder.fromString(sortOrder));
    StopWatch watch = new StopWatch(field.getName() + " - " + field.getAlias());
    watch.start("build filters");
    // Filter by the content-type
    switch(fieldName) {
        case FIELD_NAME_CONTENT_ITEMS:
            query.filter(existsQuery(QUERY_FIELD_NAME_CONTENT_TYPE));
            break;
        case FIELD_NAME_PAGES:
            query.filter(regexpQuery(QUERY_FIELD_NAME_CONTENT_TYPE, CONTENT_TYPE_REGEX_PAGE));
            break;
        case FIELD_NAME_COMPONENTS:
            query.filter(regexpQuery(QUERY_FIELD_NAME_CONTENT_TYPE, CONTENT_TYPE_REGEX_COMPONENT));
            break;
        default:
            // Get the content-type name from the field name
            query.filter(termQuery(QUERY_FIELD_NAME_CONTENT_TYPE, getOriginalName(fieldName)));
            break;
    }
    // Check the selected fields to build the ES query
    Optional<Field> itemsField = field.getSelectionSet().getSelections().stream().map(f -> (Field) f).filter(f -> f.getName().equals(FIELD_NAME_ITEMS)).findFirst();
    if (itemsField.isPresent()) {
        List<Selection> selections = itemsField.get().getSelectionSet().getSelections();
        selections.forEach(selection -> processSelection(StringUtils.EMPTY, selection, query, queryFieldIncludes, env));
    }
    // Only fetch the selected fields for better performance
    source.fetchSource(queryFieldIncludes.toArray(new String[0]), new String[0]);
    watch.stop();
    logger.debug("Executing query: {}", source);
    watch.start("searching items");
    SearchResponse response = elasticsearch.search(new SearchRequest().source(source));
    watch.stop();
    watch.start("processing items");
    result.put(FIELD_NAME_TOTAL, response.getHits().totalHits);
    if (response.getHits().totalHits > 0) {
        for (SearchHit hit : response.getHits().getHits()) {
            items.add(fixItems(hit.getSourceAsMap()));
        }
    }
    watch.stop();
    if (logger.isTraceEnabled()) {
        logger.trace(watch.prettyPrint());
    }
    return result;
}
Also used : ObjectValue(graphql.language.ObjectValue) DataFetchingEnvironment(graphql.schema.DataFetchingEnvironment) FloatValue(graphql.language.FloatValue) Value(graphql.language.Value) LoggerFactory(org.slf4j.LoggerFactory) FragmentSpread(graphql.language.FragmentSpread) HashMap(java.util.HashMap) SearchRequest(org.elasticsearch.action.search.SearchRequest) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) StringUtils(org.apache.commons.lang3.StringUtils) ElasticsearchWrapper(org.craftercms.search.elasticsearch.ElasticsearchWrapper) LinkedHashMap(java.util.LinkedHashMap) Selection(graphql.language.Selection) VariableReference(graphql.language.VariableReference) Map(java.util.Map) SearchResponse(org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) DataFetcher(graphql.schema.DataFetcher) LinkedList(java.util.LinkedList) SearchHit(org.elasticsearch.search.SearchHit) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) Logger(org.slf4j.Logger) MapUtils(org.apache.commons.collections.MapUtils) ObjectField(graphql.language.ObjectField) StopWatch(org.springframework.util.StopWatch) Field(graphql.language.Field) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) Argument(graphql.language.Argument) List(java.util.List) StringValue(graphql.language.StringValue) ArrayValue(graphql.language.ArrayValue) IntValue(graphql.language.IntValue) SortOrder(org.elasticsearch.search.sort.SortOrder) Optional(java.util.Optional) FragmentDefinition(graphql.language.FragmentDefinition) Required(org.springframework.beans.factory.annotation.Required) InlineFragment(graphql.language.InlineFragment) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Collections(java.util.Collections) SchemaUtils(org.craftercms.engine.graphql.SchemaUtils) BooleanValue(graphql.language.BooleanValue) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Selection(graphql.language.Selection) LinkedList(java.util.LinkedList) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) StopWatch(org.springframework.util.StopWatch) SearchResponse(org.elasticsearch.action.search.SearchResponse) ObjectField(graphql.language.ObjectField) Field(graphql.language.Field) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map)

Example 85 with BoolQueryBuilder

use of org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder in project engine by craftercms.

the class SiteAwareElasticsearchService method updateFilters.

@Override
protected void updateFilters(final SearchRequest request) {
    super.updateFilters(request);
    BoolQueryBuilder mainQuery = (BoolQueryBuilder) request.source().query();
    Authentication auth = null;
    SecurityContext context = SecurityContextHolder.getContext();
    if (context != null) {
        auth = context.getAuthentication();
    }
    // Include all public items
    BoolQueryBuilder securityQuery = boolQuery().should(boolQuery().mustNot(existsQuery(roleFieldName))).should(matchQuery(roleFieldName, "anonymous"));
    if (auth != null && !(auth instanceof AnonymousAuthenticationToken) && isNotEmpty(auth.getAuthorities())) {
        logger.debug("Filtering search results for roles: {}", auth.getAuthorities());
        securityQuery.should(matchQuery(roleFieldName, auth.getAuthorities().stream().map(GrantedAuthority::getAuthority).map(role -> role + " " + (startsWith(role, ROLE_PREFIX) ? removeStart(role, ROLE_PREFIX) : appendIfMissing(role, ROLE_PREFIX))).collect(joining(" "))));
    } else {
        logger.debug("Filtering search to show only public items");
    }
    mainQuery.filter(boolQuery().must(securityQuery));
}
Also used : LoggerFactory(org.slf4j.LoggerFactory) ArrayUtils(org.apache.commons.lang3.ArrayUtils) SearchRequest(org.elasticsearch.action.search.SearchRequest) AbstractElasticsearchWrapper(org.craftercms.search.elasticsearch.impl.AbstractElasticsearchWrapper) ArrayList(java.util.ArrayList) IndicesOptions(org.elasticsearch.action.support.IndicesOptions) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) SecurityContextHolder(org.springframework.security.core.context.SecurityContextHolder) Logger(org.slf4j.Logger) SiteContext(org.craftercms.engine.service.context.SiteContext) QueryBuilders.boolQuery(org.elasticsearch.index.query.QueryBuilders.boolQuery) StringUtils.startsWith(org.apache.commons.lang3.StringUtils.startsWith) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) StringUtils.appendIfMissing(org.apache.commons.lang3.StringUtils.appendIfMissing) Collectors.joining(java.util.stream.Collectors.joining) GrantedAuthority(org.springframework.security.core.GrantedAuthority) List(java.util.List) CollectionUtils.isNotEmpty(org.apache.commons.collections4.CollectionUtils.isNotEmpty) QueryBuilders.matchQuery(org.elasticsearch.index.query.QueryBuilders.matchQuery) SecurityContext(org.springframework.security.core.context.SecurityContext) AnonymousAuthenticationToken(org.springframework.security.authentication.AnonymousAuthenticationToken) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Authentication(org.springframework.security.core.Authentication) QueryBuilders.existsQuery(org.elasticsearch.index.query.QueryBuilders.existsQuery) StringUtils.removeStart(org.apache.commons.lang3.StringUtils.removeStart) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Authentication(org.springframework.security.core.Authentication) GrantedAuthority(org.springframework.security.core.GrantedAuthority) SecurityContext(org.springframework.security.core.context.SecurityContext) AnonymousAuthenticationToken(org.springframework.security.authentication.AnonymousAuthenticationToken)

Aggregations

BoolQueryBuilder (org.elasticsearch.index.query.BoolQueryBuilder)201 SearchResponse (org.elasticsearch.action.search.SearchResponse)73 QueryBuilder (org.elasticsearch.index.query.QueryBuilder)58 SearchHit (org.elasticsearch.search.SearchHit)41 SearchRequestBuilder (org.elasticsearch.action.search.SearchRequestBuilder)38 SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)30 Map (java.util.Map)29 ArrayList (java.util.ArrayList)27 HashMap (java.util.HashMap)25 List (java.util.List)23 SearchRequest (org.elasticsearch.action.search.SearchRequest)21 Test (org.junit.Test)18 RangeQueryBuilder (org.elasticsearch.index.query.RangeQueryBuilder)16 TermsAggregationBuilder (org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder)16 QueryBuilders (org.elasticsearch.index.query.QueryBuilders)14 FilterAggregationBuilder (org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder)13 Logger (org.slf4j.Logger)12 LoggerFactory (org.slf4j.LoggerFactory)12 LinkedList (java.util.LinkedList)11 FieldSortBuilder (org.elasticsearch.search.sort.FieldSortBuilder)11