Search in sources :

Example 66 with SearchHit

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit in project metron by apache.

the class ElasticsearchMetaAlertDao method getAllAlertsForMetaAlert.

@SuppressWarnings("unchecked")
protected List<Map<String, Object>> getAllAlertsForMetaAlert(Document update) throws IOException {
    Document latest = indexDao.getLatest(update.getGuid(), MetaAlertDao.METAALERT_TYPE);
    if (latest == null) {
        return new ArrayList<>();
    }
    List<String> guids = new ArrayList<>();
    List<Map<String, Object>> latestAlerts = (List<Map<String, Object>>) latest.getDocument().get(MetaAlertDao.ALERT_FIELD);
    for (Map<String, Object> alert : latestAlerts) {
        guids.add((String) alert.get(Constants.GUID));
    }
    List<Map<String, Object>> alerts = new ArrayList<>();
    QueryBuilder query = QueryBuilders.idsQuery().addIds(guids.toArray(new String[0]));
    SearchRequestBuilder request = elasticsearchDao.getClient().prepareSearch().setQuery(query);
    org.elasticsearch.action.search.SearchResponse response = request.get();
    for (SearchHit hit : response.getHits().getHits()) {
        alerts.add(hit.sourceAsMap());
    }
    return alerts;
}
Also used : SearchRequestBuilder(org.elasticsearch.action.search.SearchRequestBuilder) SearchHit(org.elasticsearch.search.SearchHit) QueryStringQueryBuilder(org.elasticsearch.index.query.QueryStringQueryBuilder) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) Document(org.apache.metron.indexing.dao.update.Document)

Example 67 with SearchHit

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit in project alien4cloud by alien4cloud.

the class ApplicationController method search.

/**
 * Search for an application.
 *
 * @param searchRequest The element that contains criterias for search operation.
 * @return A rest response that contains a {@link FacetedSearchResult} containing applications.
 */
@ApiOperation(value = "Search for applications", notes = "Returns a search result with that contains applications matching the request. A application is returned only if the connected user has at least one application role in [ APPLICATION_MANAGER | APPLICATION_USER | APPLICATION_DEVOPS | DEPLOYMENT_MANAGER ]")
@RequestMapping(value = "/search", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
@PreAuthorize("isAuthenticated()")
public RestResponse<FacetedSearchResult> search(@RequestBody FilteredSearchRequest searchRequest) {
    FilterBuilder authorizationFilter = AuthorizationUtil.getResourceAuthorizationFilters();
    // We want to sort applications by deployed/undeployed and then application name.
    // Query all application ids and name.
    QueryBuilder queryBuilder = alienDAO.buildSearchQuery(Application.class, searchRequest.getQuery()).setFilters(searchRequest.getFilters(), authorizationFilter).queryBuilder();
    SearchResponse response = alienDAO.getClient().prepareSearch(alienDAO.getIndexForType(Application.class)).setQuery(queryBuilder).setFetchSource(new String[] { "name" }, null).setSize(Integer.MAX_VALUE).get();
    // Get their status (deployed vs undeployed)
    List<DeployedAppHolder> appHolders = Lists.newLinkedList();
    for (SearchHit hit : response.getHits().getHits()) {
        String id = hit.getId();
        String appName = (String) hit.getSource().get("name");
        boolean isDeployed = alienDAO.buildQuery(Deployment.class).setFilters(fromKeyValueCouples("sourceId", id, "endDate", null)).count() > 0;
        appHolders.add(new DeployedAppHolder(id, appName, isDeployed));
    }
    // Sort to have first all deployed apps sorted by name and then all undeployed apps sorted by name.
    Collections.sort(appHolders);
    // Compute the list of app ids to fetch based on the query pagination parameters
    List<String> appIdsToFetch = Lists.newArrayList();
    int to = searchRequest.getFrom() + searchRequest.getSize();
    for (int i = searchRequest.getFrom(); i < appHolders.size() && i < to; i++) {
        appIdsToFetch.add(appHolders.get(i).appId);
    }
    List<Application> applications;
    if (appIdsToFetch.size() == 0) {
        applications = Lists.newArrayList();
    } else {
        applications = alienDAO.findByIds(Application.class, appIdsToFetch.toArray(new String[appIdsToFetch.size()]));
    }
    return RestResponseBuilder.<FacetedSearchResult>builder().data(new FacetedSearchResult<>(searchRequest.getFrom(), to, response.getTookInMillis(), appHolders.size(), new String[] { Application.class.getSimpleName() }, applications.toArray(new Application[applications.size()]), Maps.newHashMap())).build();
}
Also used : SearchHit(org.elasticsearch.search.SearchHit) Deployment(alien4cloud.model.deployment.Deployment) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) SearchResponse(org.elasticsearch.action.search.SearchResponse) FilterBuilder(org.elasticsearch.index.query.FilterBuilder) Application(alien4cloud.model.application.Application) FacetedSearchResult(alien4cloud.dao.model.FacetedSearchResult) ApiOperation(io.swagger.annotations.ApiOperation) PreAuthorize(org.springframework.security.access.prepost.PreAuthorize)

Example 68 with SearchHit

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit in project alien4cloud by alien4cloud.

the class ESGenericSearchDAO method fillMultipleDataResult.

private <T> void fillMultipleDataResult(Class<T> clazz, SearchResponse searchResponse, GetMultipleDataResult<T> finalResponse, int from, boolean managePagination) throws IOException {
    if (managePagination) {
        int to = from + searchResponse.getHits().getHits().length - 1;
        finalResponse.setFrom(from);
        finalResponse.setTo(to);
        finalResponse.setTotalResults(searchResponse.getHits().getTotalHits());
        finalResponse.setQueryDuration(searchResponse.getTookInMillis());
    }
    String[] resultTypes = new String[searchResponse.getHits().getHits().length];
    T[] resultData = (T[]) Array.newInstance(clazz, resultTypes.length);
    for (int i = 0; i < resultTypes.length; i++) {
        SearchHit hit = searchResponse.getHits().getAt(i);
        resultTypes[i] = hit.getType();
        resultData[i] = hitToObject(hit);
    }
    finalResponse.setData(resultData);
    finalResponse.setTypes(resultTypes);
}
Also used : SearchHit(org.elasticsearch.search.SearchHit)

Example 69 with SearchHit

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit in project alien4cloud by alien4cloud.

the class AbstractToscaIndexSearchService method search.

public FacetedSearchResult search(Class<? extends T> clazz, String query, Integer size, Map<String, String[]> filters) {
    TopHitsBuilder topHitAggregation = AggregationBuilders.topHits("highest_version").setSize(1).addSort(new FieldSortBuilder("nestedVersion.majorVersion").order(SortOrder.DESC)).addSort(new FieldSortBuilder("nestedVersion.minorVersion").order(SortOrder.DESC)).addSort(new FieldSortBuilder("nestedVersion.incrementalVersion").order(SortOrder.DESC)).addSort(new FieldSortBuilder("nestedVersion.qualifier").order(SortOrder.DESC).missing("_first"));
    AggregationBuilder aggregation = AggregationBuilders.terms("query_aggregation").field(getAggregationField()).size(size).subAggregation(topHitAggregation);
    FacetedSearchResult<? extends T> searchResult = alienDAO.buildSearchQuery(clazz, query).setFilters(FilterUtil.singleKeyFilter(filters, "workspace", AlienConstants.GLOBAL_WORKSPACE_ID)).prepareSearch().setFetchContext(FetchContext.SUMMARY, topHitAggregation).facetedSearch(new IAggregationQueryManager() {

        @Override
        public AggregationBuilder getQueryAggregation() {
            return aggregation;
        }

        @Override
        @SneakyThrows({ IOException.class })
        public void setData(ObjectMapper objectMapper, Function getClassFromType, FacetedSearchResult result, Aggregation aggregation) {
            List<Object> resultData = Lists.newArrayList();
            List<String> resultTypes = Lists.newArrayList();
            if (aggregation == null) {
                result.setData(getArray(0));
                result.setTypes(new String[0]);
            }
            for (Terms.Bucket bucket : safe(((Terms) aggregation).getBuckets())) {
                TopHits topHits = bucket.getAggregations().get("highest_version");
                for (SearchHit hit : topHits.getHits()) {
                    resultTypes.add(hit.getType());
                    resultData.add(objectMapper.readValue(hit.getSourceAsString(), ((Function<String, Class>) getClassFromType).apply(hit.getType())));
                }
            }
            result.setData(resultData.toArray(getArray(resultData.size())));
            result.setTypes(resultTypes.toArray(new String[resultTypes.size()]));
            result.setFrom(0);
            result.setTo(resultData.size());
            if (size == Integer.MAX_VALUE || resultData.size() < size) {
                result.setTotalResults(resultData.size());
            } else {
                // just to show that there is more results to fetch but iteration is not possible through aggregations.
                result.setTotalResults(resultData.size() + ((Terms) aggregation).getSumOfOtherDocCounts());
            }
        }
    });
    return searchResult;
}
Also used : TopHitsBuilder(org.elasticsearch.search.aggregations.metrics.tophits.TopHitsBuilder) AggregationBuilder(org.elasticsearch.search.aggregations.AggregationBuilder) SearchHit(org.elasticsearch.search.SearchHit) Terms(org.elasticsearch.search.aggregations.bucket.terms.Terms) SneakyThrows(lombok.SneakyThrows) FieldSortBuilder(org.elasticsearch.search.sort.FieldSortBuilder) Aggregation(org.elasticsearch.search.aggregations.Aggregation) Function(java.util.function.Function) TopHits(org.elasticsearch.search.aggregations.metrics.tophits.TopHits) IAggregationQueryManager(alien4cloud.dao.IAggregationQueryManager) List(java.util.List) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) FacetedSearchResult(alien4cloud.dao.model.FacetedSearchResult)

Example 70 with SearchHit

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit in project metron by apache.

the class ElasticsearchRetrieveLatestDao method searchByGuids.

/**
 * Return the search hit based on the UUID and sensor type.
 * A callback can be specified to transform the hit into a type T.
 * If more than one hit happens, the first one will be returned.
 */
<T> List<T> searchByGuids(Collection<String> guids, Collection<String> sensorTypes, Function<SearchHit, Optional<T>> callback) throws IOException {
    if (guids == null || guids.isEmpty()) {
        return Collections.emptyList();
    }
    // should match any of the guids
    // the 'guid' field must be of type 'keyword' or this term query will not match
    BoolQueryBuilder guidQuery = boolQuery().must(termsQuery(Constants.GUID, guids));
    // should match any of the sensor types
    BoolQueryBuilder sensorQuery = boolQuery();
    sensorTypes.forEach(sensorType -> sensorQuery.should(typeQuery(sensorType + "_doc")));
    // must have a match for both guid and sensor
    BoolQueryBuilder query = boolQuery().must(guidQuery).must(sensorQuery);
    // submit the search
    SearchResponse response;
    try {
        SearchSourceBuilder source = new SearchSourceBuilder().query(query).size(guids.size());
        SearchRequest request = new SearchRequest().source(source);
        response = submitter.submitSearch(request);
    } catch (InvalidSearchException e) {
        throw new IOException(e);
    }
    // transform the search hits to results using the callback
    List<T> results = new ArrayList<>();
    for (SearchHit hit : response.getHits()) {
        Optional<T> result = callback.apply(hit);
        result.ifPresent(r -> results.add(r));
    }
    return results;
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) InvalidSearchException(org.apache.metron.indexing.dao.search.InvalidSearchException) SearchHit(org.elasticsearch.search.SearchHit) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) ArrayList(java.util.ArrayList) IOException(java.io.IOException) SearchResponse(org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Aggregations

SearchHit (org.elasticsearch.search.SearchHit)318 SearchResponse (org.elasticsearch.action.search.SearchResponse)225 SearchHits (org.elasticsearch.search.SearchHits)87 ArrayList (java.util.ArrayList)83 SearchRequestBuilder (org.elasticsearch.action.search.SearchRequestBuilder)64 HashMap (java.util.HashMap)37 Map (java.util.Map)37 BoolQueryBuilder (org.elasticsearch.index.query.BoolQueryBuilder)37 IOException (java.io.IOException)35 Test (org.junit.Test)32 TimeValue (org.elasticsearch.common.unit.TimeValue)29 ElasticsearchAssertions.assertSearchResponse (org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse)25 QueryBuilder (org.elasticsearch.index.query.QueryBuilder)23 SearchRequest (org.elasticsearch.action.search.SearchRequest)22 List (java.util.List)17 IndexRequestBuilder (org.elasticsearch.action.index.IndexRequestBuilder)17 SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)17 SearchHitField (org.elasticsearch.search.SearchHitField)16 HashSet (java.util.HashSet)15 Text (org.elasticsearch.common.text.Text)14