Search in sources :

Example 11 with CountRequest

use of org.elasticsearch.client.core.CountRequest in project geo-platform by geosdi.

the class ElasticSearchRestDAO method count.

/**
 * @param theQueryBuilder
 * @return {@link Long}
 * @throws Exception
 */
@Override
public Long count(@Nonnull(when = NEVER) QueryBuilder theQueryBuilder) throws Exception {
    checkArgument(theQueryBuilder != null, "The Parameter queryBuilder must not be null.");
    CountRequest countRequest = new CountRequest(this.getIndexName());
    countRequest.query(theQueryBuilder);
    logger.trace("#########################QueryBuilder : \n{}\n\n", theQueryBuilder.toString());
    CountResponse countResponse = this.elasticSearchRestHighLevelClient.count(countRequest, DEFAULT);
    if (countResponse.status() != OK) {
        throw new IllegalStateException("Problem in Search : " + countResponse.status());
    }
    return countResponse.getCount();
}
Also used : CountRequest(org.elasticsearch.client.core.CountRequest) CountResponse(org.elasticsearch.client.core.CountResponse)

Example 12 with CountRequest

use of org.elasticsearch.client.core.CountRequest in project geo-platform by geosdi.

the class PageableElasticSearchRestDAO method count.

/**
 * @param page
 * @return {@link Long}
 * @throws Exception
 */
@Override
public <P extends Page> Long count(@Nonnull(when = NEVER) P page) throws Exception {
    checkArgument((page != null), "Page must not be null.");
    CountRequest countRequest = new CountRequest(this.getIndexName());
    SearchSourceBuilder searchSourceBuilder = page.buildPage(new SearchSourceBuilder());
    countRequest.query(searchSourceBuilder.query());
    logger.trace("#########################Builder : {}\n\n", searchSourceBuilder.toString());
    CountResponse countResponse = this.elasticSearchRestHighLevelClient.count(countRequest, DEFAULT);
    if (countResponse.status() != OK) {
        throw new IllegalStateException("Problem in Search : " + countResponse.status());
    }
    return countResponse.getCount();
}
Also used : CountRequest(org.elasticsearch.client.core.CountRequest) CountResponse(org.elasticsearch.client.core.CountResponse) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder)

Example 13 with CountRequest

use of org.elasticsearch.client.core.CountRequest in project openk9 by smclab.

the class IndexWriterEndpoins method _cleanOrphanEntitiesConsumer.

private void _cleanOrphanEntitiesConsumer(Message<Long> message) {
    Long tenantId = message.body();
    RestHighLevelClient client = _restHighLevelClientProvider.get();
    try {
        final Scroll scroll = new Scroll(TimeValue.timeValueSeconds(20));
        String entityIndexName = tenantId + "-entity";
        SearchRequest searchRequest = new SearchRequest(entityIndexName);
        searchRequest.scroll(scroll);
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        String scrollId = searchResponse.getScrollId();
        SearchHit[] searchHits = searchResponse.getHits().getHits();
        Collection<String> entitiesToDelete = new HashSet<>();
        Collection<String> entityNames = new ArrayList<>();
        while (searchHits != null && searchHits.length > 0) {
            for (SearchHit searchHit : searchHits) {
                Map<String, Object> source = searchHit.getSourceAsMap();
                Object id = source.get("id");
                String name = String.valueOf(source.get("name"));
                String nestEntityPath = "entities";
                String nestIdPath = nestEntityPath + ".id";
                CountRequest countRequest = new CountRequest(tenantId + "-*-data");
                countRequest.query(matchQuery(nestIdPath, id));
                CountResponse countResponse = client.count(countRequest, RequestOptions.DEFAULT);
                if (countResponse.getCount() == 0) {
                    entitiesToDelete.add(searchHit.getId());
                    entityNames.add(name);
                }
            }
            SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
            scrollRequest.scroll(scroll);
            searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
            scrollId = searchResponse.getScrollId();
            searchHits = searchResponse.getHits().getHits();
        }
        ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
        clearScrollRequest.addScrollId(scrollId);
        ClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
        boolean succeeded = clearScrollResponse.isSucceeded();
        if (!entitiesToDelete.isEmpty()) {
            BulkRequest bulkRequest = new BulkRequest();
            bulkRequest.add(entitiesToDelete.stream().map(id -> new DeleteRequest(entityIndexName, id)).collect(Collectors.toList())).setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
            BulkResponse bulkResponse = client.bulk(bulkRequest, RequestOptions.DEFAULT);
        }
        message.reply("Entities deleted " + entityNames);
    } catch (Exception e) {
        message.reply(e.getMessage());
    }
}
Also used : LoggerFactory(org.slf4j.LoggerFactory) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) JsonFactory(io.openk9.json.api.JsonFactory) DocumentEntityRequest(io.openk9.index.writer.entity.model.DocumentEntityRequest) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest) ReactorNettyUtils(io.openk9.reactor.netty.util.ReactorNettyUtils) IndexRequest(org.elasticsearch.action.index.IndexRequest) EventBus(io.vertx.core.eventbus.EventBus) Map(java.util.Map) SearchResponse(org.elasticsearch.action.search.SearchResponse) RequestOptions(org.elasticsearch.client.RequestOptions) PluginDriverManagerClient(io.openk9.plugin.driver.manager.client.api.PluginDriverManagerClient) DeleteByQueryRequest(org.elasticsearch.index.reindex.DeleteByQueryRequest) CountRequest(org.elasticsearch.client.core.CountRequest) TimeValue(org.elasticsearch.core.TimeValue) SearchHit(org.elasticsearch.search.SearchHit) ReactorActionListener(io.openk9.search.client.api.ReactorActionListener) Collection(java.util.Collection) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) Message(io.vertx.core.eventbus.Message) DatasourceClient(io.openk9.datasource.client.api.DatasourceClient) Scroll(org.elasticsearch.search.Scroll) Collectors(java.util.stream.Collectors) BundleContext(org.osgi.framework.BundleContext) HttpServerRequest(reactor.netty.http.server.HttpServerRequest) List(java.util.List) QueryBuilders.matchQuery(org.elasticsearch.index.query.QueryBuilders.matchQuery) Builder(lombok.Builder) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) RestHighLevelClientProvider(io.openk9.search.client.api.RestHighLevelClientProvider) HttpServerRoutes(reactor.netty.http.server.HttpServerRoutes) HttpServerResponse(reactor.netty.http.server.HttpServerResponse) MatchQueryBuilder(org.elasticsearch.index.query.MatchQueryBuilder) XContentType(org.elasticsearch.common.xcontent.XContentType) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) HashMap(java.util.HashMap) SearchRequest(org.elasticsearch.action.search.SearchRequest) HttpResponseWriter(io.openk9.http.util.HttpResponseWriter) Function(java.util.function.Function) ClearScrollResponse(org.elasticsearch.action.search.ClearScrollResponse) CountResponse(org.elasticsearch.client.core.CountResponse) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) WriteRequest(org.elasticsearch.action.support.WriteRequest) Component(org.osgi.service.component.annotations.Component) Operator(org.elasticsearch.index.query.Operator) Cancellable(org.elasticsearch.client.Cancellable) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) Activate(org.osgi.service.component.annotations.Activate) BulkByScrollResponse(org.elasticsearch.index.reindex.BulkByScrollResponse) RouterHandler(io.openk9.http.web.RouterHandler) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) Logger(org.slf4j.Logger) Datasource(io.openk9.model.Datasource) Publisher(org.reactivestreams.Publisher) Mono(reactor.core.publisher.Mono) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Data(lombok.Data) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) AllArgsConstructor(lombok.AllArgsConstructor) Reference(org.osgi.service.component.annotations.Reference) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) NoArgsConstructor(lombok.NoArgsConstructor) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) ArrayList(java.util.ArrayList) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) CountRequest(org.elasticsearch.client.core.CountRequest) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) ClearScrollResponse(org.elasticsearch.action.search.ClearScrollResponse) HashSet(java.util.HashSet) Scroll(org.elasticsearch.search.Scroll) CountResponse(org.elasticsearch.client.core.CountResponse) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) SearchResponse(org.elasticsearch.action.search.SearchResponse) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest)

Aggregations

CountRequest (org.elasticsearch.client.core.CountRequest)11 CountResponse (org.elasticsearch.client.core.CountResponse)8 IOException (java.io.IOException)4 SearchRequest (org.elasticsearch.action.search.SearchRequest)3 SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)3 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)2 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)2 GetRequest (org.elasticsearch.action.get.GetRequest)2 IndexRequest (org.elasticsearch.action.index.IndexRequest)2 ClearScrollRequest (org.elasticsearch.action.search.ClearScrollRequest)2 SearchResponse (org.elasticsearch.action.search.SearchResponse)2 SearchScrollRequest (org.elasticsearch.action.search.SearchScrollRequest)2 AbstractEsClientInstrumentationTest (co.elastic.apm.agent.esrestclient.AbstractEsClientInstrumentationTest)1 Span (co.elastic.apm.agent.impl.transaction.Span)1 Stopwatch (com.google.common.base.Stopwatch)1 DatasourceClient (io.openk9.datasource.client.api.DatasourceClient)1 HttpResponseWriter (io.openk9.http.util.HttpResponseWriter)1 RouterHandler (io.openk9.http.web.RouterHandler)1 DocumentEntityRequest (io.openk9.index.writer.entity.model.DocumentEntityRequest)1 JsonFactory (io.openk9.json.api.JsonFactory)1