Search in sources :

Example 6 with Query

use of co.elastic.clients.elasticsearch._types.query_dsl.Query in project core-ng-project by neowu.

the class ElasticSearchTypeImpl method forEach.

@Override
public void forEach(ForEach<T> forEach) {
    var watch = new StopWatch();
    long start = System.nanoTime();
    long esClientTook = 0;
    long esServerTook = 0;
    validate(forEach);
    Time keepAlive = Time.of(t -> t.time(forEach.scrollTimeout.toNanos() + "nanos"));
    String index = forEach.index == null ? this.index : forEach.index;
    int totalHits = 0;
    try {
        var response = elasticSearch.client.search(builder -> builder.index(index).scroll(keepAlive).query(forEach.query).sort(s -> s.field(f -> f.field("_doc"))).size(forEach.limit), documentClass);
        var holder = new ScrollIdHolder();
        holder.scrollId = response.scrollId();
        while (true) {
            esServerTook += response.took() * 1_000_000;
            var hits = response.hits().hits();
            esClientTook += System.nanoTime() - start;
            if (hits.isEmpty())
                break;
            totalHits += hits.size();
            for (var hit : hits) {
                forEach.consumer.accept(hit.source());
            }
            start = System.nanoTime();
            response = elasticSearch.client.scroll(builder -> builder.scrollId(holder.scrollId).scroll(keepAlive), documentClass);
            holder.scrollId = response.scrollId();
        }
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    } catch (ElasticsearchException e) {
        throw elasticSearch.searchException(e);
    } finally {
        long elapsed = watch.elapsed();
        ActionLogContext.track("elasticsearch", elapsed, totalHits, 0);
        logger.debug("forEach, totalHits={}, esServerTook={}, esClientTook={}, elapsed={}", totalHits, esServerTook, esClientTook, elapsed);
    }
}
Also used : AnalyzeResponse(co.elastic.clients.elasticsearch.indices.AnalyzeResponse) BulkOperation(co.elastic.clients.elasticsearch.core.bulk.BulkOperation) CompletionSuggestOption(co.elastic.clients.elasticsearch.core.search.CompletionSuggestOption) GetRequest(core.framework.search.GetRequest) CodeBuilder(core.framework.internal.asm.CodeBuilder) ActionLogContext(core.framework.log.ActionLogContext) LoggerFactory(org.slf4j.LoggerFactory) UpdateRequest(core.framework.search.UpdateRequest) Index(core.framework.search.Index) BulkIndexRequest(core.framework.search.BulkIndexRequest) IndexRequest(core.framework.search.IndexRequest) StopWatch(core.framework.util.StopWatch) ArrayList(java.util.ArrayList) DeleteResponse(co.elastic.clients.elasticsearch.core.DeleteResponse) BulkDeleteRequest(core.framework.search.BulkDeleteRequest) Markers.errorCode(core.framework.log.Markers.errorCode) CompleteRequest(core.framework.search.CompleteRequest) Duration(java.time.Duration) Map(java.util.Map) SearchException(core.framework.search.SearchException) SearchRequest(core.framework.search.SearchRequest) SearchResponse(core.framework.search.SearchResponse) ErrorCause(co.elastic.clients.elasticsearch._types.ErrorCause) Time(co.elastic.clients.elasticsearch._types.Time) ElasticsearchException(co.elastic.clients.elasticsearch._types.ElasticsearchException) Validator(core.framework.internal.validate.Validator) Logger(org.slf4j.Logger) JsonData(co.elastic.clients.json.JsonData) GetResponse(co.elastic.clients.elasticsearch.core.GetResponse) Collection(java.util.Collection) Suggester(co.elastic.clients.elasticsearch.core.search.Suggester) IOException(java.io.IOException) AnalyzeToken(co.elastic.clients.elasticsearch.indices.analyze.AnalyzeToken) ForEach(core.framework.search.ForEach) Collectors(java.util.stream.Collectors) Result(co.elastic.clients.elasticsearch._types.Result) UncheckedIOException(java.io.UncheckedIOException) DeleteRequest(core.framework.search.DeleteRequest) Hit(co.elastic.clients.elasticsearch.core.search.Hit) ElasticSearchType(core.framework.search.ElasticSearchType) List(java.util.List) AnalyzeRequest(core.framework.search.AnalyzeRequest) BulkResponse(co.elastic.clients.elasticsearch.core.BulkResponse) BulkResponseItem(co.elastic.clients.elasticsearch.core.bulk.BulkResponseItem) Strings(core.framework.util.Strings) Optional(java.util.Optional) Time(co.elastic.clients.elasticsearch._types.Time) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) ElasticsearchException(co.elastic.clients.elasticsearch._types.ElasticsearchException) StopWatch(core.framework.util.StopWatch)

Example 7 with Query

use of co.elastic.clients.elasticsearch._types.query_dsl.Query in project core-ng-project by neowu.

the class ElasticSearchTypeImpl method search.

@Override
public SearchResponse<T> search(SearchRequest request) {
    var watch = new StopWatch();
    validate(request);
    long esTook = 0;
    String index = request.index == null ? this.index : request.index;
    int hits = 0;
    try {
        var searchRequest = co.elastic.clients.elasticsearch.core.SearchRequest.of(builder -> {
            builder.index(index).query(request.query).aggregations(request.aggregations).sort(request.sorts).searchType(request.type).from(request.skip).size(request.limit);
            if (request.trackTotalHitsUpTo != null)
                builder.trackTotalHits(t -> t.count(request.trackTotalHitsUpTo));
            return builder;
        });
        var response = elasticSearch.client.search(searchRequest, documentClass);
        esTook = response.took() * 1_000_000;
        hits = response.hits().hits().size();
        long total = response.hits().total().value();
        List<T> items = response.hits().hits().stream().map(Hit::source).toList();
        return new SearchResponse<>(items, total, response.aggregations());
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    } catch (ElasticsearchException e) {
        throw elasticSearch.searchException(e);
    } finally {
        long elapsed = watch.elapsed();
        ActionLogContext.track("elasticsearch", elapsed, hits, 0);
        logger.debug("search, hits={}, esTook={}, elapsed={}", hits, esTook, elapsed);
        checkSlowOperation(elapsed);
    }
}
Also used : AnalyzeResponse(co.elastic.clients.elasticsearch.indices.AnalyzeResponse) BulkOperation(co.elastic.clients.elasticsearch.core.bulk.BulkOperation) CompletionSuggestOption(co.elastic.clients.elasticsearch.core.search.CompletionSuggestOption) GetRequest(core.framework.search.GetRequest) CodeBuilder(core.framework.internal.asm.CodeBuilder) ActionLogContext(core.framework.log.ActionLogContext) LoggerFactory(org.slf4j.LoggerFactory) UpdateRequest(core.framework.search.UpdateRequest) Index(core.framework.search.Index) BulkIndexRequest(core.framework.search.BulkIndexRequest) IndexRequest(core.framework.search.IndexRequest) StopWatch(core.framework.util.StopWatch) ArrayList(java.util.ArrayList) DeleteResponse(co.elastic.clients.elasticsearch.core.DeleteResponse) BulkDeleteRequest(core.framework.search.BulkDeleteRequest) Markers.errorCode(core.framework.log.Markers.errorCode) CompleteRequest(core.framework.search.CompleteRequest) Duration(java.time.Duration) Map(java.util.Map) SearchException(core.framework.search.SearchException) SearchRequest(core.framework.search.SearchRequest) SearchResponse(core.framework.search.SearchResponse) ErrorCause(co.elastic.clients.elasticsearch._types.ErrorCause) Time(co.elastic.clients.elasticsearch._types.Time) ElasticsearchException(co.elastic.clients.elasticsearch._types.ElasticsearchException) Validator(core.framework.internal.validate.Validator) Logger(org.slf4j.Logger) JsonData(co.elastic.clients.json.JsonData) GetResponse(co.elastic.clients.elasticsearch.core.GetResponse) Collection(java.util.Collection) Suggester(co.elastic.clients.elasticsearch.core.search.Suggester) IOException(java.io.IOException) AnalyzeToken(co.elastic.clients.elasticsearch.indices.analyze.AnalyzeToken) ForEach(core.framework.search.ForEach) Collectors(java.util.stream.Collectors) Result(co.elastic.clients.elasticsearch._types.Result) UncheckedIOException(java.io.UncheckedIOException) DeleteRequest(core.framework.search.DeleteRequest) Hit(co.elastic.clients.elasticsearch.core.search.Hit) ElasticSearchType(core.framework.search.ElasticSearchType) List(java.util.List) AnalyzeRequest(core.framework.search.AnalyzeRequest) BulkResponse(co.elastic.clients.elasticsearch.core.BulkResponse) BulkResponseItem(co.elastic.clients.elasticsearch.core.bulk.BulkResponseItem) Strings(core.framework.util.Strings) Optional(java.util.Optional) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) ElasticsearchException(co.elastic.clients.elasticsearch._types.ElasticsearchException) StopWatch(core.framework.util.StopWatch) SearchResponse(core.framework.search.SearchResponse)

Example 8 with Query

use of co.elastic.clients.elasticsearch._types.query_dsl.Query in project syncope by apache.

the class ElasticsearchAnySearchDAO method getQuery.

protected Query getQuery(final AssignableCond cond) {
    Realm realm = check(cond);
    List<Query> queries = new ArrayList<>();
    if (cond.isFromGroup()) {
        realmDAO.findDescendants(realm).forEach(current -> queries.add(new Query.Builder().term(QueryBuilders.term().field("realm").value(FieldValue.of(current.getFullPath())).build()).build()));
    } else {
        for (Realm current = realm; current.getParent() != null; current = current.getParent()) {
            queries.add(new Query.Builder().term(QueryBuilders.term().field("realm").value(FieldValue.of(current.getFullPath())).build()).build());
        }
        queries.add(new Query.Builder().term(QueryBuilders.term().field("realm").value(FieldValue.of(realmDAO.getRoot().getFullPath())).build()).build());
    }
    return new Query.Builder().disMax(QueryBuilders.disMax().queries(queries).build()).build();
}
Also used : Query(co.elastic.clients.elasticsearch._types.query_dsl.Query) ArrayList(java.util.ArrayList) Realm(org.apache.syncope.core.persistence.api.entity.Realm) DynRealm(org.apache.syncope.core.persistence.api.entity.DynRealm)

Example 9 with Query

use of co.elastic.clients.elasticsearch._types.query_dsl.Query in project syncope by apache.

the class ElasticsearchAnySearchDAOTest method getAdminRealmsFilter_realm.

@Test
public void getAdminRealmsFilter_realm() throws IOException {
    // 1. mock
    Realm root = mock(Realm.class);
    when(root.getFullPath()).thenReturn(SyncopeConstants.ROOT_REALM);
    when(realmDAO.findByFullPath(SyncopeConstants.ROOT_REALM)).thenReturn(root);
    when(realmDAO.findDescendants(root)).thenReturn(List.of(root));
    // 2. test
    Set<String> adminRealms = Set.of(SyncopeConstants.ROOT_REALM);
    Triple<Optional<Query>, Set<String>, Set<String>> filter = searchDAO.getAdminRealmsFilter(AnyTypeKind.USER, adminRealms);
    assertThat(new Query.Builder().disMax(QueryBuilders.disMax().queries(new Query.Builder().term(QueryBuilders.term().field("realm").value(FieldValue.of(SyncopeConstants.ROOT_REALM)).build()).build()).build()).build()).usingRecursiveComparison().isEqualTo(filter.getLeft().get());
    assertEquals(Set.of(), filter.getMiddle());
    assertEquals(Set.of(), filter.getRight());
}
Also used : Set(java.util.Set) Optional(java.util.Optional) Query(co.elastic.clients.elasticsearch._types.query_dsl.Query) Realm(org.apache.syncope.core.persistence.api.entity.Realm) DynRealm(org.apache.syncope.core.persistence.api.entity.DynRealm) Test(org.junit.jupiter.api.Test)

Example 10 with Query

use of co.elastic.clients.elasticsearch._types.query_dsl.Query in project opensearch-java by opensearch-project.

the class ApiConventionsTest method builderIntervals.

@Test(expected = TransportException.class)
public void builderIntervals() throws Exception {
    OpenSearchClient client = new OpenSearchClient(transport);
    // tag::builder-intervals
    SearchResponse<SomeApplicationData> results = client.search(_0 -> _0.query(_1 -> _1.intervals(_2 -> _2.field("my_text").allOf(_3 -> _3.ordered(true).intervals(_4 -> _4.match(_5 -> _5.query("my favorite food").maxGaps(0).ordered(true))).intervals(_4 -> _4.anyOf(_5 -> _5.intervals(_6 -> _6.match(_7 -> _7.query("hot water"))).intervals(_6 -> _6.match(_7 -> _7.query("cold porridge")))))))), // <1>
    SomeApplicationData.class);
// end::builder-intervals
}
Also used : OpenSearchAsyncClient(org.opensearch.client.opensearch.OpenSearchAsyncClient) Arrays(java.util.Arrays) Aggregation(org.opensearch.client.opensearch._types.aggregations.Aggregation) Alias(org.opensearch.client.opensearch.indices.Alias) Query(org.opensearch.client.opensearch._types.query_dsl.Query) CreateIndexRequest(org.opensearch.client.opensearch.indices.CreateIndexRequest) LogManager(java.util.logging.LogManager) OpenSearchClient(org.opensearch.client.opensearch.OpenSearchClient) Test(org.junit.Test) HashMap(java.util.HashMap) CreateIndexResponse(org.opensearch.client.opensearch.indices.CreateIndexResponse) Logger(java.util.logging.Logger) SearchResponse(org.opensearch.client.opensearch.core.SearchResponse) OpenSearchTransport(org.opensearch.client.transport.OpenSearchTransport) TransportException(org.opensearch.client.transport.TransportException) List(java.util.List) SearchRequest(org.opensearch.client.opensearch.core.SearchRequest) NodeStatistics(org.opensearch.client.opensearch._types.NodeStatistics) SortOrder(org.opensearch.client.opensearch._types.SortOrder) ApiTypeHelper(org.opensearch.client.util.ApiTypeHelper) Map(java.util.Map) Assert(org.junit.Assert) OpenSearchClient(org.opensearch.client.opensearch.OpenSearchClient) Test(org.junit.Test)

Aggregations

Query (co.elastic.clients.elasticsearch._types.query_dsl.Query)5 Test (org.junit.Test)5 Query (org.opensearch.client.opensearch._types.query_dsl.Query)5 List (java.util.List)4 Map (java.util.Map)4 Optional (java.util.Optional)4 ArrayList (java.util.ArrayList)3 ElasticsearchException (co.elastic.clients.elasticsearch._types.ElasticsearchException)2 ErrorCause (co.elastic.clients.elasticsearch._types.ErrorCause)2 Result (co.elastic.clients.elasticsearch._types.Result)2 Time (co.elastic.clients.elasticsearch._types.Time)2 BulkResponse (co.elastic.clients.elasticsearch.core.BulkResponse)2 DeleteResponse (co.elastic.clients.elasticsearch.core.DeleteResponse)2 GetResponse (co.elastic.clients.elasticsearch.core.GetResponse)2 BulkOperation (co.elastic.clients.elasticsearch.core.bulk.BulkOperation)2 BulkResponseItem (co.elastic.clients.elasticsearch.core.bulk.BulkResponseItem)2 CompletionSuggestOption (co.elastic.clients.elasticsearch.core.search.CompletionSuggestOption)2 Hit (co.elastic.clients.elasticsearch.core.search.Hit)2 Suggester (co.elastic.clients.elasticsearch.core.search.Suggester)2 AnalyzeResponse (co.elastic.clients.elasticsearch.indices.AnalyzeResponse)2