Search in sources :

Example 1 with SearchHit

use of org.springframework.data.elasticsearch.core.SearchHit in project spring-data-elasticsearch by spring-projects.

the class ReactiveSearchAfterIntegrationTests method shouldReadPagesWithSearchAfter.

// #1143
@Test
@DisplayName("should read pages with search_after")
void shouldReadPagesWithSearchAfter() {
    List<Entity> entities = IntStream.rangeClosed(1, 10).mapToObj(i -> new Entity((long) i, "message " + i)).collect(Collectors.toList());
    operations.saveAll(Mono.just(entities), Entity.class).blockLast();
    Query query = Query.findAll();
    query.setPageable(PageRequest.of(0, 3));
    query.addSort(Sort.by(Sort.Direction.ASC, "id"));
    List<Object> searchAfter = null;
    List<Entity> foundEntities = new ArrayList<>();
    int loop = 0;
    do {
        query.setSearchAfter(searchAfter);
        List<SearchHit<Entity>> searchHits = operations.search(query, Entity.class).collectList().block();
        if (searchHits.size() == 0) {
            break;
        }
        foundEntities.addAll(searchHits.stream().map(searchHit -> searchHit.getContent()).collect(Collectors.toList()));
        searchAfter = searchHits.get((int) (searchHits.size() - 1)).getSortValues();
        if (++loop > 10) {
            fail("loop not terminating");
        }
    } while (true);
    assertThat(foundEntities).containsExactlyElementsOf(entities);
}
Also used : IntStream(java.util.stream.IntStream) ReactiveElasticsearchOperations(org.springframework.data.elasticsearch.core.ReactiveElasticsearchOperations) Field(org.springframework.data.elasticsearch.annotations.Field) Autowired(org.springframework.beans.factory.annotation.Autowired) PageRequest(org.springframework.data.domain.PageRequest) Mono(reactor.core.publisher.Mono) Query(org.springframework.data.elasticsearch.core.query.Query) Collectors(java.util.stream.Collectors) ReactiveElasticsearchRestTemplateConfiguration(org.springframework.data.elasticsearch.junit.jupiter.ReactiveElasticsearchRestTemplateConfiguration) ArrayList(java.util.ArrayList) DisplayName(org.junit.jupiter.api.DisplayName) Test(org.junit.jupiter.api.Test) SpringIntegrationTest(org.springframework.data.elasticsearch.junit.jupiter.SpringIntegrationTest) List(java.util.List) FieldType(org.springframework.data.elasticsearch.annotations.FieldType) Document(org.springframework.data.elasticsearch.annotations.Document) ContextConfiguration(org.springframework.test.context.ContextConfiguration) Assertions(org.assertj.core.api.Assertions) Sort(org.springframework.data.domain.Sort) Nullable(org.springframework.lang.Nullable) Id(org.springframework.data.annotation.Id) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) Query(org.springframework.data.elasticsearch.core.query.Query) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) ArrayList(java.util.ArrayList) Test(org.junit.jupiter.api.Test) SpringIntegrationTest(org.springframework.data.elasticsearch.junit.jupiter.SpringIntegrationTest) DisplayName(org.junit.jupiter.api.DisplayName)

Example 2 with SearchHit

use of org.springframework.data.elasticsearch.core.SearchHit in project fh by assecopl.

the class GenericDtoService method listDto.

@Override
public List<LIST> listDto(QUERY query) {
    BoolQueryBuilder queryBuilder = createQueryBuilderInternal(query);
    NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder().withQuery(queryBuilder);
    if (query.getFirstRow() != null && query.getSize() != null) {
        Sort sort;
        if (query.getSortProperty() != null) {
            sort = Sort.by((query.getAscending() != null && !query.getAscending()) ? Sort.Direction.DESC : Sort.Direction.ASC, query.getSortProperty());
        } else {
            sort = Sort.by(Sort.Direction.ASC, "id");
        }
        Pageable pageable;
        // bez sortowania jesli SortProperty = "0"
        if (query.getSortProperty() != null && query.getSortProperty().equals("0"))
            pageable = PageRequest.of(query.getFirstRow() / query.getSize(), query.getSize());
        else
            pageable = PageRequest.of(query.getFirstRow() / query.getSize(), query.getSize(), sort);
        searchQueryBuilder = searchQueryBuilder.withPageable(pageable);
    }
    NativeSearchQuery searchQuery = searchQueryBuilder.build();
    List<LIST> list = new ArrayList<>();
    try {
        IndexCoordinates indexCoordinates = elasticsearchTemplate.getIndexCoordinatesFor(listClazz);
        SearchHits<LIST> res = elasticsearchTemplate.search(searchQuery, listClazz, indexCoordinates);
        list = res.getSearchHits().stream().map(SearchHit::getContent).collect(Collectors.toList());
    } catch (Exception e) {
        log.warn("{}", ExceptionUtils.getStackTrace(e));
    }
    return list;
}
Also used : Pageable(org.springframework.data.domain.Pageable) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) Sort(org.springframework.data.domain.Sort) NativeSearchQueryBuilder(org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder) NativeSearchQuery(org.springframework.data.elasticsearch.core.query.NativeSearchQuery) IndexCoordinates(org.springframework.data.elasticsearch.core.mapping.IndexCoordinates)

Example 3 with SearchHit

use of org.springframework.data.elasticsearch.core.SearchHit in project openvsx by eclipse.

the class DatabaseSearchService method search.

@Cacheable("database.search")
public SearchHits<ExtensionSearch> search(ISearchService.Options options, Pageable pageRequest) {
    // grab all extensions
    var matchingExtensions = repositories.findAllActiveExtensions();
    // no extensions in the database
    if (matchingExtensions.isEmpty()) {
        Aggregations aggregations = new Aggregations(Collections.emptyList());
        return new SearchHitsImpl<ExtensionSearch>(0, TotalHitsRelation.OFF, 0f, "", Collections.emptyList(), aggregations);
    }
    // filter category
    if (options.category != null) {
        matchingExtensions = matchingExtensions.filter(extension -> extension.getLatest().getCategories().stream().anyMatch(category -> category.toLowerCase().equals(options.category.toLowerCase())));
    }
    // filter text
    if (options.queryString != null) {
        matchingExtensions = matchingExtensions.filter(extension -> extension.getName().toLowerCase().contains(options.queryString.toLowerCase()) || extension.getNamespace().getName().contains(options.queryString.toLowerCase()) || (extension.getLatest().getDescription() != null && extension.getLatest().getDescription().toLowerCase().contains(options.queryString.toLowerCase())) || (extension.getLatest().getDisplayName() != null && extension.getLatest().getDisplayName().toLowerCase().contains(options.queryString.toLowerCase())));
    }
    List<ExtensionSearch> sortedExtensions;
    if ("relevance".equals(options.sortBy)) {
        // for relevance we're using relevance service to get the relevance item
        var searchStats = new SearchStats(repositories);
        // needs to add relevance on extensions
        sortedExtensions = new ArrayList<>(matchingExtensions.map(extension -> relevanceService.toSearchEntry(extension, searchStats)).toList());
        // sort it
        sortedExtensions.sort(new RelevanceComparator());
    } else {
        sortedExtensions = matchingExtensions.stream().map(extension -> extension.toSearch()).collect(Collectors.toList());
        if ("downloadCount".equals(options.sortBy)) {
            sortedExtensions.sort(new DownloadedCountComparator());
        } else if ("averageRating".equals(options.sortBy)) {
            sortedExtensions.sort(new AverageRatingComparator());
        } else if ("timestamp".equals(options.sortBy)) {
            sortedExtensions.sort(new TimestampComparator());
        }
    }
    // 'asc' | 'desc';
    if ("desc".equals(options.sortOrder)) {
        // reverse the order
        Collections.reverse(sortedExtensions);
    }
    // Paging
    var totalHits = sortedExtensions.size();
    if (pageRequest != null) {
        var pageNumber = pageRequest.getPageNumber();
        var pageSize = pageRequest.getPageSize();
        var toSkip = 0;
        if (pageNumber >= 1) {
            // page is zero indexed
            toSkip = pageNumber * (pageSize - 1) + pageNumber;
        }
        // if something to skip, remove the first elements
        if (toSkip > 0 && toSkip < sortedExtensions.size()) {
            sortedExtensions = sortedExtensions.subList(toSkip, sortedExtensions.size());
        }
        // keep only the pageSize elements
        if (sortedExtensions.size() > pageSize) {
            sortedExtensions = sortedExtensions.subList(0, pageSize);
        }
    }
    List<SearchHit<ExtensionSearch>> searchHits;
    if (sortedExtensions.isEmpty()) {
        searchHits = Collections.emptyList();
    } else {
        // client is interested only in the extension IDs
        searchHits = sortedExtensions.stream().map(extensionSearch -> {
            return new SearchHit<ExtensionSearch>(null, null, 0.0f, Collections.emptyList().toArray(), Collections.emptyMap(), extensionSearch);
        }).collect(Collectors.toList());
    }
    Aggregations aggregations = new Aggregations(Collections.emptyList());
    SearchHits<ExtensionSearch> searchHitsResult = new SearchHitsImpl<ExtensionSearch>(totalHits, TotalHitsRelation.OFF, 0f, "", searchHits, aggregations);
    return searchHitsResult;
}
Also used : Extension(org.eclipse.openvsx.entities.Extension) Aggregations(org.elasticsearch.search.aggregations.Aggregations) SearchHits(org.springframework.data.elasticsearch.core.SearchHits) Cacheable(org.springframework.cache.annotation.Cacheable) SearchHitsImpl(org.springframework.data.elasticsearch.core.SearchHitsImpl) Autowired(org.springframework.beans.factory.annotation.Autowired) CacheEvict(org.springframework.cache.annotation.CacheEvict) Collectors(java.util.stream.Collectors) ArrayList(java.util.ArrayList) Value(org.springframework.beans.factory.annotation.Value) RepositoryService(org.eclipse.openvsx.repositories.RepositoryService) List(java.util.List) Component(org.springframework.stereotype.Component) SearchStats(org.eclipse.openvsx.search.RelevanceService.SearchStats) Pageable(org.springframework.data.domain.Pageable) Comparator(java.util.Comparator) Collections(java.util.Collections) TotalHitsRelation(org.springframework.data.elasticsearch.core.TotalHitsRelation) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) Aggregations(org.elasticsearch.search.aggregations.Aggregations) SearchHitsImpl(org.springframework.data.elasticsearch.core.SearchHitsImpl) SearchStats(org.eclipse.openvsx.search.RelevanceService.SearchStats) Cacheable(org.springframework.cache.annotation.Cacheable)

Example 4 with SearchHit

use of org.springframework.data.elasticsearch.core.SearchHit in project spring-data-elasticsearch by spring-projects.

the class SimpleElasticsearchRepository method findAll.

@SuppressWarnings("unchecked")
@Override
public Iterable<T> findAll(Sort sort) {
    int itemCount = (int) this.count();
    if (itemCount == 0) {
        return new PageImpl<>(Collections.emptyList());
    }
    NativeSearchQuery query = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).withPageable(PageRequest.of(0, itemCount, sort)).build();
    List<SearchHit<T>> searchHitList = execute(operations -> operations.search(query, entityClass, getIndexCoordinates()).getSearchHits());
    return (List<T>) SearchHitSupport.unwrapSearchHits(searchHitList);
}
Also used : PageImpl(org.springframework.data.domain.PageImpl) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) ArrayList(java.util.ArrayList) List(java.util.List) NativeSearchQueryBuilder(org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder) NativeSearchQuery(org.springframework.data.elasticsearch.core.query.NativeSearchQuery)

Example 5 with SearchHit

use of org.springframework.data.elasticsearch.core.SearchHit in project openvsx by eclipse.

the class LocalRegistryService method toSearchEntry.

private SearchEntryJson toSearchEntry(SearchHit<ExtensionSearch> searchHit, String serverUrl, ISearchService.Options options) {
    var searchItem = searchHit.getContent();
    var extension = entityManager.find(Extension.class, searchItem.id);
    if (extension == null || !extension.isActive()) {
        extension = new Extension();
        extension.setId(searchItem.id);
        search.removeSearchEntry(extension);
        return null;
    }
    var extVer = extension.getLatest();
    var entry = extVer.toSearchEntryJson();
    entry.url = createApiUrl(serverUrl, "api", entry.namespace, entry.name);
    entry.files = Maps.newLinkedHashMapWithExpectedSize(2);
    storageUtil.addFileUrls(extVer, serverUrl, entry.files, DOWNLOAD, ICON);
    if (options.includeAllVersions) {
        var allVersions = Lists.newArrayList(repositories.findActiveVersions(extension));
        Collections.sort(allVersions, ExtensionVersion.SORT_COMPARATOR);
        entry.allVersions = CollectionUtil.map(allVersions, ev -> toVersionReference(ev, entry, serverUrl));
    }
    return entry;
}
Also used : Extension(org.eclipse.openvsx.entities.Extension) Autowired(org.springframework.beans.factory.annotation.Autowired) QueryParamJson(org.eclipse.openvsx.json.QueryParamJson) CacheControl(org.springframework.http.CacheControl) ExtensionSearch(org.eclipse.openvsx.search.ExtensionSearch) ISearchService(org.eclipse.openvsx.search.ISearchService) FileResource(org.eclipse.openvsx.entities.FileResource) Namespace(org.eclipse.openvsx.entities.Namespace) ReviewJson(org.eclipse.openvsx.json.ReviewJson) ExtensionJson(org.eclipse.openvsx.json.ExtensionJson) SearchUtilService(org.eclipse.openvsx.search.SearchUtilService) ResponseStatusException(org.springframework.web.server.ResponseStatusException) Transactional(javax.transaction.Transactional) PageRequest(org.springframework.data.domain.PageRequest) ExtensionReview(org.eclipse.openvsx.entities.ExtensionReview) Collectors(java.util.stream.Collectors) Stream(java.util.stream.Stream) Maps(org.apache.jena.ext.com.google.common.collect.Maps) org.eclipse.openvsx.dto(org.eclipse.openvsx.dto) CollectionUtil(org.eclipse.openvsx.util.CollectionUtil) QueryResultJson(org.eclipse.openvsx.json.QueryResultJson) NamespaceMembership(org.eclipse.openvsx.entities.NamespaceMembership) java.util(java.util) Iterables(com.google.common.collect.Iterables) ExtensionVersion(org.eclipse.openvsx.entities.ExtensionVersion) ReviewListJson(org.eclipse.openvsx.json.ReviewListJson) ErrorResultException(org.eclipse.openvsx.util.ErrorResultException) UrlUtil(org.eclipse.openvsx.util.UrlUtil) TimeUtil(org.eclipse.openvsx.util.TimeUtil) Strings(com.google.common.base.Strings) Lists(com.google.common.collect.Lists) NamespaceJson(org.eclipse.openvsx.json.NamespaceJson) NotFoundException(org.eclipse.openvsx.util.NotFoundException) SearchHit(org.springframework.data.elasticsearch.core.SearchHit) UrlUtil.createApiUrl(org.eclipse.openvsx.util.UrlUtil.createApiUrl) Extension(org.eclipse.openvsx.entities.Extension) SearchHits(org.springframework.data.elasticsearch.core.SearchHits) SemanticVersion(org.eclipse.openvsx.util.SemanticVersion) EntityManager(javax.persistence.EntityManager) ResultJson(org.eclipse.openvsx.json.ResultJson) TimeUnit(java.util.concurrent.TimeUnit) SearchResultJson(org.eclipse.openvsx.json.SearchResultJson) RepositoryService(org.eclipse.openvsx.repositories.RepositoryService) HttpStatus(org.springframework.http.HttpStatus) Component(org.springframework.stereotype.Component) SearchEntryJson(org.eclipse.openvsx.json.SearchEntryJson) ResponseEntity(org.springframework.http.ResponseEntity) EclipseService(org.eclipse.openvsx.eclipse.EclipseService) InputStream(java.io.InputStream) StorageUtilService(org.eclipse.openvsx.storage.StorageUtilService)

Aggregations

SearchHit (org.springframework.data.elasticsearch.core.SearchHit)8 ArrayList (java.util.ArrayList)3 List (java.util.List)3 Collectors (java.util.stream.Collectors)3 ExtensionSearch (org.eclipse.openvsx.search.ExtensionSearch)3 Aggregations (org.elasticsearch.search.aggregations.Aggregations)3 SearchHitsImpl (org.springframework.data.elasticsearch.core.SearchHitsImpl)3 Extension (org.eclipse.openvsx.entities.Extension)2 RepositoryService (org.eclipse.openvsx.repositories.RepositoryService)2 Autowired (org.springframework.beans.factory.annotation.Autowired)2 Pageable (org.springframework.data.domain.Pageable)2 Sort (org.springframework.data.domain.Sort)2 Strings (com.google.common.base.Strings)1 Iterables (com.google.common.collect.Iterables)1 Lists (com.google.common.collect.Lists)1 InputStream (java.io.InputStream)1 java.util (java.util)1 Collections (java.util.Collections)1 Comparator (java.util.Comparator)1 TimeUnit (java.util.concurrent.TimeUnit)1