Search in sources :

Example 1 with SearchHandler

use of com.yelp.nrtsearch.server.luceneserver.SearchHandler in project nrtsearch by Yelp.

the class Warmer method warmFromS3.

public void warmFromS3(IndexState indexState, int parallelism) throws IOException, SearchHandler.SearchHandlerException, InterruptedException {
    SearchHandler searchHandler = new SearchHandler(indexState.getSearchThreadPoolExecutor(), true);
    warmFromS3(indexState, parallelism, searchHandler);
}
Also used : SearchHandler(com.yelp.nrtsearch.server.luceneserver.SearchHandler)

Example 2 with SearchHandler

use of com.yelp.nrtsearch.server.luceneserver.SearchHandler in project nrtsearch by Yelp.

the class WarmerTest method testWarmFromS3_parallel.

@Test
public void testWarmFromS3_parallel() throws IOException, SearchHandler.SearchHandlerException, InterruptedException {
    Path warmingQueriesDir = folder.newFolder("warming_queries").toPath();
    int warmingCountPerQuery = 10;
    try (BufferedWriter writer = Files.newBufferedWriter(warmingQueriesDir.resolve("warming_queries.txt"))) {
        List<String> testSearchRequestsJson = getTestSearchRequestsAsJsonStrings();
        List<String> moreTestSearchRequestsJson = new ArrayList<>();
        for (int i = 0; i < warmingCountPerQuery; i++) {
            moreTestSearchRequestsJson.addAll(testSearchRequestsJson);
        }
        for (String line : moreTestSearchRequestsJson) {
            writer.write(line);
            writer.newLine();
        }
        writer.flush();
    }
    String versionHash = archiver.upload(service, resource, warmingQueriesDir, List.of(), List.of(), false);
    archiver.blessVersion(service, resource, versionHash);
    IndexState mockIndexState = mock(IndexState.class);
    SearchHandler mockSearchHandler = mock(SearchHandler.class);
    warmer.warmFromS3(mockIndexState, 3, mockSearchHandler);
    for (SearchRequest testRequest : getTestSearchRequests()) {
        verify(mockSearchHandler, times(warmingCountPerQuery)).handle(mockIndexState, testRequest);
    }
    verifyNoMoreInteractions(mockSearchHandler);
}
Also used : Path(java.nio.file.Path) SearchHandler(com.yelp.nrtsearch.server.luceneserver.SearchHandler) SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) ArrayList(java.util.ArrayList) IndexState(com.yelp.nrtsearch.server.luceneserver.IndexState) BufferedWriter(java.io.BufferedWriter) Test(org.junit.Test)

Example 3 with SearchHandler

use of com.yelp.nrtsearch.server.luceneserver.SearchHandler in project nrtsearch by Yelp.

the class WarmerTest method testWarmFromS3.

@Test
public void testWarmFromS3() throws IOException, SearchHandler.SearchHandlerException, InterruptedException {
    Path warmingQueriesDir = folder.newFolder("warming_queries").toPath();
    try (BufferedWriter writer = Files.newBufferedWriter(warmingQueriesDir.resolve("warming_queries.txt"))) {
        List<String> testSearchRequestsJson = getTestSearchRequestsAsJsonStrings();
        for (String line : testSearchRequestsJson) {
            writer.write(line);
            writer.newLine();
        }
        writer.flush();
    }
    String versionHash = archiver.upload(service, resource, warmingQueriesDir, List.of(), List.of(), false);
    archiver.blessVersion(service, resource, versionHash);
    IndexState mockIndexState = mock(IndexState.class);
    SearchHandler mockSearchHandler = mock(SearchHandler.class);
    warmer.warmFromS3(mockIndexState, 0, mockSearchHandler);
    for (SearchRequest testRequest : getTestSearchRequests()) {
        verify(mockSearchHandler).handle(mockIndexState, testRequest);
    }
    verifyNoMoreInteractions(mockSearchHandler);
}
Also used : Path(java.nio.file.Path) SearchHandler(com.yelp.nrtsearch.server.luceneserver.SearchHandler) SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) IndexState(com.yelp.nrtsearch.server.luceneserver.IndexState) BufferedWriter(java.io.BufferedWriter) Test(org.junit.Test)

Example 4 with SearchHandler

use of com.yelp.nrtsearch.server.luceneserver.SearchHandler in project nrtsearch by Yelp.

the class WarmerTest method testWarmFromS3_multiple.

@Test
public void testWarmFromS3_multiple() throws IOException, SearchHandler.SearchHandlerException, InterruptedException {
    Path warmingQueriesDir = folder.newFolder("warming_queries").toPath();
    try (BufferedWriter writer = Files.newBufferedWriter(warmingQueriesDir.resolve("warming_queries.txt"))) {
        List<String> testSearchRequestsJson = getTestSearchRequestsAsJsonStrings();
        for (String line : testSearchRequestsJson) {
            writer.write(line);
            writer.newLine();
        }
        writer.flush();
    }
    String versionHash = archiver.upload(service, resource, warmingQueriesDir, List.of(), List.of(), false);
    archiver.blessVersion(service, resource, versionHash);
    IndexState mockIndexState = mock(IndexState.class);
    SearchHandler mockSearchHandler = mock(SearchHandler.class);
    warmer.warmFromS3(mockIndexState, 0, mockSearchHandler);
    warmer.warmFromS3(mockIndexState, 0, mockSearchHandler);
    for (SearchRequest testRequest : getTestSearchRequests()) {
        verify(mockSearchHandler, times(2)).handle(mockIndexState, testRequest);
    }
    verifyNoMoreInteractions(mockSearchHandler);
}
Also used : Path(java.nio.file.Path) SearchHandler(com.yelp.nrtsearch.server.luceneserver.SearchHandler) SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) IndexState(com.yelp.nrtsearch.server.luceneserver.IndexState) BufferedWriter(java.io.BufferedWriter) Test(org.junit.Test)

Aggregations

SearchHandler (com.yelp.nrtsearch.server.luceneserver.SearchHandler)4 SearchRequest (com.yelp.nrtsearch.server.grpc.SearchRequest)3 IndexState (com.yelp.nrtsearch.server.luceneserver.IndexState)3 BufferedWriter (java.io.BufferedWriter)3 Path (java.nio.file.Path)3 Test (org.junit.Test)3 ArrayList (java.util.ArrayList)1