Search in sources :

Example 1 with SearchRequest

use of com.yelp.nrtsearch.server.grpc.SearchRequest in project nrtsearch by Yelp.

the class Warmer method backupWarmingQueriesToS3.

public synchronized void backupWarmingQueriesToS3(String service) throws IOException {
    if (Strings.isNullOrEmpty(service)) {
        service = this.service;
    }
    // TODO: tmpDirectory used for simplicity but we might want to provide directory via config
    Path tmpDirectory = Paths.get(System.getProperty("java.io.tmpdir"));
    Path warmingQueriesDir = null;
    Path warmingQueriesFile = null;
    BufferedWriter writer = null;
    try {
        // Creating a directory since the Archiver requires a directory
        warmingQueriesDir = Files.createDirectory(tmpDirectory.resolve(WARMING_QUERIES_DIR));
        warmingQueriesFile = warmingQueriesDir.resolve(WARMING_QUERIES_FILE);
        writer = Files.newBufferedWriter(warmingQueriesFile);
        int count = 0;
        for (SearchRequest searchRequest : warmingRequests) {
            writer.write(JsonFormat.printer().omittingInsignificantWhitespace().print(searchRequest));
            writer.newLine();
            count++;
        }
        writer.close();
        writer = null;
        String versionHash = archiver.upload(service, resource, warmingQueriesDir, List.of(), List.of(), true);
        archiver.blessVersion(service, resource, versionHash);
        logger.info("Backed up {} warming queries for index: {} to service: {}, resource: {}", count, index, service, resource);
    } finally {
        if (writer != null) {
            writer.close();
        }
        if (warmingQueriesFile != null && Files.exists(warmingQueriesFile)) {
            Files.delete(warmingQueriesFile);
        }
        if (warmingQueriesDir != null && Files.exists(warmingQueriesDir)) {
            Files.delete(warmingQueriesDir);
        }
    }
}
Also used : Path(java.nio.file.Path) SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) BufferedWriter(java.io.BufferedWriter)

Example 2 with SearchRequest

use of com.yelp.nrtsearch.server.grpc.SearchRequest in project nrtsearch by Yelp.

the class WarmerTest method getTestSearchRequests.

private List<SearchRequest> getTestSearchRequests() {
    List<SearchRequest> testRequests = new ArrayList<>();
    for (int i = 0; i < 2; i++) {
        SearchRequest searchRequest = SearchRequest.newBuilder().setIndexName(index).setQuery(Query.newBuilder().setTermQuery(TermQuery.newBuilder().setField("field" + i).build()).build()).build();
        testRequests.add(searchRequest);
    }
    return testRequests;
}
Also used : SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) ArrayList(java.util.ArrayList)

Example 3 with SearchRequest

use of com.yelp.nrtsearch.server.grpc.SearchRequest in project nrtsearch by Yelp.

the class SharedDocContextTest method testSharedDocContext.

@Test
public void testSharedDocContext() {
    SearchRequest request = SearchRequest.newBuilder().setTopHits(10).setStartHit(0).setIndexName(DEFAULT_TEST_INDEX).addRetrieveFields("int_score").setQuery(Query.newBuilder().setRangeQuery(RangeQuery.newBuilder().setField("int_field").setLower("20").setUpper("29").build()).build()).addFetchTasks(com.yelp.nrtsearch.server.grpc.FetchTask.newBuilder().setName("phase_1").build()).addFetchTasks(com.yelp.nrtsearch.server.grpc.FetchTask.newBuilder().setName("phase_2").build()).build();
    SearchResponse response = getGrpcServer().getBlockingStub().search(request);
    assertEquals(10, response.getHitsCount());
    for (Hit hit : response.getHitsList()) {
        assertEquals(hit.getFieldsOrThrow("_context_value").getFieldValue(0).getIntValue(), hit.getFieldsOrThrow("int_score").getFieldValue(0).getIntValue() * 2);
    }
}
Also used : SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) Hit(com.yelp.nrtsearch.server.grpc.SearchResponse.Hit) SearchResponse(com.yelp.nrtsearch.server.grpc.SearchResponse) Test(org.junit.Test)

Example 4 with SearchRequest

use of com.yelp.nrtsearch.server.grpc.SearchRequest in project nrtsearch by Yelp.

the class DocCollectorTest method testNoTimeoutWrapper.

@Test
public void testNoTimeoutWrapper() {
    SearchRequest request = SearchRequest.newBuilder().setTopHits(10).build();
    TestDocCollector docCollector = new TestDocCollector(request);
    assertTrue(docCollector.getManager() instanceof TestDocCollector.TestCollectorManager);
    assertTrue(docCollector.getWrappedManager() instanceof SearchCollectorManager);
    assertSame(docCollector.getManager(), ((SearchCollectorManager) docCollector.getWrappedManager()).getDocCollectorManger());
}
Also used : SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) SearchCollectorManager(com.yelp.nrtsearch.server.luceneserver.search.SearchCollectorManager) Test(org.junit.Test)

Example 5 with SearchRequest

use of com.yelp.nrtsearch.server.grpc.SearchRequest in project nrtsearch by Yelp.

the class DocCollectorTest method testHasStatsWrapper.

@Test
public void testHasStatsWrapper() {
    SearchRequest request = SearchRequest.newBuilder().setTopHits(10).setProfile(true).build();
    TestDocCollector docCollector = new TestDocCollector(request);
    assertTrue(docCollector.getManager() instanceof TestDocCollector.TestCollectorManager);
    assertTrue(docCollector.getWrappedManager() instanceof SearchStatsWrapper);
    SearchStatsWrapper<?> searchStatsWrapper = (SearchStatsWrapper<?>) docCollector.getWrappedManager();
    assertTrue(searchStatsWrapper.getWrapped() instanceof SearchCollectorManager);
    SearchCollectorManager searchCollectorManager = (SearchCollectorManager) searchStatsWrapper.getWrapped();
    assertSame(docCollector.getManager(), searchCollectorManager.getDocCollectorManger());
}
Also used : SearchRequest(com.yelp.nrtsearch.server.grpc.SearchRequest) SearchCollectorManager(com.yelp.nrtsearch.server.luceneserver.search.SearchCollectorManager) SearchStatsWrapper(com.yelp.nrtsearch.server.luceneserver.search.SearchStatsWrapper) Test(org.junit.Test)

Aggregations

SearchRequest (com.yelp.nrtsearch.server.grpc.SearchRequest)28 Test (org.junit.Test)21 IndexState (com.yelp.nrtsearch.server.luceneserver.IndexState)10 SearchCutoffWrapper (com.yelp.nrtsearch.server.luceneserver.search.SearchCutoffWrapper)7 SearchResponse (com.yelp.nrtsearch.server.grpc.SearchResponse)6 Path (java.nio.file.Path)5 SearchHandler (com.yelp.nrtsearch.server.luceneserver.SearchHandler)4 TerminateAfterWrapper (com.yelp.nrtsearch.server.luceneserver.search.TerminateAfterWrapper)4 BufferedWriter (java.io.BufferedWriter)4 ArrayList (java.util.ArrayList)4 SearchStatsWrapper (com.yelp.nrtsearch.server.luceneserver.search.SearchStatsWrapper)3 StatusRuntimeException (io.grpc.StatusRuntimeException)3 IOException (java.io.IOException)3 CollectorResult (com.yelp.nrtsearch.server.grpc.CollectorResult)2 IndexableFieldDef (com.yelp.nrtsearch.server.luceneserver.field.IndexableFieldDef)2 SearchCollectorManager (com.yelp.nrtsearch.server.luceneserver.search.SearchCollectorManager)2 Collections (java.util.Collections)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2