Search in sources :

Example 1 with FutureResult

use of com.yahoo.search.searchchain.FutureResult in project vespa by vespa-engine.

the class FederationSearcher method fill.

@Override
public void fill(Result result, String summaryClass, Execution execution) {
    UniqueExecutionsToResults uniqueExecutionsToResults = new UniqueExecutionsToResults();
    addResultsToFill(result.hits(), result, summaryClass, uniqueExecutionsToResults);
    Set<Entry<Chain<Searcher>, Map<Query, Result>>> resultsForAllChains = uniqueExecutionsToResults.resultsToFill.entrySet();
    int numberOfCallsToFillNeeded = 0;
    for (Entry<Chain<Searcher>, Map<Query, Result>> resultsToFillForAChain : resultsForAllChains) {
        numberOfCallsToFillNeeded += resultsToFillForAChain.getValue().size();
    }
    List<Pair<Result, FutureResult>> futureFilledResults = new ArrayList<>();
    for (Entry<Chain<Searcher>, Map<Query, Result>> resultsToFillForAChain : resultsForAllChains) {
        Chain<Searcher> chain = resultsToFillForAChain.getKey();
        Execution chainExecution = (chain == null) ? execution : new Execution(chain, execution.context());
        for (Entry<Query, Result> resultsToFillForAChainAndQuery : resultsToFillForAChain.getValue().entrySet()) {
            Result resultToFill = resultsToFillForAChainAndQuery.getValue();
            if (numberOfCallsToFillNeeded == 1) {
                chainExecution.fill(resultToFill, summaryClass);
                propagateErrors(resultToFill, result);
            } else {
                AsyncExecution asyncFill = new AsyncExecution(chainExecution);
                futureFilledResults.add(new Pair<>(resultToFill, asyncFill.fill(resultToFill, summaryClass)));
            }
        }
    }
    for (Pair<Result, FutureResult> futureFilledResult : futureFilledResults) {
        // futureFilledResult is a pair of a result to be filled and the future in which that same result is filled
        Optional<Result> filledResult = futureFilledResult.getSecond().getIfAvailable(result.getQuery().getTimeLeft(), TimeUnit.MILLISECONDS);
        if (filledResult.isPresent()) {
            // fill completed
            propagateErrors(filledResult.get(), result);
        } else {
            // fill timed out: Remove these hits as they are incomplete and may cause a race when accessed later
            result.hits().addError(futureFilledResult.getSecond().createTimeoutError());
            for (Iterator<Hit> i = futureFilledResult.getFirst().hits().unorderedDeepIterator(); i.hasNext(); ) {
                // Note that some of these hits may be filled, but as the fill thread may still be working on them
                // and we do not synchronize with it we need to discard all
                Hit removed = result.hits().remove(i.next().getId());
            }
        }
    }
}
Also used : Chain(com.yahoo.component.chain.Chain) Query(com.yahoo.search.Query) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Searcher(com.yahoo.search.Searcher) ForkingSearcher(com.yahoo.search.searchchain.ForkingSearcher) ArrayList(java.util.ArrayList) Result(com.yahoo.search.Result) FutureResult(com.yahoo.search.searchchain.FutureResult) Entry(java.util.Map.Entry) Hit(com.yahoo.search.result.Hit) Execution(com.yahoo.search.searchchain.Execution) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) FutureResult(com.yahoo.search.searchchain.FutureResult) Map(java.util.Map) IdentityHashMap(java.util.IdentityHashMap) CopyOnWriteHashMap(com.yahoo.concurrent.CopyOnWriteHashMap) Pair(com.yahoo.collections.Pair)

Example 2 with FutureResult

use of com.yahoo.search.searchchain.FutureResult in project vespa by vespa-engine.

the class FederationSearcher method searchAsynchronously.

private FutureResult searchAsynchronously(Query query, Execution execution, Window window, Target target) {
    long timeout = target.federationOptions().getSearchChainExecutionTimeoutInMilliseconds(query.getTimeLeft());
    if (timeout <= 0)
        return new FutureResult(() -> new Result(query, ErrorMessage.createTimeout("Timed out before federation")), execution, query);
    Query clonedQuery = cloneFederationQuery(query, window, timeout, target);
    return new AsyncExecution(target.getChain(), execution).search(clonedQuery);
}
Also used : FutureResult(com.yahoo.search.searchchain.FutureResult) Query(com.yahoo.search.Query) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Result(com.yahoo.search.Result) FutureResult(com.yahoo.search.searchchain.FutureResult)

Example 3 with FutureResult

use of com.yahoo.search.searchchain.FutureResult in project vespa by vespa-engine.

the class AsyncExecutionTestCase method testAsync.

// This should take ~50+ ms
public void testAsync() {
    List<Searcher> searchList = new ArrayList<>();
    searchList.add(new WaitingSearcher("one", 60000));
    searchList.add(new WaitingSearcher("two", 0));
    Chain<Searcher> searchChain = new Chain<>(new ComponentId("chain"), searchList);
    AsyncExecution asyncExecution = new AsyncExecution(searchChain, Execution.Context.createContextStub());
    FutureResult future = asyncExecution.search(new Query("?hits=0"));
    Result result = future.get(0, TimeUnit.MILLISECONDS);
    assertTrue(result.hits().getError() != null);
}
Also used : Chain(com.yahoo.component.chain.Chain) FutureResult(com.yahoo.search.searchchain.FutureResult) Query(com.yahoo.search.Query) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Searcher(com.yahoo.search.Searcher) ArrayList(java.util.ArrayList) ComponentId(com.yahoo.component.ComponentId) FutureResult(com.yahoo.search.searchchain.FutureResult) Result(com.yahoo.search.Result)

Example 4 with FutureResult

use of com.yahoo.search.searchchain.FutureResult in project vespa by vespa-engine.

the class AsyncExecutionTestCase method testAsyncThroughSync.

public void testAsyncThroughSync() {
    Query query = new Query("?query=test");
    Searcher searcher = new ResultProducingSearcher();
    FutureResult futureResult = new AsyncExecution(new Execution(searcher, Execution.Context.createContextStub())).search(query);
    List<FutureResult> futureResultList = new ArrayList<>();
    futureResultList.add(futureResult);
    AsyncExecution.waitForAll(futureResultList, 1000);
    Result result = futureResult.get();
    assertEquals(1, result.hits().size());
    assertEquals("hello", result.hits().get(0).getField("test"));
}
Also used : Execution(com.yahoo.search.searchchain.Execution) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Query(com.yahoo.search.Query) FutureResult(com.yahoo.search.searchchain.FutureResult) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Searcher(com.yahoo.search.Searcher) ArrayList(java.util.ArrayList) FutureResult(com.yahoo.search.searchchain.FutureResult) Result(com.yahoo.search.Result)

Example 5 with FutureResult

use of com.yahoo.search.searchchain.FutureResult in project vespa by vespa-engine.

the class AsyncExecutionTestCase method testWaitForAll.

public void testWaitForAll() {
    Chain<Searcher> slowChain = new Chain<>(new ComponentId("slow"), Arrays.asList(new Searcher[] { new WaitingSearcher("slow", 30000) }));
    Chain<Searcher> fastChain = new Chain<>(new ComponentId("fast"), Arrays.asList(new Searcher[] { new SimpleSearcher() }));
    FutureResult slowFuture = new AsyncExecution(slowChain, Execution.Context.createContextStub()).search(new Query("?hits=0"));
    FutureResult fastFuture = new AsyncExecution(fastChain, Execution.Context.createContextStub()).search(new Query("?hits=0"));
    fastFuture.get();
    FutureResult[] reslist = new FutureResult[] { slowFuture, fastFuture };
    List<Result> results = AsyncExecution.waitForAll(Arrays.asList(reslist), 0);
    // assertTrue(slowFuture.isCancelled());
    assertTrue(fastFuture.isDone() && !fastFuture.isCancelled());
    assertNotNull(results.get(0).hits().getErrorHit());
    assertNull(results.get(1).hits().getErrorHit());
}
Also used : Chain(com.yahoo.component.chain.Chain) FutureResult(com.yahoo.search.searchchain.FutureResult) Query(com.yahoo.search.Query) AsyncExecution(com.yahoo.search.searchchain.AsyncExecution) Searcher(com.yahoo.search.Searcher) ComponentId(com.yahoo.component.ComponentId) FutureResult(com.yahoo.search.searchchain.FutureResult) Result(com.yahoo.search.Result)

Aggregations

Query (com.yahoo.search.Query)6 Result (com.yahoo.search.Result)6 AsyncExecution (com.yahoo.search.searchchain.AsyncExecution)6 FutureResult (com.yahoo.search.searchchain.FutureResult)6 Searcher (com.yahoo.search.Searcher)5 Chain (com.yahoo.component.chain.Chain)4 Execution (com.yahoo.search.searchchain.Execution)3 ArrayList (java.util.ArrayList)3 ComponentId (com.yahoo.component.ComponentId)2 Pair (com.yahoo.collections.Pair)1 CopyOnWriteHashMap (com.yahoo.concurrent.CopyOnWriteHashMap)1 Hit (com.yahoo.search.result.Hit)1 ForkingSearcher (com.yahoo.search.searchchain.ForkingSearcher)1 IdentityHashMap (java.util.IdentityHashMap)1 Map (java.util.Map)1 Entry (java.util.Map.Entry)1