Search in sources :

Example 51 with IntArrayList

use of com.carrotsearch.hppc.IntArrayList in project elasticsearch by elastic.

the class MultiGetShardResponse method readFrom.

@Override
public void readFrom(StreamInput in) throws IOException {
    super.readFrom(in);
    int size = in.readVInt();
    locations = new IntArrayList(size);
    responses = new ArrayList<>(size);
    failures = new ArrayList<>(size);
    for (int i = 0; i < size; i++) {
        locations.add(in.readVInt());
        if (in.readBoolean()) {
            GetResponse response = new GetResponse();
            response.readFrom(in);
            responses.add(response);
        } else {
            responses.add(null);
        }
        if (in.readBoolean()) {
            failures.add(MultiGetResponse.Failure.readFailure(in));
        } else {
            failures.add(null);
        }
    }
}
Also used : IntArrayList(com.carrotsearch.hppc.IntArrayList)

Example 52 with IntArrayList

use of com.carrotsearch.hppc.IntArrayList in project elasticsearch by elastic.

the class FetchSearchPhase method innerRun.

private void innerRun() throws IOException {
    final int numShards = context.getNumShards();
    final boolean isScrollSearch = context.getRequest().scroll() != null;
    ScoreDoc[] sortedShardDocs = searchPhaseController.sortDocs(isScrollSearch, queryResults);
    String scrollId = isScrollSearch ? TransportSearchHelper.buildScrollId(queryResults) : null;
    List<AtomicArray.Entry<QuerySearchResultProvider>> queryResultsAsList = queryResults.asList();
    final SearchPhaseController.ReducedQueryPhase reducedQueryPhase = resultConsumer.reduce();
    final boolean queryAndFetchOptimization = queryResults.length() == 1;
    final Runnable finishPhase = () -> moveToNextPhase(searchPhaseController, sortedShardDocs, scrollId, reducedQueryPhase, queryAndFetchOptimization ? queryResults : fetchResults);
    if (queryAndFetchOptimization) {
        assert queryResults.get(0) == null || queryResults.get(0).fetchResult() != null;
        // query AND fetch optimization
        finishPhase.run();
    } else {
        final IntArrayList[] docIdsToLoad = searchPhaseController.fillDocIdsToLoad(numShards, sortedShardDocs);
        if (sortedShardDocs.length == 0) {
            // no docs to fetch -- sidestep everything and return
            queryResultsAsList.stream().map(e -> e.value.queryResult()).forEach(// we have to release contexts here to free up resources
            this::releaseIrrelevantSearchContext);
            finishPhase.run();
        } else {
            final ScoreDoc[] lastEmittedDocPerShard = isScrollSearch ? searchPhaseController.getLastEmittedDocPerShard(reducedQueryPhase, sortedShardDocs, numShards) : null;
            final CountedCollector<FetchSearchResult> counter = new CountedCollector<>(fetchResults::set, // we count down every shard in the result no matter if we got any results or not
            docIdsToLoad.length, finishPhase, context);
            for (int i = 0; i < docIdsToLoad.length; i++) {
                IntArrayList entry = docIdsToLoad[i];
                QuerySearchResultProvider queryResult = queryResults.get(i);
                if (entry == null) {
                    // no results for this shard ID
                    if (queryResult != null) {
                        // if we got some hits from this shard we have to release the context there
                        // we do this as we go since it will free up resources and passing on the request on the
                        // transport layer is cheap.
                        releaseIrrelevantSearchContext(queryResult.queryResult());
                    }
                    // in any case we count down this result since we don't talk to this shard anymore
                    counter.countDown();
                } else {
                    Transport.Connection connection = context.getConnection(queryResult.shardTarget().getNodeId());
                    ShardFetchSearchRequest fetchSearchRequest = createFetchRequest(queryResult.queryResult().id(), i, entry, lastEmittedDocPerShard);
                    executeFetch(i, queryResult.shardTarget(), counter, fetchSearchRequest, queryResult.queryResult(), connection);
                }
            }
        }
    }
}
Also used : SearchShardTarget(org.elasticsearch.search.SearchShardTarget) InternalSearchResponse(org.elasticsearch.search.internal.InternalSearchResponse) Transport(org.elasticsearch.transport.Transport) ScoreDoc(org.apache.lucene.search.ScoreDoc) AtomicArray(org.elasticsearch.common.util.concurrent.AtomicArray) IOException(java.io.IOException) ShardFetchSearchRequest(org.elasticsearch.search.fetch.ShardFetchSearchRequest) ParameterizedMessage(org.apache.logging.log4j.message.ParameterizedMessage) Function(java.util.function.Function) QuerySearchResultProvider(org.elasticsearch.search.query.QuerySearchResultProvider) List(java.util.List) Logger(org.apache.logging.log4j.Logger) FetchSearchResult(org.elasticsearch.search.fetch.FetchSearchResult) QuerySearchResult(org.elasticsearch.search.query.QuerySearchResult) Supplier(org.apache.logging.log4j.util.Supplier) IntArrayList(com.carrotsearch.hppc.IntArrayList) ActionRunnable(org.elasticsearch.action.ActionRunnable) ActionListener(org.elasticsearch.action.ActionListener) QuerySearchResultProvider(org.elasticsearch.search.query.QuerySearchResultProvider) FetchSearchResult(org.elasticsearch.search.fetch.FetchSearchResult) ScoreDoc(org.apache.lucene.search.ScoreDoc) ShardFetchSearchRequest(org.elasticsearch.search.fetch.ShardFetchSearchRequest) ActionRunnable(org.elasticsearch.action.ActionRunnable) IntArrayList(com.carrotsearch.hppc.IntArrayList) Transport(org.elasticsearch.transport.Transport)

Example 53 with IntArrayList

use of com.carrotsearch.hppc.IntArrayList in project elasticsearch by elastic.

the class MultiTermVectorsShardRequest method readFrom.

@Override
public void readFrom(StreamInput in) throws IOException {
    super.readFrom(in);
    int size = in.readVInt();
    locations = new IntArrayList(size);
    requests = new ArrayList<>(size);
    for (int i = 0; i < size; i++) {
        locations.add(in.readVInt());
        requests.add(TermVectorsRequest.readTermVectorsRequest(in));
    }
    preference = in.readOptionalString();
}
Also used : IntArrayList(com.carrotsearch.hppc.IntArrayList)

Example 54 with IntArrayList

use of com.carrotsearch.hppc.IntArrayList in project crate by crate.

the class ShardRequest method readFrom.

@Override
public void readFrom(StreamInput in) throws IOException {
    super.readFrom(in);
    routing = in.readOptionalString();
    jobId = new UUID(in.readLong(), in.readLong());
    int size = in.readVInt();
    locations = new IntArrayList(size);
    for (int i = 0; i < size; i++) {
        locations.add(in.readVInt());
    }
}
Also used : UUID(java.util.UUID) IntArrayList(com.carrotsearch.hppc.IntArrayList)

Example 55 with IntArrayList

use of com.carrotsearch.hppc.IntArrayList in project crate by crate.

the class ShardResponse method readFrom.

@Override
public void readFrom(StreamInput in) throws IOException {
    super.readFrom(in);
    int size = in.readVInt();
    locations = new IntArrayList(size);
    failures = new ArrayList<>(size);
    for (int i = 0; i < size; i++) {
        locations.add(in.readVInt());
        if (in.readBoolean()) {
            failures.add(Failure.readFailure(in));
        } else {
            failures.add(null);
        }
    }
    if (in.readBoolean()) {
        failure = in.readThrowable();
    }
}
Also used : IntArrayList(com.carrotsearch.hppc.IntArrayList)

Aggregations

IntArrayList (com.carrotsearch.hppc.IntArrayList)94 Test (org.junit.jupiter.api.Test)16 RepeatedTest (org.junit.jupiter.api.RepeatedTest)13 GHPoint (com.graphhopper.util.shapes.GHPoint)10 ArrayList (java.util.ArrayList)9 Test (org.junit.Test)9 GraphHopperStorage (com.graphhopper.storage.GraphHopperStorage)7 IntObjectHashMap (com.carrotsearch.hppc.IntObjectHashMap)6 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)6 IntIndexedContainer (com.carrotsearch.hppc.IntIndexedContainer)5 HashMap (java.util.HashMap)5 UUID (java.util.UUID)5 GHIntArrayList (com.graphhopper.coll.GHIntArrayList)4 EdgeIteratorState (com.graphhopper.util.EdgeIteratorState)4 RelationName (io.crate.metadata.RelationName)4 IOException (java.io.IOException)4 IntObjectMap (com.carrotsearch.hppc.IntObjectMap)3 IntCursor (com.carrotsearch.hppc.cursors.IntCursor)3 PrepEdgeFilter (com.graphhopper.routing.subnetwork.PrepareRoutingSubnetworks.PrepEdgeFilter)3 FlagEncoder (com.graphhopper.routing.util.FlagEncoder)3