Search in sources :

Example 1 with Tuple

use of org.elasticsearch.common.collect.Tuple in project crate by crate.

the class ExecutionPhasesTask method createHandlerPhaseAndReceivers.

private List<Tuple<ExecutionPhase, BatchConsumer>> createHandlerPhaseAndReceivers(List<ExecutionPhase> handlerPhases, List<BatchConsumer> handlerReceivers, InitializationTracker initializationTracker) {
    List<Tuple<ExecutionPhase, BatchConsumer>> handlerPhaseAndReceiver = new ArrayList<>();
    ListIterator<BatchConsumer> consumerIt = handlerReceivers.listIterator();
    for (ExecutionPhase handlerPhase : handlerPhases) {
        InterceptingBatchConsumer interceptingBatchConsumer = new InterceptingBatchConsumer(jobId(), consumerIt.next(), initializationTracker, transportKillJobsNodeAction);
        handlerPhaseAndReceiver.add(new Tuple<>(handlerPhase, interceptingBatchConsumer));
    }
    return handlerPhaseAndReceiver;
}
Also used : ExecutionPhase(io.crate.planner.node.ExecutionPhase) Tuple(org.elasticsearch.common.collect.Tuple) BatchConsumer(io.crate.data.BatchConsumer) CollectingBatchConsumer(io.crate.data.CollectingBatchConsumer)

Example 2 with Tuple

use of org.elasticsearch.common.collect.Tuple in project crate by crate.

the class WhereClauseAnalyzer method tieBreakPartitionQueries.

private static WhereClause tieBreakPartitionQueries(EvaluatingNormalizer normalizer, Map<Symbol, List<Literal>> queryPartitionMap, WhereClause whereClause, TransactionContext transactionContext) throws UnsupportedOperationException {
    /*
         * Got multiple normalized queries which all could match.
         * This might be the case if one partition resolved to null
         *
         * e.g.
         *
         *  p = 1 and x = 2
         *
         * might lead to
         *
         *  null and x = 2
         *  true and x = 2
         *
         * At this point it is unknown if they really match.
         * In order to figure out if they could potentially match all conditions involving references are now set to true
         *
         *  null and true   -> can't match
         *  true and true   -> can match, can use this query + partition
         *
         * If there is still more than 1 query that can match it's not possible to execute the query :(
         */
    List<Tuple<Symbol, List<Literal>>> canMatch = new ArrayList<>();
    SymbolToTrueVisitor symbolToTrueVisitor = new SymbolToTrueVisitor();
    for (Map.Entry<Symbol, List<Literal>> entry : queryPartitionMap.entrySet()) {
        Symbol query = entry.getKey();
        List<Literal> partitions = entry.getValue();
        Symbol symbol = symbolToTrueVisitor.process(query, null);
        Symbol normalized = normalizer.normalize(symbol, transactionContext);
        assert normalized instanceof Literal : "after normalization and replacing all reference occurrences with true there must only be a literal left";
        Object value = ((Literal) normalized).value();
        if (value != null && (Boolean) value) {
            canMatch.add(new Tuple<>(query, partitions));
        }
    }
    if (canMatch.size() == 1) {
        Tuple<Symbol, List<Literal>> symbolListTuple = canMatch.get(0);
        WhereClause where = new WhereClause(symbolListTuple.v1(), whereClause.docKeys().orElse(null), new ArrayList<String>(symbolListTuple.v2().size()));
        where.partitions(symbolListTuple.v2());
        return where;
    }
    throw new UnsupportedOperationException("logical conjunction of the conditions in the WHERE clause which " + "involve partitioned columns led to a query that can't be executed.");
}
Also used : Symbol(io.crate.analyze.symbol.Symbol) WhereClause(io.crate.analyze.WhereClause) SymbolToTrueVisitor(io.crate.analyze.SymbolToTrueVisitor) Literal(io.crate.analyze.symbol.Literal) ImmutableList(com.google.common.collect.ImmutableList) Tuple(org.elasticsearch.common.collect.Tuple)

Example 3 with Tuple

use of org.elasticsearch.common.collect.Tuple in project elasticsearch by elastic.

the class InternalBinaryRangeTests method randomSortedRanges.

@Before
public void randomSortedRanges() {
    int numRanges = randomIntBetween(1, 10);
    Tuple<BytesRef, BytesRef>[] ranges = new Tuple[numRanges];
    for (int i = 0; i < numRanges; i++) {
        BytesRef[] values = new BytesRef[2];
        values[0] = new BytesRef(randomAsciiOfLength(15));
        values[1] = new BytesRef(randomAsciiOfLength(15));
        Arrays.sort(values);
        ranges[i] = new Tuple(values[0], values[1]);
    }
    Arrays.sort(ranges, (t1, t2) -> t1.v1().compareTo(t2.v1()));
    RANGES = ranges;
}
Also used : Tuple(org.elasticsearch.common.collect.Tuple) BytesRef(org.apache.lucene.util.BytesRef) Before(org.junit.Before)

Example 4 with Tuple

use of org.elasticsearch.common.collect.Tuple in project elasticsearch by elastic.

the class ParentToChildrenAggregatorTests method testParentChild.

public void testParentChild() throws IOException {
    Directory directory = newDirectory();
    RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory);
    final Map<String, Tuple<Integer, Integer>> expectedParentChildRelations = setupIndex(indexWriter);
    indexWriter.close();
    IndexReader indexReader = ElasticsearchDirectoryReader.wrap(DirectoryReader.open(directory), new ShardId(new Index("foo", "_na_"), 1));
    // TODO set "maybeWrap" to true for IndexSearcher once #23338 is resolved
    IndexSearcher indexSearcher = newSearcher(indexReader, false, true);
    testCase(new MatchAllDocsQuery(), indexSearcher, child -> {
        int expectedTotalChildren = 0;
        int expectedMinValue = Integer.MAX_VALUE;
        for (Tuple<Integer, Integer> expectedValues : expectedParentChildRelations.values()) {
            expectedTotalChildren += expectedValues.v1();
            expectedMinValue = Math.min(expectedMinValue, expectedValues.v2());
        }
        assertEquals(expectedTotalChildren, child.getDocCount());
        assertEquals(expectedMinValue, ((InternalMin) child.getAggregations().get("in_child")).getValue(), Double.MIN_VALUE);
    });
    for (String parent : expectedParentChildRelations.keySet()) {
        testCase(new TermInSetQuery(UidFieldMapper.NAME, new BytesRef(Uid.createUid(PARENT_TYPE, parent))), indexSearcher, child -> {
            assertEquals((long) expectedParentChildRelations.get(parent).v1(), child.getDocCount());
            assertEquals(expectedParentChildRelations.get(parent).v2(), ((InternalMin) child.getAggregations().get("in_child")).getValue(), Double.MIN_VALUE);
        });
    }
    indexReader.close();
    directory.close();
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) Index(org.elasticsearch.index.Index) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) ShardId(org.elasticsearch.index.shard.ShardId) TermInSetQuery(org.apache.lucene.search.TermInSetQuery) IndexReader(org.apache.lucene.index.IndexReader) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) Tuple(org.elasticsearch.common.collect.Tuple) BytesRef(org.apache.lucene.util.BytesRef) Directory(org.apache.lucene.store.Directory)

Example 5 with Tuple

use of org.elasticsearch.common.collect.Tuple in project elasticsearch by elastic.

the class Netty4HttpRequestSizeLimitIT method testLimitsInFlightRequests.

public void testLimitsInFlightRequests() throws Exception {
    ensureGreen();
    // we use the limit size as a (very) rough indication on how many requests we should sent to hit the limit
    int numRequests = LIMIT.bytesAsInt() / 100;
    StringBuilder bulkRequest = new StringBuilder();
    for (int i = 0; i < numRequests; i++) {
        bulkRequest.append("{\"index\": {}}");
        bulkRequest.append(System.lineSeparator());
        bulkRequest.append("{ \"field\" : \"value\" }");
        bulkRequest.append(System.lineSeparator());
    }
    @SuppressWarnings("unchecked") Tuple<String, CharSequence>[] requests = new Tuple[150];
    for (int i = 0; i < requests.length; i++) {
        requests[i] = Tuple.tuple("/index/type/_bulk", bulkRequest);
    }
    HttpServerTransport httpServerTransport = internalCluster().getInstance(HttpServerTransport.class);
    TransportAddress transportAddress = (TransportAddress) randomFrom(httpServerTransport.boundAddress().boundAddresses());
    try (Netty4HttpClient nettyHttpClient = new Netty4HttpClient()) {
        Collection<FullHttpResponse> singleResponse = nettyHttpClient.post(transportAddress.address(), requests[0]);
        assertThat(singleResponse, hasSize(1));
        assertAtLeastOnceExpectedStatus(singleResponse, HttpResponseStatus.OK);
        Collection<FullHttpResponse> multipleResponses = nettyHttpClient.post(transportAddress.address(), requests);
        assertThat(multipleResponses, hasSize(requests.length));
        assertAtLeastOnceExpectedStatus(multipleResponses, HttpResponseStatus.SERVICE_UNAVAILABLE);
    }
}
Also used : TransportAddress(org.elasticsearch.common.transport.TransportAddress) FullHttpResponse(io.netty.handler.codec.http.FullHttpResponse) HttpServerTransport(org.elasticsearch.http.HttpServerTransport) Tuple(org.elasticsearch.common.collect.Tuple)

Aggregations

Tuple (org.elasticsearch.common.collect.Tuple)50 IOException (java.io.IOException)18 ArrayList (java.util.ArrayList)17 HashMap (java.util.HashMap)12 List (java.util.List)12 Map (java.util.Map)12 Settings (org.elasticsearch.common.settings.Settings)8 CountDownLatch (java.util.concurrent.CountDownLatch)7 HashSet (java.util.HashSet)6 ParameterizedMessage (org.apache.logging.log4j.message.ParameterizedMessage)6 Matchers.containsString (org.hamcrest.Matchers.containsString)6 CopyOnWriteArrayList (java.util.concurrent.CopyOnWriteArrayList)5 Supplier (org.apache.logging.log4j.util.Supplier)5 TaskInfo (org.elasticsearch.tasks.TaskInfo)5 Set (java.util.Set)4 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 Consumer (java.util.function.Consumer)4 ElasticsearchException (org.elasticsearch.ElasticsearchException)4 Version (org.elasticsearch.Version)4 DiscoveryNode (org.elasticsearch.cluster.node.DiscoveryNode)4