Search in sources :

Example 1 with FetchProjection

use of io.crate.execution.dsl.projection.FetchProjection in project crate by crate.

the class FetchProjector method create.

public static Projector create(FetchProjection projection, RamAccounting ramAccounting, LongSupplier getBucketsBytesThreshold, TransactionContext txnCtx, NodeContext nodeCtx, FetchOperation fetchOperation) {
    final FetchRows fetchRows = FetchRows.create(txnCtx, nodeCtx, projection.fetchSources(), projection.outputSymbols());
    EstimateCellsSize estimateRowSize = new EstimateCellsSize(projection.inputTypes());
    return (BatchIterator<Row> source) -> {
        final long maxBucketsSizeInBytes = getBucketsBytesThreshold.getAsLong();
        BatchIterator<ReaderBuckets> buckets = BatchIterators.partition(source, projection.getFetchSize(), () -> new ReaderBuckets(fetchRows, projection::getFetchSourceByReader, estimateRowSize, ramAccounting), ReaderBuckets::add, readerBuckets -> readerBuckets.ramBytesUsed() > maxBucketsSizeInBytes);
        return new AsyncFlatMapBatchIterator<>(buckets, new FetchMapper(fetchOperation, projection.nodeReaders()));
    };
}
Also used : TransactionContext(io.crate.metadata.TransactionContext) ByteSizeUnit(org.elasticsearch.common.unit.ByteSizeUnit) NodeContext(io.crate.metadata.NodeContext) LongSupplier(java.util.function.LongSupplier) BatchIterator(io.crate.data.BatchIterator) RamAccounting(io.crate.breaker.RamAccounting) Projector(io.crate.data.Projector) EstimateCellsSize(io.crate.breaker.EstimateCellsSize) BatchIterators(io.crate.data.BatchIterators) Row(io.crate.data.Row) AsyncFlatMapBatchIterator(io.crate.data.AsyncFlatMapBatchIterator) CircuitBreaker(org.elasticsearch.common.breaker.CircuitBreaker) FetchProjection(io.crate.execution.dsl.projection.FetchProjection) EstimateCellsSize(io.crate.breaker.EstimateCellsSize) BatchIterator(io.crate.data.BatchIterator) AsyncFlatMapBatchIterator(io.crate.data.AsyncFlatMapBatchIterator) Row(io.crate.data.Row)

Example 2 with FetchProjection

use of io.crate.execution.dsl.projection.FetchProjection in project crate by crate.

the class Fetch method build.

@Override
public ExecutionPlan build(PlannerContext plannerContext, Set<PlanHint> hints, ProjectionBuilder projectionBuilder, int limit, int offset, @Nullable OrderBy order, @Nullable Integer pageSizeHint, Row params, SubQueryResults subQueryResults) {
    plannerContext.newReaderAllocations();
    var executionPlan = Merge.ensureOnHandler(source.build(plannerContext, hints, projectionBuilder, limit, offset, order, pageSizeHint, params, subQueryResults), plannerContext);
    ReaderAllocations readerAllocations = plannerContext.buildReaderAllocations();
    Function<Symbol, Symbol> paramBinder = new SubQueryAndParamBinder(params, subQueryResults);
    FetchPhase fetchPhase = new FetchPhase(plannerContext.nextExecutionPhaseId(), readerAllocations.nodeReaders().keySet(), readerAllocations.bases(), readerAllocations.tableIndices(), fetchRefs);
    ArrayList<Symbol> boundOutputs = new ArrayList<>(replacedOutputs.size());
    for (var entry : replacedOutputs.entrySet()) {
        Symbol key = entry.getKey();
        Symbol value = entry.getValue();
        if (source.outputs().contains(key)) {
            boundOutputs.add(paramBinder.apply(key));
        } else {
            boundOutputs.add(paramBinder.apply(value));
        }
    }
    List<DataType<?>> inputTypes = Symbols.typeView(source.outputs());
    List<Symbol> fetchOutputs = InputColumns.create(boundOutputs, new InputColumns.SourceSymbols(source.outputs()));
    FetchProjection fetchProjection = new FetchProjection(fetchPhase.phaseId(), plannerContext.fetchSize(), fetchSourceByRelation, fetchOutputs, inputTypes, readerAllocations.nodeReaders(), readerAllocations.indices(), readerAllocations.indicesToIdents());
    executionPlan.addProjection(fetchProjection);
    return new QueryThenFetch(executionPlan, fetchPhase);
}
Also used : InputColumns(io.crate.execution.dsl.projection.builder.InputColumns) QueryThenFetch(io.crate.planner.node.dql.QueryThenFetch) Symbol(io.crate.expression.symbol.Symbol) ArrayList(java.util.ArrayList) FetchProjection(io.crate.execution.dsl.projection.FetchProjection) ReaderAllocations(io.crate.planner.ReaderAllocations) FetchPhase(io.crate.execution.dsl.phases.FetchPhase) DataType(io.crate.types.DataType)

Aggregations

FetchProjection (io.crate.execution.dsl.projection.FetchProjection)2 EstimateCellsSize (io.crate.breaker.EstimateCellsSize)1 RamAccounting (io.crate.breaker.RamAccounting)1 AsyncFlatMapBatchIterator (io.crate.data.AsyncFlatMapBatchIterator)1 BatchIterator (io.crate.data.BatchIterator)1 BatchIterators (io.crate.data.BatchIterators)1 Projector (io.crate.data.Projector)1 Row (io.crate.data.Row)1 FetchPhase (io.crate.execution.dsl.phases.FetchPhase)1 InputColumns (io.crate.execution.dsl.projection.builder.InputColumns)1 Symbol (io.crate.expression.symbol.Symbol)1 NodeContext (io.crate.metadata.NodeContext)1 TransactionContext (io.crate.metadata.TransactionContext)1 ReaderAllocations (io.crate.planner.ReaderAllocations)1 QueryThenFetch (io.crate.planner.node.dql.QueryThenFetch)1 DataType (io.crate.types.DataType)1 ArrayList (java.util.ArrayList)1 LongSupplier (java.util.function.LongSupplier)1 CircuitBreaker (org.elasticsearch.common.breaker.CircuitBreaker)1 ByteSizeUnit (org.elasticsearch.common.unit.ByteSizeUnit)1