Search in sources :

Example 1 with GroupKeys

use of io.ordinate.engine.builder.GroupKeys in project Mycat2 by MyCATApache.

the class GroupByKeyWithAggPlan method buildRecordSink.

@NotNull
private RecordSink[] buildRecordSink(List<Field> fields) {
    RecordSink[] recordSinks = new RecordSink[groupByKeys.length];
    int groupIndex = 0;
    for (GroupKeys groupByKey : groupByKeys) {
        IntInnerType[] intPairs = new IntInnerType[groupByKey.getKeys().length];
        int[] keys = groupByKey.getKeys();
        int index = 0;
        for (int key : keys) {
            Field field = fields.get(key);
            intPairs[index] = IntInnerType.of(index, InnerType.from(field.getType()));
            index++;
        }
        recordSinks[groupIndex] = RecordSinkFactory.INSTANCE.buildRecordSink(intPairs);
        groupIndex++;
    }
    return recordSinks;
}
Also used : Field(org.apache.arrow.vector.types.pojo.Field) GroupKeys(io.ordinate.engine.builder.GroupKeys) IntInnerType(io.ordinate.engine.schema.IntInnerType) NotNull(org.jetbrains.annotations.NotNull)

Example 2 with GroupKeys

use of io.ordinate.engine.builder.GroupKeys in project Mycat2 by MyCATApache.

the class GroupByKeyPlan method execute.

@Override
public Observable<VectorSchemaRoot> execute(RootContext rootContext) {
    List<Field> fields = schema().getFields();
    InnerType[] innerTypes = schema().getFields().stream().map(i -> InnerType.from(i.getType())).toArray(n -> new InnerType[n]);
    Map map = MapFactory.createMap(innerTypes);
    RecordSink[] recordSinks = new RecordSink[groupByKeys.length];
    int groupIndex = 0;
    for (GroupKeys groupByKey : groupByKeys) {
        IntInnerType[] intPairs = new IntInnerType[groupByKey.getKeys().length];
        int[] keys = groupByKey.getKeys();
        int index = 0;
        for (int key : keys) {
            Field field = fields.get(key);
            intPairs[index] = IntInnerType.of(index, InnerType.from(field.getType()));
            index++;
        }
        recordSinks[groupIndex] = RecordSinkFactory.INSTANCE.buildRecordSink(intPairs);
        groupIndex++;
    }
    return inputPlan.execute(rootContext).flatMap(new Function<VectorSchemaRoot, ObservableSource<? extends VectorSchemaRoot>>() {

        @Override
        public ObservableSource<? extends VectorSchemaRoot> apply(VectorSchemaRoot input) throws Throwable {
            int rowCount = input.getRowCount();
            VectorBatchRecord record = new VectorBatchRecord(input);
            VectorSchemaRoot output = rootContext.getVectorSchemaRoot(schema, rowCount * recordSinks.length);
            int outputRowId = 0;
            for (int i = 0; i < recordSinks.length; i++) {
                RecordSink recordSink = recordSinks[i];
                for (int rowId = 0; rowId < rowCount; rowId++) {
                    record.setPosition(rowId);
                    MapKey key = map.withKey();
                    RecordSetter recordSinkSPI = RecordSinkFactory.INSTANCE.getRecordSinkSPI(key);
                    recordSink.copy(record, recordSinkSPI);
                    if (key.create()) {
                        recordSink.copy(record, outputRowId, output);
                        outputRowId++;
                    // output
                    } else {
                    // skip
                    }
                }
            }
            if (outputRowId == 0) {
                output.close();
                return Observable.empty();
            }
            output.setRowCount(outputRowId);
            inputPlan.eachFree(input);
            return Observable.fromArray(output);
        }
    }).doOnComplete(new Action() {

        @Override
        public void run() throws Throwable {
            map.close();
        }
    });
}
Also used : MapFactory(io.ordinate.engine.structure.MapFactory) ObservableSource(io.reactivex.rxjava3.core.ObservableSource) Schema(org.apache.arrow.vector.types.pojo.Schema) InnerType(io.ordinate.engine.schema.InnerType) Logger(org.slf4j.Logger) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) LoggerFactory(org.slf4j.LoggerFactory) RecordSinkFactory(io.ordinate.engine.record.RecordSinkFactory) RecordSetter(io.ordinate.engine.record.RecordSetter) Field(org.apache.arrow.vector.types.pojo.Field) RootContext(io.ordinate.engine.record.RootContext) MapKey(io.questdb.cairo.map.MapKey) Action(io.reactivex.rxjava3.functions.Action) Map(io.questdb.cairo.map.Map) GroupKeys(io.ordinate.engine.builder.GroupKeys) List(java.util.List) IntInnerType(io.ordinate.engine.schema.IntInnerType) Function(io.reactivex.rxjava3.functions.Function) Observable(io.reactivex.rxjava3.core.Observable) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord) Collections(java.util.Collections) RecordSink(io.ordinate.engine.record.RecordSink) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) RecordSetter(io.ordinate.engine.record.RecordSetter) Action(io.reactivex.rxjava3.functions.Action) InnerType(io.ordinate.engine.schema.InnerType) IntInnerType(io.ordinate.engine.schema.IntInnerType) IntInnerType(io.ordinate.engine.schema.IntInnerType) Field(org.apache.arrow.vector.types.pojo.Field) Function(io.reactivex.rxjava3.functions.Function) MapKey(io.questdb.cairo.map.MapKey) GroupKeys(io.ordinate.engine.builder.GroupKeys) RecordSink(io.ordinate.engine.record.RecordSink) Map(io.questdb.cairo.map.Map) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord)

Aggregations

GroupKeys (io.ordinate.engine.builder.GroupKeys)2 IntInnerType (io.ordinate.engine.schema.IntInnerType)2 Field (org.apache.arrow.vector.types.pojo.Field)2 RecordSetter (io.ordinate.engine.record.RecordSetter)1 RecordSink (io.ordinate.engine.record.RecordSink)1 RecordSinkFactory (io.ordinate.engine.record.RecordSinkFactory)1 RootContext (io.ordinate.engine.record.RootContext)1 VectorBatchRecord (io.ordinate.engine.record.VectorBatchRecord)1 InnerType (io.ordinate.engine.schema.InnerType)1 MapFactory (io.ordinate.engine.structure.MapFactory)1 Map (io.questdb.cairo.map.Map)1 MapKey (io.questdb.cairo.map.MapKey)1 Observable (io.reactivex.rxjava3.core.Observable)1 ObservableSource (io.reactivex.rxjava3.core.ObservableSource)1 Action (io.reactivex.rxjava3.functions.Action)1 Function (io.reactivex.rxjava3.functions.Function)1 Collections (java.util.Collections)1 List (java.util.List)1 VectorSchemaRoot (org.apache.arrow.vector.VectorSchemaRoot)1 Schema (org.apache.arrow.vector.types.pojo.Schema)1