Search in sources :

Example 1 with RecordSetter

use of io.ordinate.engine.record.RecordSetter in project Mycat2 by MyCATApache.

the class DistinctPlan method execute.

@Override
public Observable<VectorSchemaRoot> execute(RootContext rootContext) {
    // DistinctContext distinctContext = new DistinctContext();
    IntInnerType[] intPairs = getIntTypes();
    Map map = MapFactory.createMap(intPairs);
    RecordSink recordSink = RecordSinkFactory.INSTANCE.buildRecordSink(intPairs);
    return input.execute(rootContext).flatMap(input -> {
        int rowCount = input.getRowCount();
        int columnCount = input.getFieldVectors().size();
        VectorSchemaRoot output = rootContext.getVectorSchemaRoot(schema(), rowCount);
        VectorBatchRecord record = new VectorBatchRecord(input);
        int outputRowId = 0;
        for (int rowId = 0; rowId < rowCount; rowId++) {
            record.setPosition(rowId);
            MapKey key = map.withKey();
            RecordSetter recordSinkSPI = RecordSinkFactory.INSTANCE.getRecordSinkSPI(key);
            recordSink.copy(record, recordSinkSPI);
            if (key.create()) {
                recordSink.copy(record, outputRowId, output);
                outputRowId++;
            // output
            } else {
            // skip
            }
        }
        if (outputRowId == 0) {
            output.close();
            return Observable.empty();
        }
        output.setRowCount(outputRowId);
        return Observable.fromArray(output);
    }).doOnComplete(new Action() {

        @Override
        public void run() throws Throwable {
            map.close();
        }
    });
}
Also used : MapFactory(io.ordinate.engine.structure.MapFactory) Schema(org.apache.arrow.vector.types.pojo.Schema) Logger(org.slf4j.Logger) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) LoggerFactory(org.slf4j.LoggerFactory) RecordSinkFactory(io.ordinate.engine.record.RecordSinkFactory) RecordSetter(io.ordinate.engine.record.RecordSetter) RootContext(io.ordinate.engine.record.RootContext) MapKey(io.questdb.cairo.map.MapKey) Action(io.reactivex.rxjava3.functions.Action) Map(io.questdb.cairo.map.Map) List(java.util.List) IntInnerType(io.ordinate.engine.schema.IntInnerType) Observable(io.reactivex.rxjava3.core.Observable) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord) Collections(java.util.Collections) RecordSink(io.ordinate.engine.record.RecordSink) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) RecordSetter(io.ordinate.engine.record.RecordSetter) Action(io.reactivex.rxjava3.functions.Action) MapKey(io.questdb.cairo.map.MapKey) RecordSink(io.ordinate.engine.record.RecordSink) Map(io.questdb.cairo.map.Map) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord) IntInnerType(io.ordinate.engine.schema.IntInnerType)

Example 2 with RecordSetter

use of io.ordinate.engine.record.RecordSetter in project Mycat2 by MyCATApache.

the class GroupByKeyPlan method execute.

@Override
public Observable<VectorSchemaRoot> execute(RootContext rootContext) {
    List<Field> fields = schema().getFields();
    InnerType[] innerTypes = schema().getFields().stream().map(i -> InnerType.from(i.getType())).toArray(n -> new InnerType[n]);
    Map map = MapFactory.createMap(innerTypes);
    RecordSink[] recordSinks = new RecordSink[groupByKeys.length];
    int groupIndex = 0;
    for (GroupKeys groupByKey : groupByKeys) {
        IntInnerType[] intPairs = new IntInnerType[groupByKey.getKeys().length];
        int[] keys = groupByKey.getKeys();
        int index = 0;
        for (int key : keys) {
            Field field = fields.get(key);
            intPairs[index] = IntInnerType.of(index, InnerType.from(field.getType()));
            index++;
        }
        recordSinks[groupIndex] = RecordSinkFactory.INSTANCE.buildRecordSink(intPairs);
        groupIndex++;
    }
    return inputPlan.execute(rootContext).flatMap(new Function<VectorSchemaRoot, ObservableSource<? extends VectorSchemaRoot>>() {

        @Override
        public ObservableSource<? extends VectorSchemaRoot> apply(VectorSchemaRoot input) throws Throwable {
            int rowCount = input.getRowCount();
            VectorBatchRecord record = new VectorBatchRecord(input);
            VectorSchemaRoot output = rootContext.getVectorSchemaRoot(schema, rowCount * recordSinks.length);
            int outputRowId = 0;
            for (int i = 0; i < recordSinks.length; i++) {
                RecordSink recordSink = recordSinks[i];
                for (int rowId = 0; rowId < rowCount; rowId++) {
                    record.setPosition(rowId);
                    MapKey key = map.withKey();
                    RecordSetter recordSinkSPI = RecordSinkFactory.INSTANCE.getRecordSinkSPI(key);
                    recordSink.copy(record, recordSinkSPI);
                    if (key.create()) {
                        recordSink.copy(record, outputRowId, output);
                        outputRowId++;
                    // output
                    } else {
                    // skip
                    }
                }
            }
            if (outputRowId == 0) {
                output.close();
                return Observable.empty();
            }
            output.setRowCount(outputRowId);
            inputPlan.eachFree(input);
            return Observable.fromArray(output);
        }
    }).doOnComplete(new Action() {

        @Override
        public void run() throws Throwable {
            map.close();
        }
    });
}
Also used : MapFactory(io.ordinate.engine.structure.MapFactory) ObservableSource(io.reactivex.rxjava3.core.ObservableSource) Schema(org.apache.arrow.vector.types.pojo.Schema) InnerType(io.ordinate.engine.schema.InnerType) Logger(org.slf4j.Logger) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) LoggerFactory(org.slf4j.LoggerFactory) RecordSinkFactory(io.ordinate.engine.record.RecordSinkFactory) RecordSetter(io.ordinate.engine.record.RecordSetter) Field(org.apache.arrow.vector.types.pojo.Field) RootContext(io.ordinate.engine.record.RootContext) MapKey(io.questdb.cairo.map.MapKey) Action(io.reactivex.rxjava3.functions.Action) Map(io.questdb.cairo.map.Map) GroupKeys(io.ordinate.engine.builder.GroupKeys) List(java.util.List) IntInnerType(io.ordinate.engine.schema.IntInnerType) Function(io.reactivex.rxjava3.functions.Function) Observable(io.reactivex.rxjava3.core.Observable) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord) Collections(java.util.Collections) RecordSink(io.ordinate.engine.record.RecordSink) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) RecordSetter(io.ordinate.engine.record.RecordSetter) Action(io.reactivex.rxjava3.functions.Action) InnerType(io.ordinate.engine.schema.InnerType) IntInnerType(io.ordinate.engine.schema.IntInnerType) IntInnerType(io.ordinate.engine.schema.IntInnerType) Field(org.apache.arrow.vector.types.pojo.Field) Function(io.reactivex.rxjava3.functions.Function) MapKey(io.questdb.cairo.map.MapKey) GroupKeys(io.ordinate.engine.builder.GroupKeys) RecordSink(io.ordinate.engine.record.RecordSink) Map(io.questdb.cairo.map.Map) VectorBatchRecord(io.ordinate.engine.record.VectorBatchRecord)

Aggregations

RecordSetter (io.ordinate.engine.record.RecordSetter)2 RecordSink (io.ordinate.engine.record.RecordSink)2 RecordSinkFactory (io.ordinate.engine.record.RecordSinkFactory)2 RootContext (io.ordinate.engine.record.RootContext)2 VectorBatchRecord (io.ordinate.engine.record.VectorBatchRecord)2 IntInnerType (io.ordinate.engine.schema.IntInnerType)2 MapFactory (io.ordinate.engine.structure.MapFactory)2 Map (io.questdb.cairo.map.Map)2 MapKey (io.questdb.cairo.map.MapKey)2 Observable (io.reactivex.rxjava3.core.Observable)2 Action (io.reactivex.rxjava3.functions.Action)2 Collections (java.util.Collections)2 List (java.util.List)2 VectorSchemaRoot (org.apache.arrow.vector.VectorSchemaRoot)2 Schema (org.apache.arrow.vector.types.pojo.Schema)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 GroupKeys (io.ordinate.engine.builder.GroupKeys)1 InnerType (io.ordinate.engine.schema.InnerType)1 ObservableSource (io.reactivex.rxjava3.core.ObservableSource)1