Search in sources :

Example 6 with PathSegment

use of org.apache.drill.common.expression.PathSegment in project drill by apache.

the class MaprDBJsonRecordReader method getFieldPathForProjection.

/*
   * Extracts contiguous named segments from the SchemaPath, starting from the
   * root segment and build the FieldPath from it for projection.
   *
   * This is due to bug 22726 and 22727, which cause DB's DocumentReaders to
   * behave incorrectly for sparse lists, hence we avoid projecting beyond the
   * first encountered ARRAY field and let Drill handle the projection.
   */
private static FieldPath getFieldPathForProjection(SchemaPath column) {
    Stack<PathSegment.NameSegment> pathSegments = new Stack<PathSegment.NameSegment>();
    PathSegment seg = column.getRootSegment();
    while (seg != null && seg.isNamed()) {
        pathSegments.push((PathSegment.NameSegment) seg);
        seg = seg.getChild();
    }
    FieldSegment.NameSegment child = null;
    while (!pathSegments.isEmpty()) {
        child = new FieldSegment.NameSegment(pathSegments.pop().getPath(), child, false);
    }
    return new FieldPath(child);
}
Also used : FieldSegment(org.ojai.FieldSegment) FieldPath(org.ojai.FieldPath) PathSegment(org.apache.drill.common.expression.PathSegment) Stack(java.util.Stack)

Example 7 with PathSegment

use of org.apache.drill.common.expression.PathSegment in project drill by apache.

the class PrelUtil method getColumns.

public static ProjectPushInfo getColumns(RelDataType rowType, List<RexNode> projects) {
    final List<String> fieldNames = rowType.getFieldNames();
    if (fieldNames.isEmpty()) {
        return null;
    }
    RefFieldsVisitor v = new RefFieldsVisitor(rowType);
    for (RexNode exp : projects) {
        PathSegment segment = exp.accept(v);
        v.addColumn(segment);
    }
    return v.getInfo();
}
Also used : PathSegment(org.apache.drill.common.expression.PathSegment) RexNode(org.apache.calcite.rex.RexNode)

Example 8 with PathSegment

use of org.apache.drill.common.expression.PathSegment in project drill by apache.

the class HBaseRecordReader method transformColumns.

@Override
protected Collection<SchemaPath> transformColumns(Collection<SchemaPath> columns) {
    Set<SchemaPath> transformed = Sets.newLinkedHashSet();
    rowKeyOnly = true;
    if (!isStarQuery()) {
        for (SchemaPath column : columns) {
            if (column.getRootSegment().getPath().equalsIgnoreCase(ROW_KEY)) {
                transformed.add(ROW_KEY_PATH);
                continue;
            }
            rowKeyOnly = false;
            NameSegment root = column.getRootSegment();
            byte[] family = root.getPath().getBytes();
            transformed.add(SchemaPath.getSimplePath(root.getPath()));
            PathSegment child = root.getChild();
            if (child != null && child.isNamed()) {
                byte[] qualifier = child.getNameSegment().getPath().getBytes();
                hbaseScan.addColumn(family, qualifier);
            } else {
                hbaseScan.addFamily(family);
            }
        }
        /* if only the row key was requested, add a FirstKeyOnlyFilter to the scan
       * to fetch only one KV from each row. If a filter is already part of this
       * scan, add the FirstKeyOnlyFilter as the LAST filter of a MUST_PASS_ALL
       * FilterList.
       */
        if (rowKeyOnly) {
            hbaseScan.setFilter(HBaseUtils.andFilterAtIndex(hbaseScan.getFilter(), HBaseUtils.LAST_FILTER, new FirstKeyOnlyFilter()));
        }
    } else {
        rowKeyOnly = false;
        transformed.add(ROW_KEY_PATH);
    }
    return transformed;
}
Also used : NameSegment(org.apache.drill.common.expression.PathSegment.NameSegment) SchemaPath(org.apache.drill.common.expression.SchemaPath) FirstKeyOnlyFilter(org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter) PathSegment(org.apache.drill.common.expression.PathSegment)

Aggregations

PathSegment (org.apache.drill.common.expression.PathSegment)8 SchemaPath (org.apache.drill.common.expression.SchemaPath)4 MajorType (org.apache.drill.common.types.TypeProtos.MajorType)2 BaseWriter (org.apache.drill.exec.vector.complex.writer.BaseWriter)2 BitSet (java.util.BitSet)1 Stack (java.util.Stack)1 RexNode (org.apache.calcite.rex.RexNode)1 NameSegment (org.apache.drill.common.expression.PathSegment.NameSegment)1 MinorType (org.apache.drill.common.types.TypeProtos.MinorType)1 TypedFieldId (org.apache.drill.exec.record.TypedFieldId)1 ValueVector (org.apache.drill.exec.vector.ValueVector)1 ListWriter (org.apache.drill.exec.vector.complex.writer.BaseWriter.ListWriter)1 MapWriter (org.apache.drill.exec.vector.complex.writer.BaseWriter.MapWriter)1 FirstKeyOnlyFilter (org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter)1 ColumnDescriptor (org.apache.parquet.column.ColumnDescriptor)1 GroupType (org.apache.parquet.schema.GroupType)1 MessageType (org.apache.parquet.schema.MessageType)1 Type (org.apache.parquet.schema.Type)1 FieldPath (org.ojai.FieldPath)1 FieldSegment (org.ojai.FieldSegment)1