Search in sources :

Example 1 with VariableWidthVector

use of org.apache.drill.exec.vector.VariableWidthVector in project drill by apache.

the class BatchValidator method validateRepeatedVector.

private void validateRepeatedVector(String name, BaseRepeatedValueVector vector) {
    int dataLength = Integer.MAX_VALUE;
    if (vector instanceof RepeatedVarCharVector) {
        dataLength = ((RepeatedVarCharVector) vector).getOffsetVector().getValueCapacity();
    } else if (vector instanceof RepeatedFixedWidthVectorLike) {
        dataLength = ((BaseDataValueVector) ((BaseRepeatedValueVector) vector).getDataVector()).getBuffer().capacity();
    }
    int itemCount = validateOffsetVector(name + "-offsets", vector.getOffsetVector(), rowCount, dataLength);
    // Special handling of repeated VarChar vectors
    // The nested data vectors are not quite exactly like top-level vectors.
    @SuppressWarnings("resource") ValueVector dataVector = vector.getDataVector();
    if (dataVector instanceof VariableWidthVector) {
        validateVariableWidthVector(name + "-data", (VariableWidthVector) dataVector, itemCount);
    }
}
Also used : ValueVector(org.apache.drill.exec.vector.ValueVector) BaseRepeatedValueVector(org.apache.drill.exec.vector.complex.BaseRepeatedValueVector) BaseDataValueVector(org.apache.drill.exec.vector.BaseDataValueVector) RepeatedVarCharVector(org.apache.drill.exec.vector.RepeatedVarCharVector) RepeatedFixedWidthVectorLike(org.apache.drill.exec.vector.complex.RepeatedFixedWidthVectorLike) BaseRepeatedValueVector(org.apache.drill.exec.vector.complex.BaseRepeatedValueVector) VariableWidthVector(org.apache.drill.exec.vector.VariableWidthVector)

Example 2 with VariableWidthVector

use of org.apache.drill.exec.vector.VariableWidthVector in project drill by apache.

the class DrillParquetReader method getPercentFilled.

private int getPercentFilled() {
    int filled = 0;
    for (final ValueVector v : primitiveVectors) {
        filled = Math.max(filled, v.getAccessor().getValueCount() * 100 / v.getValueCapacity());
        if (v instanceof VariableWidthVector) {
            filled = Math.max(filled, ((VariableWidthVector) v).getCurrentSizeInBytes() * 100 / ((VariableWidthVector) v).getByteCapacity());
        }
    // TODO - need to re-enable this
    //      if (v instanceof RepeatedFixedWidthVector) {
    //        filled = Math.max(filled, ((RepeatedFixedWidthVector) v).getAccessor().getGroupCount() * 100)
    //      }
    }
    logger.debug("Percent filled: {}", filled);
    return filled;
}
Also used : ValueVector(org.apache.drill.exec.vector.ValueVector) VariableWidthVector(org.apache.drill.exec.vector.VariableWidthVector)

Aggregations

ValueVector (org.apache.drill.exec.vector.ValueVector)2 VariableWidthVector (org.apache.drill.exec.vector.VariableWidthVector)2 BaseDataValueVector (org.apache.drill.exec.vector.BaseDataValueVector)1 RepeatedVarCharVector (org.apache.drill.exec.vector.RepeatedVarCharVector)1 BaseRepeatedValueVector (org.apache.drill.exec.vector.complex.BaseRepeatedValueVector)1 RepeatedFixedWidthVectorLike (org.apache.drill.exec.vector.complex.RepeatedFixedWidthVectorLike)1