Search in sources :

Example 1 with ColumnStatCollector

use of org.apache.drill.exec.store.parquet.stat.ColumnStatCollector in project drill by apache.

the class ParquetRGFilterEvaluator method evalFilter.

public static boolean evalFilter(LogicalExpression expr, ParquetMetadata footer, int rowGroupIndex, OptionManager options, FragmentContext fragmentContext, Map<String, String> implicitColValues) {
    // figure out the set of columns referenced in expression.
    final Set<SchemaPath> schemaPathsInExpr = expr.accept(new FieldReferenceFinder(), null);
    final ColumnStatCollector columnStatCollector = new ParquetFooterStatCollector(footer, rowGroupIndex, implicitColValues, true, options);
    Map<SchemaPath, ColumnStatistics> columnStatisticsMap = columnStatCollector.collectColStat(schemaPathsInExpr);
    boolean canDrop = canDrop(expr, columnStatisticsMap, footer.getBlocks().get(rowGroupIndex).getRowCount(), fragmentContext, fragmentContext.getFunctionRegistry());
    return canDrop;
}
Also used : ColumnStatistics(org.apache.drill.exec.store.parquet.stat.ColumnStatistics) SchemaPath(org.apache.drill.common.expression.SchemaPath) ColumnStatCollector(org.apache.drill.exec.store.parquet.stat.ColumnStatCollector) ParquetFooterStatCollector(org.apache.drill.exec.store.parquet.stat.ParquetFooterStatCollector)

Example 2 with ColumnStatCollector

use of org.apache.drill.exec.store.parquet.stat.ColumnStatCollector in project drill by axbaretto.

the class ParquetRGFilterEvaluator method evalFilter.

public static boolean evalFilter(LogicalExpression expr, ParquetMetadata footer, int rowGroupIndex, OptionManager options, FragmentContext fragmentContext, Map<String, String> implicitColValues) {
    // figure out the set of columns referenced in expression.
    final Set<SchemaPath> schemaPathsInExpr = expr.accept(new FieldReferenceFinder(), null);
    final ColumnStatCollector columnStatCollector = new ParquetFooterStatCollector(footer, rowGroupIndex, implicitColValues, true, options);
    Map<SchemaPath, ColumnStatistics> columnStatisticsMap = columnStatCollector.collectColStat(schemaPathsInExpr);
    boolean canDrop = canDrop(expr, columnStatisticsMap, footer.getBlocks().get(rowGroupIndex).getRowCount(), fragmentContext, fragmentContext.getFunctionRegistry());
    return canDrop;
}
Also used : ColumnStatistics(org.apache.drill.exec.store.parquet.stat.ColumnStatistics) SchemaPath(org.apache.drill.common.expression.SchemaPath) ColumnStatCollector(org.apache.drill.exec.store.parquet.stat.ColumnStatCollector) ParquetFooterStatCollector(org.apache.drill.exec.store.parquet.stat.ParquetFooterStatCollector)

Aggregations

SchemaPath (org.apache.drill.common.expression.SchemaPath)2 ColumnStatCollector (org.apache.drill.exec.store.parquet.stat.ColumnStatCollector)2 ColumnStatistics (org.apache.drill.exec.store.parquet.stat.ColumnStatistics)2 ParquetFooterStatCollector (org.apache.drill.exec.store.parquet.stat.ParquetFooterStatCollector)2