Search in sources :

Example 21 with ErrorCollectorImpl

use of org.apache.drill.common.expression.ErrorCollectorImpl in project drill by axbaretto.

the class ExpressionTreeMaterializerTest method testMaterializingLateboundField.

@Test
public void testMaterializingLateboundField() throws SchemaChangeException {
    final RecordBatch batch = mock(RecordBatch.class);
    when(batch.getValueVectorId(new SchemaPath("test", ExpressionPosition.UNKNOWN))).thenReturn(new TypedFieldId(Types.required(MinorType.BIGINT), -5));
    final SchemaBuilder builder = BatchSchema.newBuilder();
    builder.addField(getField("test", bigIntType));
    final BatchSchema schema = builder.build();
    ErrorCollector ec = new ErrorCollectorImpl();
    LogicalExpression expr = ExpressionTreeMaterializer.materialize(new FieldReference("test", ExpressionPosition.UNKNOWN), batch, ec, registry);
    assertEquals(bigIntType, expr.getMajorType());
    assertFalse(ec.hasErrors());
}
Also used : ErrorCollectorImpl(org.apache.drill.common.expression.ErrorCollectorImpl) LogicalExpression(org.apache.drill.common.expression.LogicalExpression) FieldReference(org.apache.drill.common.expression.FieldReference) SchemaPath(org.apache.drill.common.expression.SchemaPath) ErrorCollector(org.apache.drill.common.expression.ErrorCollector) ExecTest(org.apache.drill.exec.ExecTest) Test(org.junit.Test) VectorTest(org.apache.drill.categories.VectorTest)

Example 22 with ErrorCollectorImpl

use of org.apache.drill.common.expression.ErrorCollectorImpl in project drill by apache.

the class SortBatch method createNewSorter.

public static Sorter createNewSorter(FragmentContext context, List<Ordering> orderings, VectorAccessible batch, MappingSet mainMapping, MappingSet leftMapping, MappingSet rightMapping) {
    CodeGenerator<Sorter> cg = CodeGenerator.get(Sorter.TEMPLATE_DEFINITION, context.getOptions());
    // This operator may be deprecated. No tests exercise it.
    // There is no way, at present, to verify if the generated code
    // works with Plain-old Java.
    // cg.plainOldJavaCapable(true);
    // Uncomment out this line to debug the generated code.
    // cg.saveCodeForDebugging(true);
    ClassGenerator<Sorter> g = cg.getRoot();
    g.setMappingSet(mainMapping);
    for (Ordering od : orderings) {
        // first, we rewrite the evaluation stack for each side of the comparison.
        ErrorCollector collector = new ErrorCollectorImpl();
        final LogicalExpression expr = ExpressionTreeMaterializer.materialize(od.getExpr(), batch, collector, context.getFunctionRegistry());
        collector.reportErrors(logger);
        g.setMappingSet(leftMapping);
        HoldingContainer left = g.addExpr(expr, ClassGenerator.BlkCreateMode.FALSE);
        g.setMappingSet(rightMapping);
        HoldingContainer right = g.addExpr(expr, ClassGenerator.BlkCreateMode.FALSE);
        g.setMappingSet(mainMapping);
        // next we wrap the two comparison sides and add the expression block for the comparison.
        LogicalExpression fh = FunctionGenerationHelper.getOrderingComparator(od.nullsSortHigh(), left, right, context.getFunctionRegistry());
        HoldingContainer out = g.addExpr(fh, ClassGenerator.BlkCreateMode.FALSE);
        JConditional jc = g.getEvalBlock()._if(out.getValue().ne(JExpr.lit(0)));
        if (od.getDirection() == Direction.ASCENDING) {
            jc._then()._return(out.getValue());
        } else {
            jc._then()._return(out.getValue().minus());
        }
    }
    g.getEvalBlock()._return(JExpr.lit(0));
    return context.getImplementationClass(cg);
}
Also used : ErrorCollectorImpl(org.apache.drill.common.expression.ErrorCollectorImpl) LogicalExpression(org.apache.drill.common.expression.LogicalExpression) HoldingContainer(org.apache.drill.exec.expr.ClassGenerator.HoldingContainer) Ordering(org.apache.drill.common.logical.data.Order.Ordering) ErrorCollector(org.apache.drill.common.expression.ErrorCollector) JConditional(com.sun.codemodel.JConditional)

Example 23 with ErrorCollectorImpl

use of org.apache.drill.common.expression.ErrorCollectorImpl in project drill by apache.

the class ExpressionTreeMaterializer method materializeAndCheckErrors.

public static LogicalExpression materializeAndCheckErrors(LogicalExpression expr, VectorAccessible batch, FunctionLookupContext functionLookupContext) {
    ErrorCollector collector = new ErrorCollectorImpl();
    LogicalExpression e = ExpressionTreeMaterializer.materialize(expr, batch, collector, functionLookupContext, false, false);
    collector.reportErrors(logger);
    return e;
}
Also used : ErrorCollectorImpl(org.apache.drill.common.expression.ErrorCollectorImpl) LogicalExpression(org.apache.drill.common.expression.LogicalExpression) ErrorCollector(org.apache.drill.common.expression.ErrorCollector)

Example 24 with ErrorCollectorImpl

use of org.apache.drill.common.expression.ErrorCollectorImpl in project drill by apache.

the class MergeSortWrapper method createNewMSorter.

private MSorter createNewMSorter(List<Ordering> orderings, MappingSet mainMapping, MappingSet leftMapping, MappingSet rightMapping) {
    CodeGenerator<MSorter> cg = CodeGenerator.get(MSorter.TEMPLATE_DEFINITION, context.getFragmentContext().getOptions());
    cg.plainJavaCapable(true);
    // Uncomment out this line to debug the generated code.
    // cg.saveCodeForDebugging(true);
    ClassGenerator<MSorter> g = cg.getRoot();
    g.setMappingSet(mainMapping);
    for (Ordering od : orderings) {
        // first, we rewrite the evaluation stack for each side of the comparison.
        ErrorCollector collector = new ErrorCollectorImpl();
        final LogicalExpression expr = ExpressionTreeMaterializer.materialize(od.getExpr(), destContainer, collector, context.getFragmentContext().getFunctionRegistry());
        if (collector.hasErrors()) {
            throw UserException.unsupportedError().message("Failure while materializing expression. " + collector.toErrorString()).build(logger);
        }
        g.setMappingSet(leftMapping);
        HoldingContainer left = g.addExpr(expr, ClassGenerator.BlkCreateMode.FALSE);
        g.setMappingSet(rightMapping);
        HoldingContainer right = g.addExpr(expr, ClassGenerator.BlkCreateMode.FALSE);
        g.setMappingSet(mainMapping);
        // next we wrap the two comparison sides and add the expression block for the comparison.
        LogicalExpression fh = FunctionGenerationHelper.getOrderingComparator(od.nullsSortHigh(), left, right, context.getFragmentContext().getFunctionRegistry());
        HoldingContainer out = g.addExpr(fh, ClassGenerator.BlkCreateMode.FALSE);
        JConditional jc = g.getEvalBlock()._if(out.getValue().ne(JExpr.lit(0)));
        if (od.getDirection() == Direction.ASCENDING) {
            jc._then()._return(out.getValue());
        } else {
            jc._then()._return(out.getValue().minus());
        }
        g.rotateBlock();
    }
    g.rotateBlock();
    g.getEvalBlock()._return(JExpr.lit(0));
    return getInstance(cg, logger);
}
Also used : ErrorCollectorImpl(org.apache.drill.common.expression.ErrorCollectorImpl) LogicalExpression(org.apache.drill.common.expression.LogicalExpression) HoldingContainer(org.apache.drill.exec.expr.ClassGenerator.HoldingContainer) Ordering(org.apache.drill.common.logical.data.Order.Ordering) ErrorCollector(org.apache.drill.common.expression.ErrorCollector) JConditional(com.sun.codemodel.JConditional)

Example 25 with ErrorCollectorImpl

use of org.apache.drill.common.expression.ErrorCollectorImpl in project drill by apache.

the class FilterEvaluatorUtils method matches.

public static RowsMatch matches(LogicalExpression expr, Map<SchemaPath, ColumnStatistics<?>> columnsStatistics, TupleMetadata schema, long rowCount, UdfUtilities udfUtilities, FunctionLookupContext functionImplementationRegistry, Set<SchemaPath> schemaPathsInExpr) {
    ErrorCollector errorCollector = new ErrorCollectorImpl();
    LogicalExpression materializedFilter = ExpressionTreeMaterializer.materializeFilterExpr(expr, schema, errorCollector, functionImplementationRegistry);
    if (errorCollector.hasErrors()) {
        logger.error("{} error(s) encountered when materialize filter expression : {}", errorCollector.getErrorCount(), errorCollector.toErrorString());
        return RowsMatch.SOME;
    }
    Set<LogicalExpression> constantBoundaries = ConstantExpressionIdentifier.getConstantExpressionSet(materializedFilter);
    FilterPredicate<?> parquetPredicate = FilterBuilder.buildFilterPredicate(materializedFilter, constantBoundaries, udfUtilities, true);
    return matches(parquetPredicate, columnsStatistics, rowCount, schema, schemaPathsInExpr);
}
Also used : ErrorCollectorImpl(org.apache.drill.common.expression.ErrorCollectorImpl) LogicalExpression(org.apache.drill.common.expression.LogicalExpression) ErrorCollector(org.apache.drill.common.expression.ErrorCollector)

Aggregations

ErrorCollectorImpl (org.apache.drill.common.expression.ErrorCollectorImpl)78 LogicalExpression (org.apache.drill.common.expression.LogicalExpression)78 ErrorCollector (org.apache.drill.common.expression.ErrorCollector)74 SchemaChangeException (org.apache.drill.exec.exception.SchemaChangeException)36 HoldingContainer (org.apache.drill.exec.expr.ClassGenerator.HoldingContainer)25 Ordering (org.apache.drill.common.logical.data.Order.Ordering)23 TypedFieldId (org.apache.drill.exec.record.TypedFieldId)22 JConditional (com.sun.codemodel.JConditional)21 MaterializedField (org.apache.drill.exec.record.MaterializedField)19 ValueVector (org.apache.drill.exec.vector.ValueVector)18 SchemaPath (org.apache.drill.common.expression.SchemaPath)13 ValueVectorWriteExpression (org.apache.drill.exec.expr.ValueVectorWriteExpression)13 TransferPair (org.apache.drill.exec.record.TransferPair)13 IOException (java.io.IOException)11 NamedExpression (org.apache.drill.common.logical.data.NamedExpression)11 FieldReference (org.apache.drill.common.expression.FieldReference)10 ExecTest (org.apache.drill.exec.ExecTest)9 Test (org.junit.Test)9 ClassTransformationException (org.apache.drill.exec.exception.ClassTransformationException)8 IfExpression (org.apache.drill.common.expression.IfExpression)7