Search in sources :

Example 11 with OperatorType

use of com.facebook.presto.common.function.OperatorType in project presto by prestodb.

the class AbstractGreatestLeast method specialize.

@Override
public BuiltInScalarFunctionImplementation specialize(BoundVariables boundVariables, int arity, FunctionAndTypeManager functionAndTypeManager) {
    Type type = boundVariables.getTypeVariable("E");
    checkArgument(type.isOrderable(), "Type must be orderable");
    MethodHandle compareMethod = functionAndTypeManager.getJavaScalarFunctionImplementation(functionAndTypeManager.resolveOperator(operatorType, fromTypes(type, type))).getMethodHandle();
    List<Class<?>> javaTypes = IntStream.range(0, arity).mapToObj(i -> type.getJavaType()).collect(toImmutableList());
    Class<?> clazz = generate(javaTypes, type, compareMethod);
    MethodHandle methodHandle = methodHandle(clazz, getSignature().getNameSuffix(), javaTypes.toArray(new Class<?>[javaTypes.size()]));
    return new BuiltInScalarFunctionImplementation(false, nCopies(javaTypes.size(), valueTypeArgumentProperty(RETURN_NULL_ON_NULL)), methodHandle);
}
Also used : FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) MethodDefinition(com.facebook.presto.bytecode.MethodDefinition) BytecodeUtils.invoke(com.facebook.presto.sql.gen.BytecodeUtils.invoke) Access.a(com.facebook.presto.bytecode.Access.a) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) CallSiteBinder(com.facebook.presto.bytecode.CallSiteBinder) ParameterizedType.type(com.facebook.presto.bytecode.ParameterizedType.type) Reflection.methodHandle(com.facebook.presto.util.Reflection.methodHandle) QualifiedObjectName(com.facebook.presto.common.QualifiedObjectName) INVALID_FUNCTION_ARGUMENT(com.facebook.presto.spi.StandardErrorCode.INVALID_FUNCTION_ARGUMENT) CompilerUtils.makeClassName(com.facebook.presto.util.CompilerUtils.makeClassName) Signature.orderableTypeParameter(com.facebook.presto.spi.function.Signature.orderableTypeParameter) UsedByGeneratedCode(com.facebook.presto.annotation.UsedByGeneratedCode) IfStatement(com.facebook.presto.bytecode.control.IfStatement) Variable(com.facebook.presto.bytecode.Variable) BoundVariables(com.facebook.presto.metadata.BoundVariables) Parameter(com.facebook.presto.bytecode.Parameter) Collections.nCopies(java.util.Collections.nCopies) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) String.format(java.lang.String.format) Collectors.joining(java.util.stream.Collectors.joining) SqlFunctionVisibility(com.facebook.presto.spi.function.SqlFunctionVisibility) ArgumentProperty.valueTypeArgumentProperty(com.facebook.presto.operator.scalar.ScalarFunctionImplementationChoice.ArgumentProperty.valueTypeArgumentProperty) ClassDefinition(com.facebook.presto.bytecode.ClassDefinition) List(java.util.List) FINAL(com.facebook.presto.bytecode.Access.FINAL) Scope(com.facebook.presto.bytecode.Scope) NOT_SUPPORTED(com.facebook.presto.spi.StandardErrorCode.NOT_SUPPORTED) Parameter.arg(com.facebook.presto.bytecode.Parameter.arg) IntStream(java.util.stream.IntStream) MethodHandle(java.lang.invoke.MethodHandle) StandardTypes(com.facebook.presto.common.type.StandardTypes) FunctionKind(com.facebook.presto.spi.function.FunctionKind) PRIVATE(com.facebook.presto.bytecode.Access.PRIVATE) PrestoException(com.facebook.presto.spi.PrestoException) TypeSignatureProvider.fromTypes(com.facebook.presto.sql.analyzer.TypeSignatureProvider.fromTypes) ImmutableList(com.google.common.collect.ImmutableList) Objects.requireNonNull(java.util.Objects.requireNonNull) BytecodeBlock(com.facebook.presto.bytecode.BytecodeBlock) RETURN_NULL_ON_NULL(com.facebook.presto.operator.scalar.ScalarFunctionImplementationChoice.NullConvention.RETURN_NULL_ON_NULL) Type(com.facebook.presto.common.type.Type) Failures.checkCondition(com.facebook.presto.util.Failures.checkCondition) PUBLIC(com.facebook.presto.bytecode.Access.PUBLIC) CompilerUtils.defineClass(com.facebook.presto.util.CompilerUtils.defineClass) SqlScalarFunction(com.facebook.presto.metadata.SqlScalarFunction) OperatorType(com.facebook.presto.common.function.OperatorType) STATIC(com.facebook.presto.bytecode.Access.STATIC) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) Signature(com.facebook.presto.spi.function.Signature) Type(com.facebook.presto.common.type.Type) OperatorType(com.facebook.presto.common.function.OperatorType) CompilerUtils.defineClass(com.facebook.presto.util.CompilerUtils.defineClass) MethodHandle(java.lang.invoke.MethodHandle)

Example 12 with OperatorType

use of com.facebook.presto.common.function.OperatorType in project presto by prestodb.

the class ExtractSpatialJoins method tryCreateSpatialJoin.

private static Result tryCreateSpatialJoin(Context context, JoinNode joinNode, RowExpression filter, PlanNodeId nodeId, List<VariableReferenceExpression> outputVariables, CallExpression spatialComparison, Metadata metadata, SplitManager splitManager, PageSourceManager pageSourceManager) {
    FunctionMetadata spatialComparisonMetadata = metadata.getFunctionAndTypeManager().getFunctionMetadata(spatialComparison.getFunctionHandle());
    checkArgument(spatialComparison.getArguments().size() == 2 && spatialComparisonMetadata.getOperatorType().isPresent());
    PlanNode leftNode = joinNode.getLeft();
    PlanNode rightNode = joinNode.getRight();
    List<VariableReferenceExpression> leftVariables = leftNode.getOutputVariables();
    List<VariableReferenceExpression> rightVariables = rightNode.getOutputVariables();
    RowExpression radius;
    Optional<VariableReferenceExpression> newRadiusVariable;
    CallExpression newComparison;
    if (spatialComparisonMetadata.getOperatorType().get() == OperatorType.LESS_THAN || spatialComparisonMetadata.getOperatorType().get() == OperatorType.LESS_THAN_OR_EQUAL) {
        // ST_Distance(a, b) <= r
        radius = spatialComparison.getArguments().get(1);
        Set<VariableReferenceExpression> radiusVariables = extractUnique(radius);
        if (radiusVariables.isEmpty() || (rightVariables.containsAll(radiusVariables) && containsNone(leftVariables, radiusVariables))) {
            newRadiusVariable = newRadiusVariable(context, radius);
            newComparison = new CallExpression(spatialComparison.getSourceLocation(), spatialComparison.getDisplayName(), spatialComparison.getFunctionHandle(), spatialComparison.getType(), ImmutableList.of(spatialComparison.getArguments().get(0), mapToExpression(newRadiusVariable, radius)));
        } else {
            return Result.empty();
        }
    } else {
        // r >= ST_Distance(a, b)
        radius = spatialComparison.getArguments().get(0);
        Set<VariableReferenceExpression> radiusVariables = extractUnique(radius);
        if (radiusVariables.isEmpty() || (rightVariables.containsAll(radiusVariables) && containsNone(leftVariables, radiusVariables))) {
            newRadiusVariable = newRadiusVariable(context, radius);
            OperatorType flippedOperatorType = flip(spatialComparisonMetadata.getOperatorType().get());
            FunctionHandle flippedHandle = getFlippedFunctionHandle(spatialComparison, metadata.getFunctionAndTypeManager());
            newComparison = new CallExpression(spatialComparison.getSourceLocation(), // TODO verify if this is the correct function displayName
            flippedOperatorType.getOperator(), flippedHandle, spatialComparison.getType(), ImmutableList.of(spatialComparison.getArguments().get(1), mapToExpression(newRadiusVariable, radius)));
        } else {
            return Result.empty();
        }
    }
    RowExpression newFilter = replaceExpression(filter, ImmutableMap.of(spatialComparison, newComparison));
    PlanNode newRightNode = newRadiusVariable.map(variable -> addProjection(context, rightNode, variable, radius)).orElse(rightNode);
    JoinNode newJoinNode = new JoinNode(joinNode.getSourceLocation(), joinNode.getId(), joinNode.getType(), leftNode, newRightNode, joinNode.getCriteria(), joinNode.getOutputVariables(), Optional.of(newFilter), joinNode.getLeftHashVariable(), joinNode.getRightHashVariable(), joinNode.getDistributionType(), joinNode.getDynamicFilters());
    return tryCreateSpatialJoin(context, newJoinNode, newFilter, nodeId, outputVariables, (CallExpression) newComparison.getArguments().get(0), Optional.of(newComparison.getArguments().get(1)), metadata, splitManager, pageSourceManager);
}
Also used : FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) WarningCollector(com.facebook.presto.spi.WarningCollector) Page(com.facebook.presto.common.Page) SpatialJoinNode(com.facebook.presto.sql.planner.plan.SpatialJoinNode) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) TypeSignature(com.facebook.presto.common.type.TypeSignature) MoreFutures.getFutureValue(com.facebook.airlift.concurrent.MoreFutures.getFutureValue) NOT_PARTITIONED(com.facebook.presto.spi.connector.NotPartitionedPartitionHandle.NOT_PARTITIONED) SpatialJoinUtils.getFlippedFunctionHandle(com.facebook.presto.util.SpatialJoinUtils.getFlippedFunctionHandle) Pattern(com.facebook.presto.matching.Pattern) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Capture(com.facebook.presto.matching.Capture) SpatialJoinUtils.flip(com.facebook.presto.util.SpatialJoinUtils.flip) Map(java.util.Map) UNGROUPED_SCHEDULING(com.facebook.presto.spi.connector.ConnectorSplitManager.SplitSchedulingStrategy.UNGROUPED_SCHEDULING) LOCAL(com.facebook.presto.spi.plan.ProjectNode.Locality.LOCAL) QualifiedObjectName(com.facebook.presto.common.QualifiedObjectName) SystemSessionProperties.isSpatialJoinEnabled(com.facebook.presto.SystemSessionProperties.isSpatialJoinEnabled) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice) CallExpression(com.facebook.presto.spi.relation.CallExpression) Splitter(com.google.common.base.Splitter) SplitSource(com.facebook.presto.split.SplitSource) Lifespan(com.facebook.presto.execution.Lifespan) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Collection(java.util.Collection) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) Set(java.util.Set) SplitManager(com.facebook.presto.split.SplitManager) String.format(java.lang.String.format) KDB_TREE(com.facebook.presto.common.type.KdbTreeType.KDB_TREE) UncheckedIOException(java.io.UncheckedIOException) FunctionMetadata(com.facebook.presto.spi.function.FunctionMetadata) List(java.util.List) SpatialJoinUtils.extractSupportedSpatialFunctions(com.facebook.presto.util.SpatialJoinUtils.extractSupportedSpatialFunctions) KdbTreeUtils(com.facebook.presto.geospatial.KdbTreeUtils) ProjectNode(com.facebook.presto.spi.plan.ProjectNode) Capture.newCapture(com.facebook.presto.matching.Capture.newCapture) INTEGER(com.facebook.presto.common.type.IntegerType.INTEGER) Optional(java.util.Optional) CAST(com.facebook.presto.metadata.CastType.CAST) VariablesExtractor.extractUnique(com.facebook.presto.sql.planner.VariablesExtractor.extractUnique) INNER(com.facebook.presto.sql.planner.plan.JoinNode.Type.INNER) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) Iterables(com.google.common.collect.Iterables) TableLayoutResult(com.facebook.presto.metadata.TableLayoutResult) VARCHAR(com.facebook.presto.common.type.VarcharType.VARCHAR) Captures(com.facebook.presto.matching.Captures) Assignments(com.facebook.presto.spi.plan.Assignments) Result(com.facebook.presto.sql.planner.iterative.Rule.Result) PrestoException(com.facebook.presto.spi.PrestoException) Patterns.join(com.facebook.presto.sql.planner.plan.Patterns.join) TypeSignatureProvider.fromTypes(com.facebook.presto.sql.analyzer.TypeSignatureProvider.fromTypes) Patterns.filter(com.facebook.presto.sql.planner.plan.Patterns.filter) FilterNode(com.facebook.presto.spi.plan.FilterNode) SplitBatch(com.facebook.presto.split.SplitSource.SplitBatch) RowExpressionNodeInliner(com.facebook.presto.expressions.RowExpressionNodeInliner) ImmutableList(com.google.common.collect.ImmutableList) PageSourceManager(com.facebook.presto.split.PageSourceManager) Verify.verify(com.google.common.base.Verify.verify) Objects.requireNonNull(java.util.Objects.requireNonNull) SpatialJoinUtils.extractSupportedSpatialComparisons(com.facebook.presto.util.SpatialJoinUtils.extractSupportedSpatialComparisons) ArrayType(com.facebook.presto.common.type.ArrayType) TableHandle(com.facebook.presto.spi.TableHandle) Expressions(com.facebook.presto.sql.relational.Expressions) KdbTree(com.facebook.presto.geospatial.KdbTree) UnnestNode(com.facebook.presto.sql.planner.plan.UnnestNode) Type(com.facebook.presto.common.type.Type) RowExpression(com.facebook.presto.spi.relation.RowExpression) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) INVALID_SPATIAL_PARTITIONING(com.facebook.presto.spi.StandardErrorCode.INVALID_SPATIAL_PARTITIONING) SystemSessionProperties.getSpatialPartitioningTableName(com.facebook.presto.SystemSessionProperties.getSpatialPartitioningTableName) Session(com.facebook.presto.Session) Rule(com.facebook.presto.sql.planner.iterative.Rule) Constraint(com.facebook.presto.spi.Constraint) IOException(java.io.IOException) OperatorType(com.facebook.presto.common.function.OperatorType) Patterns.source(com.facebook.presto.sql.planner.plan.Patterns.source) PlanNode(com.facebook.presto.spi.plan.PlanNode) ConnectorPageSource(com.facebook.presto.spi.ConnectorPageSource) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) LEFT(com.facebook.presto.sql.planner.plan.JoinNode.Type.LEFT) ColumnHandle(com.facebook.presto.spi.ColumnHandle) FunctionHandle(com.facebook.presto.spi.function.FunctionHandle) Split(com.facebook.presto.metadata.Split) Context(com.facebook.presto.sql.planner.iterative.Rule.Context) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Metadata(com.facebook.presto.metadata.Metadata) RowExpressionNodeInliner.replaceExpression(com.facebook.presto.expressions.RowExpressionNodeInliner.replaceExpression) FunctionMetadata(com.facebook.presto.spi.function.FunctionMetadata) PlanNode(com.facebook.presto.spi.plan.PlanNode) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) SpatialJoinNode(com.facebook.presto.sql.planner.plan.SpatialJoinNode) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) RowExpression(com.facebook.presto.spi.relation.RowExpression) CallExpression(com.facebook.presto.spi.relation.CallExpression) SpatialJoinUtils.getFlippedFunctionHandle(com.facebook.presto.util.SpatialJoinUtils.getFlippedFunctionHandle) FunctionHandle(com.facebook.presto.spi.function.FunctionHandle) OperatorType(com.facebook.presto.common.function.OperatorType)

Example 13 with OperatorType

use of com.facebook.presto.common.function.OperatorType in project presto by prestodb.

the class DruidFilterExpressionConverter method visitCall.

@Override
public DruidExpression visitCall(CallExpression call, Function<VariableReferenceExpression, Selection> context) {
    FunctionHandle functionHandle = call.getFunctionHandle();
    if (standardFunctionResolution.isNotFunction(functionHandle)) {
        return handleNot(call, context);
    }
    if (standardFunctionResolution.isCastFunction(functionHandle)) {
        return handleCast(call, context);
    }
    if (standardFunctionResolution.isBetweenFunction(functionHandle)) {
        return handleBetween(call, context);
    }
    FunctionMetadata functionMetadata = functionMetadataManager.getFunctionMetadata(call.getFunctionHandle());
    Optional<OperatorType> operatorTypeOptional = functionMetadata.getOperatorType();
    if (operatorTypeOptional.isPresent()) {
        OperatorType operatorType = operatorTypeOptional.get();
        if (operatorType.isArithmeticOperator()) {
            throw new PrestoException(DRUID_PUSHDOWN_UNSUPPORTED_EXPRESSION, "Arithmetic expressions are not supported in Druid filter: " + call);
        }
        if (operatorType.isComparisonOperator()) {
            return handleLogicalBinary(operatorType.getOperator(), call, context);
        }
    }
    throw new PrestoException(DRUID_PUSHDOWN_UNSUPPORTED_EXPRESSION, "Function " + call + " not supported in Druid filter");
}
Also used : FunctionMetadata(com.facebook.presto.spi.function.FunctionMetadata) PrestoException(com.facebook.presto.spi.PrestoException) FunctionHandle(com.facebook.presto.spi.function.FunctionHandle) OperatorType(com.facebook.presto.common.function.OperatorType)

Example 14 with OperatorType

use of com.facebook.presto.common.function.OperatorType in project presto by prestodb.

the class TestFunctionAndTypeManager method testOperatorTypes.

@Test
public void testOperatorTypes() {
    FunctionAndTypeManager functionAndTypeManager = createTestFunctionAndTypeManager();
    FunctionResolution functionResolution = new FunctionResolution(functionAndTypeManager);
    assertTrue(functionAndTypeManager.getFunctionMetadata(functionResolution.arithmeticFunction(ADD, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isArithmeticOperator).orElse(false));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.arithmeticFunction(ADD, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isComparisonOperator).orElse(true));
    assertTrue(functionAndTypeManager.getFunctionMetadata(functionResolution.comparisonFunction(GREATER_THAN, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isComparisonOperator).orElse(false));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.comparisonFunction(GREATER_THAN, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isArithmeticOperator).orElse(true));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.notFunction()).getOperatorType().isPresent());
}
Also used : FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) OperatorType.tryGetOperatorType(com.facebook.presto.common.function.OperatorType.tryGetOperatorType) OperatorType(com.facebook.presto.common.function.OperatorType) Test(org.testng.annotations.Test)

Example 15 with OperatorType

use of com.facebook.presto.common.function.OperatorType in project presto by prestodb.

the class TestFunctionAndTypeManager method testExactMatchBeforeCoercion.

@Test
public void testExactMatchBeforeCoercion() {
    FunctionAndTypeManager functionAndTypeManager = createTestFunctionAndTypeManager();
    boolean foundOperator = false;
    for (SqlFunction function : functionAndTypeManager.listOperators()) {
        OperatorType operatorType = tryGetOperatorType(function.getSignature().getName()).get();
        if (operatorType == CAST || operatorType == SATURATED_FLOOR_CAST) {
            continue;
        }
        if (!function.getSignature().getTypeVariableConstraints().isEmpty()) {
            continue;
        }
        if (function.getSignature().getArgumentTypes().stream().anyMatch(TypeSignature::isCalculated)) {
            continue;
        }
        BuiltInFunctionHandle exactOperator = (BuiltInFunctionHandle) functionAndTypeManager.resolveOperator(operatorType, fromTypeSignatures(function.getSignature().getArgumentTypes()));
        assertEquals(exactOperator.getSignature(), function.getSignature());
        foundOperator = true;
    }
    assertTrue(foundOperator);
}
Also used : TypeSignature(com.facebook.presto.common.type.TypeSignature) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) OperatorType.tryGetOperatorType(com.facebook.presto.common.function.OperatorType.tryGetOperatorType) OperatorType(com.facebook.presto.common.function.OperatorType) SqlFunction(com.facebook.presto.spi.function.SqlFunction) Test(org.testng.annotations.Test)

Aggregations

OperatorType (com.facebook.presto.common.function.OperatorType)18 FunctionMetadata (com.facebook.presto.spi.function.FunctionMetadata)9 Type (com.facebook.presto.common.type.Type)5 PrestoException (com.facebook.presto.spi.PrestoException)5 CallExpression (com.facebook.presto.spi.relation.CallExpression)5 ImmutableList (com.google.common.collect.ImmutableList)5 TypeSignature.parseTypeSignature (com.facebook.presto.common.type.TypeSignature.parseTypeSignature)4 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)4 MethodHandle (java.lang.invoke.MethodHandle)4 OperatorType.tryGetOperatorType (com.facebook.presto.common.function.OperatorType.tryGetOperatorType)3 TypeSignature (com.facebook.presto.common.type.TypeSignature)3 FunctionHandle (com.facebook.presto.spi.function.FunctionHandle)3 CallSiteBinder (com.facebook.presto.bytecode.CallSiteBinder)2 ClassDefinition (com.facebook.presto.bytecode.ClassDefinition)2 DynamicClassLoader (com.facebook.presto.bytecode.DynamicClassLoader)2 QualifiedObjectName (com.facebook.presto.common.QualifiedObjectName)2 ArrayType (com.facebook.presto.common.type.ArrayType)2 FunctionAndTypeManager (com.facebook.presto.metadata.FunctionAndTypeManager)2 FunctionAndTypeManager.createTestFunctionAndTypeManager (com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager)2 Signature (com.facebook.presto.spi.function.Signature)2