Search in sources :

Example 6 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class Histogram method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type keyType, Type valueType) {
    DynamicClassLoader classLoader = new DynamicClassLoader(Histogram.class.getClassLoader());
    List<Type> inputTypes = ImmutableList.of(keyType);
    Type outputType = new MapType(keyType, valueType);
    HistogramStateSerializer stateSerializer = new HistogramStateSerializer(keyType);
    Type intermediateType = stateSerializer.getSerializedType();
    MethodHandle inputFunction = INPUT_FUNCTION.bindTo(keyType);
    MethodHandle outputFunction = OUTPUT_FUNCTION.bindTo(outputType);
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, outputType.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), createInputParameterMetadata(keyType), inputFunction, COMBINE_FUNCTION, outputFunction, HistogramState.class, stateSerializer, new HistogramStateFactory(), outputType);
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, outputType, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) MapType(com.facebook.presto.type.MapType) Type(com.facebook.presto.spi.type.Type) BigintType(com.facebook.presto.spi.type.BigintType) HistogramStateFactory(com.facebook.presto.operator.aggregation.state.HistogramStateFactory) MapType(com.facebook.presto.type.MapType) HistogramStateSerializer(com.facebook.presto.operator.aggregation.state.HistogramStateSerializer) MethodHandle(java.lang.invoke.MethodHandle)

Example 7 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class MapUnionAggregation method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type keyType, Type valueType) {
    DynamicClassLoader classLoader = new DynamicClassLoader(MapUnionAggregation.class.getClassLoader());
    Type outputType = new MapType(keyType, valueType);
    List<Type> inputTypes = ImmutableList.of(outputType);
    KeyValuePairStateSerializer stateSerializer = new KeyValuePairStateSerializer(keyType, valueType);
    Type intermediateType = stateSerializer.getSerializedType();
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, outputType.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), createInputParameterMetadata(outputType), INPUT_FUNCTION.bindTo(keyType).bindTo(valueType), COMBINE_FUNCTION, OUTPUT_FUNCTION, KeyValuePairsState.class, stateSerializer, new KeyValuePairsStateFactory(keyType, valueType), outputType);
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, outputType, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) MapType(com.facebook.presto.type.MapType) Type(com.facebook.presto.spi.type.Type) KeyValuePairStateSerializer(com.facebook.presto.operator.aggregation.state.KeyValuePairStateSerializer) KeyValuePairsStateFactory(com.facebook.presto.operator.aggregation.state.KeyValuePairsStateFactory) MapType(com.facebook.presto.type.MapType)

Example 8 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class ArrayAggregationFunction method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type type, boolean legacyArrayAgg) {
    DynamicClassLoader classLoader = new DynamicClassLoader(ArrayAggregationFunction.class.getClassLoader());
    AccumulatorStateSerializer<?> stateSerializer = new ArrayAggregationStateSerializer(type);
    AccumulatorStateFactory<?> stateFactory = new ArrayAggregationStateFactory();
    List<Type> inputTypes = ImmutableList.of(type);
    Type outputType = new ArrayType(type);
    Type intermediateType = stateSerializer.getSerializedType();
    List<ParameterMetadata> inputParameterMetadata = createInputParameterMetadata(type, legacyArrayAgg);
    MethodHandle inputFunction = INPUT_FUNCTION.bindTo(type);
    MethodHandle combineFunction = COMBINE_FUNCTION.bindTo(type);
    MethodHandle outputFunction = OUTPUT_FUNCTION.bindTo(outputType);
    Class<? extends AccumulatorState> stateInterface = ArrayAggregationState.class;
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, type.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), inputParameterMetadata, inputFunction, combineFunction, outputFunction, stateInterface, stateSerializer, stateFactory, outputType);
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, outputType, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) ArrayAggregationStateSerializer(com.facebook.presto.operator.aggregation.state.ArrayAggregationStateSerializer) ParameterMetadata(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata) ArrayType(com.facebook.presto.type.ArrayType) ArrayType(com.facebook.presto.type.ArrayType) Type(com.facebook.presto.spi.type.Type) ArrayAggregationState(com.facebook.presto.operator.aggregation.state.ArrayAggregationState) ArrayAggregationStateFactory(com.facebook.presto.operator.aggregation.state.ArrayAggregationStateFactory) MethodHandle(java.lang.invoke.MethodHandle)

Example 9 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class ConcatFunction method generateConcat.

private static Class<?> generateConcat(int arity) {
    ClassDefinition definition = new ClassDefinition(a(PUBLIC, FINAL), CompilerUtils.makeClassName("Concat" + arity + "ScalarFunction"), type(Object.class));
    // Generate constructor
    definition.declareDefaultConstructor(a(PRIVATE));
    // Generate concat()
    List<Parameter> parameters = IntStream.range(0, arity).mapToObj(i -> arg("arg" + i, Slice.class)).collect(toImmutableList());
    MethodDefinition method = definition.declareMethod(a(PUBLIC, STATIC), "concat", type(Slice.class), parameters);
    Scope scope = method.getScope();
    BytecodeBlock body = method.getBody();
    Variable length = scope.declareVariable(int.class, "length");
    body.append(length.set(constantInt(0)));
    for (int i = 0; i < arity; ++i) {
        body.append(length.set(generateCheckedAdd(length, parameters.get(i).invoke("length", int.class))));
    }
    Variable result = scope.declareVariable(Slice.class, "result");
    body.append(result.set(invokeStatic(Slices.class, "allocate", Slice.class, length)));
    Variable position = scope.declareVariable(int.class, "position");
    body.append(position.set(constantInt(0)));
    for (int i = 0; i < arity; ++i) {
        body.append(result.invoke("setBytes", void.class, position, parameters.get(i)));
        body.append(position.set(add(position, parameters.get(i).invoke("length", int.class))));
    }
    body.getVariable(result).retObject();
    return defineClass(definition, Object.class, ImmutableMap.of(), new DynamicClassLoader(ConcatFunction.class.getClassLoader()));
}
Also used : IntStream(java.util.stream.IntStream) BytecodeExpression(com.facebook.presto.bytecode.expression.BytecodeExpression) MethodHandle(java.lang.invoke.MethodHandle) TypeManager(com.facebook.presto.spi.type.TypeManager) Slice(io.airlift.slice.Slice) BytecodeExpressions.constantInt(com.facebook.presto.bytecode.expression.BytecodeExpressions.constantInt) PRIVATE(com.facebook.presto.bytecode.Access.PRIVATE) PrestoException(com.facebook.presto.spi.PrestoException) MethodDefinition(com.facebook.presto.bytecode.MethodDefinition) FunctionKind(com.facebook.presto.metadata.FunctionKind) Access.a(com.facebook.presto.bytecode.Access.a) ImmutableList(com.google.common.collect.ImmutableList) ParameterizedType.type(com.facebook.presto.bytecode.ParameterizedType.type) Reflection.methodHandle(com.facebook.presto.util.Reflection.methodHandle) Slices(io.airlift.slice.Slices) BytecodeBlock(com.facebook.presto.bytecode.BytecodeBlock) INVALID_FUNCTION_ARGUMENT(com.facebook.presto.spi.StandardErrorCode.INVALID_FUNCTION_ARGUMENT) ImmutableCollectors.toImmutableList(com.facebook.presto.util.ImmutableCollectors.toImmutableList) UsedByGeneratedCode(com.facebook.presto.annotation.UsedByGeneratedCode) BytecodeExpressions.add(com.facebook.presto.bytecode.expression.BytecodeExpressions.add) PUBLIC(com.facebook.presto.bytecode.Access.PUBLIC) Variable(com.facebook.presto.bytecode.Variable) BoundVariables(com.facebook.presto.metadata.BoundVariables) Parameter(com.facebook.presto.bytecode.Parameter) ImmutableMap(com.google.common.collect.ImmutableMap) SqlScalarFunction(com.facebook.presto.metadata.SqlScalarFunction) FunctionRegistry(com.facebook.presto.metadata.FunctionRegistry) Signature(com.facebook.presto.metadata.Signature) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) VarcharType.createUnboundedVarcharType(com.facebook.presto.spi.type.VarcharType.createUnboundedVarcharType) ClassDefinition(com.facebook.presto.bytecode.ClassDefinition) List(java.util.List) CompilerUtils(com.facebook.presto.bytecode.CompilerUtils) FINAL(com.facebook.presto.bytecode.Access.FINAL) STATIC(com.facebook.presto.bytecode.Access.STATIC) Math.addExact(java.lang.Math.addExact) Scope(com.facebook.presto.bytecode.Scope) CompilerUtils.defineClass(com.facebook.presto.bytecode.CompilerUtils.defineClass) BytecodeExpressions.invokeStatic(com.facebook.presto.bytecode.expression.BytecodeExpressions.invokeStatic) Collections(java.util.Collections) Parameter.arg(com.facebook.presto.bytecode.Parameter.arg) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) Variable(com.facebook.presto.bytecode.Variable) Scope(com.facebook.presto.bytecode.Scope) MethodDefinition(com.facebook.presto.bytecode.MethodDefinition) Slice(io.airlift.slice.Slice) BytecodeBlock(com.facebook.presto.bytecode.BytecodeBlock) Parameter(com.facebook.presto.bytecode.Parameter) ClassDefinition(com.facebook.presto.bytecode.ClassDefinition)

Example 10 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class AbstractGreatestLeast method generate.

private Class<?> generate(List<Class<?>> javaTypes, Type type, MethodHandle compareMethod) {
    String javaTypeName = javaTypes.stream().map(Class::getSimpleName).collect(joining());
    ClassDefinition definition = new ClassDefinition(a(PUBLIC, FINAL), CompilerUtils.makeClassName(javaTypeName + "$" + getSignature().getName()), type(Object.class));
    definition.declareDefaultConstructor(a(PRIVATE));
    List<Parameter> parameters = IntStream.range(0, javaTypes.size()).mapToObj(i -> arg("arg" + i, javaTypes.get(i))).collect(toImmutableList());
    MethodDefinition method = definition.declareMethod(a(PUBLIC, STATIC), getSignature().getName(), type(javaTypes.get(0)), parameters);
    Scope scope = method.getScope();
    BytecodeBlock body = method.getBody();
    CallSiteBinder binder = new CallSiteBinder();
    if (type.getTypeSignature().getBase().equals(StandardTypes.DOUBLE)) {
        for (Parameter parameter : parameters) {
            body.append(parameter);
            body.append(invoke(binder.bind(CHECK_NOT_NAN.bindTo(getSignature().getName())), "checkNotNaN"));
        }
    }
    Variable value = scope.declareVariable(javaTypes.get(0), "value");
    body.append(value.set(parameters.get(0)));
    for (int i = 1; i < javaTypes.size(); i++) {
        body.append(new IfStatement().condition(new BytecodeBlock().append(parameters.get(i)).append(value).append(invoke(binder.bind(compareMethod), "compare"))).ifTrue(value.set(parameters.get(i))));
    }
    body.append(value.ret());
    return defineClass(definition, Object.class, binder.getBindings(), new DynamicClassLoader(getClass().getClassLoader()));
}
Also used : IntStream(java.util.stream.IntStream) MethodHandle(java.lang.invoke.MethodHandle) TypeManager(com.facebook.presto.spi.type.TypeManager) PRIVATE(com.facebook.presto.bytecode.Access.PRIVATE) PrestoException(com.facebook.presto.spi.PrestoException) MethodDefinition(com.facebook.presto.bytecode.MethodDefinition) BytecodeUtils.invoke(com.facebook.presto.sql.gen.BytecodeUtils.invoke) FunctionKind(com.facebook.presto.metadata.FunctionKind) Access.a(com.facebook.presto.bytecode.Access.a) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) ImmutableList(com.google.common.collect.ImmutableList) BOOLEAN(com.facebook.presto.spi.type.BooleanType.BOOLEAN) ParameterizedType.type(com.facebook.presto.bytecode.ParameterizedType.type) Reflection.methodHandle(com.facebook.presto.util.Reflection.methodHandle) Type(com.facebook.presto.spi.type.Type) Objects.requireNonNull(java.util.Objects.requireNonNull) BytecodeBlock(com.facebook.presto.bytecode.BytecodeBlock) INVALID_FUNCTION_ARGUMENT(com.facebook.presto.spi.StandardErrorCode.INVALID_FUNCTION_ARGUMENT) StandardTypes(com.facebook.presto.spi.type.StandardTypes) ImmutableCollectors.toImmutableList(com.facebook.presto.util.ImmutableCollectors.toImmutableList) UsedByGeneratedCode(com.facebook.presto.annotation.UsedByGeneratedCode) IfStatement(com.facebook.presto.bytecode.control.IfStatement) PUBLIC(com.facebook.presto.bytecode.Access.PUBLIC) Variable(com.facebook.presto.bytecode.Variable) BoundVariables(com.facebook.presto.metadata.BoundVariables) Signature.orderableTypeParameter(com.facebook.presto.metadata.Signature.orderableTypeParameter) Parameter(com.facebook.presto.bytecode.Parameter) SqlScalarFunction(com.facebook.presto.metadata.SqlScalarFunction) FunctionRegistry(com.facebook.presto.metadata.FunctionRegistry) Signature(com.facebook.presto.metadata.Signature) Signature.internalOperator(com.facebook.presto.metadata.Signature.internalOperator) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) String.format(java.lang.String.format) Collectors.joining(java.util.stream.Collectors.joining) ClassDefinition(com.facebook.presto.bytecode.ClassDefinition) List(java.util.List) CompilerUtils(com.facebook.presto.bytecode.CompilerUtils) FINAL(com.facebook.presto.bytecode.Access.FINAL) STATIC(com.facebook.presto.bytecode.Access.STATIC) Scope(com.facebook.presto.bytecode.Scope) OperatorType(com.facebook.presto.spi.function.OperatorType) CompilerUtils.defineClass(com.facebook.presto.bytecode.CompilerUtils.defineClass) CallSiteBinder(com.facebook.presto.sql.gen.CallSiteBinder) TypeSignature.parseTypeSignature(com.facebook.presto.spi.type.TypeSignature.parseTypeSignature) Collections(java.util.Collections) Parameter.arg(com.facebook.presto.bytecode.Parameter.arg) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) Variable(com.facebook.presto.bytecode.Variable) BytecodeBlock(com.facebook.presto.bytecode.BytecodeBlock) ClassDefinition(com.facebook.presto.bytecode.ClassDefinition) IfStatement(com.facebook.presto.bytecode.control.IfStatement) Scope(com.facebook.presto.bytecode.Scope) MethodDefinition(com.facebook.presto.bytecode.MethodDefinition) CallSiteBinder(com.facebook.presto.sql.gen.CallSiteBinder) Signature.orderableTypeParameter(com.facebook.presto.metadata.Signature.orderableTypeParameter) Parameter(com.facebook.presto.bytecode.Parameter)

Aggregations

DynamicClassLoader (com.facebook.presto.bytecode.DynamicClassLoader)26 Type (com.facebook.presto.spi.type.Type)18 MethodHandle (java.lang.invoke.MethodHandle)11 ClassDefinition (com.facebook.presto.bytecode.ClassDefinition)7 MethodDefinition (com.facebook.presto.bytecode.MethodDefinition)6 ArrayType (com.facebook.presto.type.ArrayType)6 MapType (com.facebook.presto.type.MapType)6 Parameter (com.facebook.presto.bytecode.Parameter)5 ImmutableList (com.google.common.collect.ImmutableList)5 BytecodeBlock (com.facebook.presto.bytecode.BytecodeBlock)4 Variable (com.facebook.presto.bytecode.Variable)4 Signature (com.facebook.presto.metadata.Signature)4 ParameterMetadata (com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata)4 ImmutableCollectors.toImmutableList (com.facebook.presto.util.ImmutableCollectors.toImmutableList)4 CompilerUtils.defineClass (com.facebook.presto.bytecode.CompilerUtils.defineClass)3 Scope (com.facebook.presto.bytecode.Scope)3 BoundVariables (com.facebook.presto.metadata.BoundVariables)3 FunctionRegistry (com.facebook.presto.metadata.FunctionRegistry)3 Block (com.facebook.presto.spi.block.Block)3 BlockBuilder (com.facebook.presto.spi.block.BlockBuilder)3