Search in sources :

Example 21 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class BindableAggregationFunction method specialize.

@Override
public InternalAggregationFunction specialize(BoundVariables variables, int arity, TypeManager typeManager, FunctionRegistry functionRegistry) {
    // bind variables
    Signature boundSignature = applyBoundVariables(getSignature(), variables, arity);
    List<Type> inputTypes = boundSignature.getArgumentTypes().stream().map(x -> typeManager.getType(x)).collect(toImmutableList());
    Type outputType = typeManager.getType(boundSignature.getReturnType());
    AggregationFunction aggregationAnnotation = definitionClass.getAnnotation(AggregationFunction.class);
    requireNonNull(aggregationAnnotation, "aggregationAnnotation is null");
    DynamicClassLoader classLoader = new DynamicClassLoader(definitionClass.getClassLoader(), getClass().getClassLoader());
    AggregationMetadata metadata;
    AccumulatorStateSerializer<?> stateSerializer = StateCompiler.generateStateSerializer(stateClass, classLoader);
    Type intermediateType = stateSerializer.getSerializedType();
    Method combineFunction = AggregationCompiler.getCombineFunction(definitionClass, stateClass);
    AccumulatorStateFactory<?> stateFactory = StateCompiler.generateStateFactory(stateClass, classLoader);
    try {
        MethodHandle inputHandle = lookup().unreflect(inputFunction);
        MethodHandle combineHandle = lookup().unreflect(combineFunction);
        MethodHandle outputHandle = outputFunction == null ? null : lookup().unreflect(outputFunction);
        metadata = new AggregationMetadata(generateAggregationName(getSignature().getName(), outputType.getTypeSignature(), signaturesFromTypes(inputTypes)), getParameterMetadata(inputFunction, inputTypes), inputHandle, combineHandle, outputHandle, stateClass, stateSerializer, stateFactory, outputType);
    } catch (IllegalAccessException e) {
        throw Throwables.propagate(e);
    }
    AccumulatorFactoryBinder factory = new LazyAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(getSignature().getName(), inputTypes, intermediateType, outputType, decomposable, factory);
}
Also used : TypeSignature(com.facebook.presto.spi.type.TypeSignature) MethodHandle(java.lang.invoke.MethodHandle) Arrays(java.util.Arrays) AggregationCompiler.isParameterBlock(com.facebook.presto.operator.aggregation.AggregationCompiler.isParameterBlock) TypeManager(com.facebook.presto.spi.type.TypeManager) AggregationUtils.generateAggregationName(com.facebook.presto.operator.aggregation.AggregationUtils.generateAggregationName) MethodHandles.lookup(java.lang.invoke.MethodHandles.lookup) ParameterMetadata(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) ImmutableList(com.google.common.collect.ImmutableList) AggregationState(com.facebook.presto.spi.function.AggregationState) AggregationFunction(com.facebook.presto.spi.function.AggregationFunction) Type(com.facebook.presto.spi.type.Type) Objects.requireNonNull(java.util.Objects.requireNonNull) SqlAggregationFunction(com.facebook.presto.metadata.SqlAggregationFunction) ImmutableCollectors.toImmutableList(com.facebook.presto.util.ImmutableCollectors.toImmutableList) Method(java.lang.reflect.Method) Nullable(javax.annotation.Nullable) ParameterMetadata.fromSqlType(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata.fromSqlType) AggregationCompiler.isParameterNullable(com.facebook.presto.operator.aggregation.AggregationCompiler.isParameterNullable) SignatureBinder.applyBoundVariables(com.facebook.presto.metadata.SignatureBinder.applyBoundVariables) BoundVariables(com.facebook.presto.metadata.BoundVariables) FunctionRegistry(com.facebook.presto.metadata.FunctionRegistry) Signature(com.facebook.presto.metadata.Signature) Throwables(com.google.common.base.Throwables) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) BLOCK_INDEX(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata.ParameterType.BLOCK_INDEX) List(java.util.List) AccumulatorStateSerializer(com.facebook.presto.spi.function.AccumulatorStateSerializer) Annotation(java.lang.annotation.Annotation) AccumulatorStateFactory(com.facebook.presto.spi.function.AccumulatorStateFactory) StateCompiler(com.facebook.presto.operator.aggregation.state.StateCompiler) SqlType(com.facebook.presto.spi.function.SqlType) STATE(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata.ParameterType.STATE) DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) Method(java.lang.reflect.Method) AggregationFunction(com.facebook.presto.spi.function.AggregationFunction) SqlAggregationFunction(com.facebook.presto.metadata.SqlAggregationFunction) Type(com.facebook.presto.spi.type.Type) ParameterMetadata.fromSqlType(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata.fromSqlType) SqlType(com.facebook.presto.spi.function.SqlType) TypeSignature(com.facebook.presto.spi.type.TypeSignature) Signature(com.facebook.presto.metadata.Signature) MethodHandle(java.lang.invoke.MethodHandle)

Example 22 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class CountColumn method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type type) {
    DynamicClassLoader classLoader = new DynamicClassLoader(CountColumn.class.getClassLoader());
    AccumulatorStateSerializer<LongState> stateSerializer = StateCompiler.generateStateSerializer(LongState.class, classLoader);
    AccumulatorStateFactory<LongState> stateFactory = StateCompiler.generateStateFactory(LongState.class, classLoader);
    Type intermediateType = stateSerializer.getSerializedType();
    List<Type> inputTypes = ImmutableList.of(type);
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, BIGINT.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), createInputParameterMetadata(type), INPUT_FUNCTION, COMBINE_FUNCTION, OUTPUT_FUNCTION, LongState.class, stateSerializer, stateFactory, BIGINT);
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, BIGINT, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) Type(com.facebook.presto.spi.type.Type) LongState(com.facebook.presto.operator.aggregation.state.LongState)

Example 23 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class DecimalAverageAggregation method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type type) {
    checkArgument(type instanceof DecimalType, "type must be Decimal");
    DynamicClassLoader classLoader = new DynamicClassLoader(DecimalAverageAggregation.class.getClassLoader());
    List<Type> inputTypes = ImmutableList.of(type);
    MethodHandle inputFunction;
    MethodHandle outputFunction;
    Class<? extends AccumulatorState> stateInterface = LongDecimalWithOverflowAndLongState.class;
    AccumulatorStateSerializer<?> stateSerializer = new LongDecimalWithOverflowAndLongStateSerializer();
    if (((DecimalType) type).isShort()) {
        inputFunction = SHORT_DECIMAL_INPUT_FUNCTION;
        outputFunction = SHORT_DECIMAL_OUTPUT_FUNCTION;
    } else {
        inputFunction = LONG_DECIMAL_INPUT_FUNCTION;
        outputFunction = LONG_DECIMAL_OUTPUT_FUNCTION;
    }
    inputFunction = inputFunction.bindTo(type);
    outputFunction = outputFunction.bindTo(type);
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, type.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), createInputParameterMetadata(type), inputFunction, COMBINE_FUNCTION, outputFunction, stateInterface, stateSerializer, new LongDecimalWithOverflowAndLongStateFactory(), type);
    Type intermediateType = stateSerializer.getSerializedType();
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, type, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) LongDecimalWithOverflowAndLongStateFactory(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowAndLongStateFactory) LongDecimalWithOverflowAndLongState(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowAndLongState) DecimalType(com.facebook.presto.spi.type.DecimalType) Type(com.facebook.presto.spi.type.Type) DecimalType(com.facebook.presto.spi.type.DecimalType) LongDecimalWithOverflowAndLongStateSerializer(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowAndLongStateSerializer) MethodHandle(java.lang.invoke.MethodHandle)

Example 24 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class DecimalSumAggregation method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type inputType, Type outputType) {
    checkArgument(inputType instanceof DecimalType, "type must be Decimal");
    DynamicClassLoader classLoader = new DynamicClassLoader(DecimalSumAggregation.class.getClassLoader());
    List<Type> inputTypes = ImmutableList.of(inputType);
    MethodHandle inputFunction;
    Class<? extends AccumulatorState> stateInterface = LongDecimalWithOverflowState.class;
    AccumulatorStateSerializer<?> stateSerializer = new LongDecimalWithOverflowStateSerializer();
    if (((DecimalType) inputType).isShort()) {
        inputFunction = SHORT_DECIMAL_INPUT_FUNCTION;
    } else {
        inputFunction = LONG_DECIMAL_INPUT_FUNCTION;
    }
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, outputType.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), createInputParameterMetadata(inputType), inputFunction.bindTo(inputType), COMBINE_FUNCTION, LONG_DECIMAL_OUTPUT_FUNCTION.bindTo(outputType), stateInterface, stateSerializer, new LongDecimalWithOverflowStateFactory(), outputType);
    Type intermediateType = stateSerializer.getSerializedType();
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, outputType, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) LongDecimalWithOverflowState(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowState) DecimalType(com.facebook.presto.spi.type.DecimalType) Type(com.facebook.presto.spi.type.Type) LongDecimalWithOverflowStateFactory(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowStateFactory) DecimalType(com.facebook.presto.spi.type.DecimalType) LongDecimalWithOverflowStateSerializer(com.facebook.presto.operator.aggregation.state.LongDecimalWithOverflowStateSerializer) MethodHandle(java.lang.invoke.MethodHandle)

Example 25 with DynamicClassLoader

use of com.facebook.presto.bytecode.DynamicClassLoader in project presto by prestodb.

the class ArbitraryAggregationFunction method generateAggregation.

private static InternalAggregationFunction generateAggregation(Type type) {
    DynamicClassLoader classLoader = new DynamicClassLoader(ArbitraryAggregationFunction.class.getClassLoader());
    List<Type> inputTypes = ImmutableList.of(type);
    MethodHandle inputFunction;
    MethodHandle combineFunction;
    MethodHandle outputFunction;
    Class<? extends AccumulatorState> stateInterface;
    AccumulatorStateSerializer<?> stateSerializer;
    if (type.getJavaType() == long.class) {
        stateInterface = NullableLongState.class;
        stateSerializer = StateCompiler.generateStateSerializer(stateInterface, classLoader);
        inputFunction = LONG_INPUT_FUNCTION;
        combineFunction = LONG_COMBINE_FUNCTION;
        outputFunction = LONG_OUTPUT_FUNCTION;
    } else if (type.getJavaType() == double.class) {
        stateInterface = NullableDoubleState.class;
        stateSerializer = StateCompiler.generateStateSerializer(stateInterface, classLoader);
        inputFunction = DOUBLE_INPUT_FUNCTION;
        combineFunction = DOUBLE_COMBINE_FUNCTION;
        outputFunction = DOUBLE_OUTPUT_FUNCTION;
    } else if (type.getJavaType() == Slice.class) {
        stateInterface = SliceState.class;
        stateSerializer = StateCompiler.generateStateSerializer(stateInterface, classLoader);
        inputFunction = SLICE_INPUT_FUNCTION;
        combineFunction = SLICE_COMBINE_FUNCTION;
        outputFunction = SLICE_OUTPUT_FUNCTION;
    } else if (type.getJavaType() == boolean.class) {
        stateInterface = NullableBooleanState.class;
        stateSerializer = StateCompiler.generateStateSerializer(stateInterface, classLoader);
        inputFunction = BOOLEAN_INPUT_FUNCTION;
        combineFunction = BOOLEAN_COMBINE_FUNCTION;
        outputFunction = BOOLEAN_OUTPUT_FUNCTION;
    } else {
        stateInterface = BlockState.class;
        stateSerializer = new BlockStateSerializer(type);
        inputFunction = BLOCK_INPUT_FUNCTION;
        combineFunction = BLOCK_COMBINE_FUNCTION;
        outputFunction = BLOCK_OUTPUT_FUNCTION;
    }
    inputFunction = inputFunction.bindTo(type);
    Type intermediateType = stateSerializer.getSerializedType();
    List<ParameterMetadata> inputParameterMetadata = createInputParameterMetadata(type);
    AggregationMetadata metadata = new AggregationMetadata(generateAggregationName(NAME, type.getTypeSignature(), inputTypes.stream().map(Type::getTypeSignature).collect(toImmutableList())), inputParameterMetadata, inputFunction, combineFunction, outputFunction.bindTo(type), stateInterface, stateSerializer, StateCompiler.generateStateFactory(stateInterface, classLoader), type);
    GenericAccumulatorFactoryBinder factory = AccumulatorCompiler.generateAccumulatorFactoryBinder(metadata, classLoader);
    return new InternalAggregationFunction(NAME, inputTypes, intermediateType, type, true, factory);
}
Also used : DynamicClassLoader(com.facebook.presto.bytecode.DynamicClassLoader) BlockStateSerializer(com.facebook.presto.operator.aggregation.state.BlockStateSerializer) NullableBooleanState(com.facebook.presto.operator.aggregation.state.NullableBooleanState) ParameterMetadata(com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata) Type(com.facebook.presto.spi.type.Type) BlockState(com.facebook.presto.operator.aggregation.state.BlockState) NullableDoubleState(com.facebook.presto.operator.aggregation.state.NullableDoubleState) MethodHandle(java.lang.invoke.MethodHandle)

Aggregations

DynamicClassLoader (com.facebook.presto.bytecode.DynamicClassLoader)26 Type (com.facebook.presto.spi.type.Type)18 MethodHandle (java.lang.invoke.MethodHandle)11 ClassDefinition (com.facebook.presto.bytecode.ClassDefinition)7 MethodDefinition (com.facebook.presto.bytecode.MethodDefinition)6 ArrayType (com.facebook.presto.type.ArrayType)6 MapType (com.facebook.presto.type.MapType)6 Parameter (com.facebook.presto.bytecode.Parameter)5 ImmutableList (com.google.common.collect.ImmutableList)5 BytecodeBlock (com.facebook.presto.bytecode.BytecodeBlock)4 Variable (com.facebook.presto.bytecode.Variable)4 Signature (com.facebook.presto.metadata.Signature)4 ParameterMetadata (com.facebook.presto.operator.aggregation.AggregationMetadata.ParameterMetadata)4 ImmutableCollectors.toImmutableList (com.facebook.presto.util.ImmutableCollectors.toImmutableList)4 CompilerUtils.defineClass (com.facebook.presto.bytecode.CompilerUtils.defineClass)3 Scope (com.facebook.presto.bytecode.Scope)3 BoundVariables (com.facebook.presto.metadata.BoundVariables)3 FunctionRegistry (com.facebook.presto.metadata.FunctionRegistry)3 Block (com.facebook.presto.spi.block.Block)3 BlockBuilder (com.facebook.presto.spi.block.BlockBuilder)3