Search in sources :

Example 1 with JavaClassLookupPayload

use of org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload in project beam by apache.

the class JavaClassLookupTransformProvider method findMappingConstructorMethod.

private Method findMappingConstructorMethod(Method[] methods, JavaClassLookupPayload payload, AllowedClass allowListClass) {
    Row constructorRow = decodeRow(payload.getConstructorSchema(), payload.getConstructorPayload());
    List<Method> mappingConstructorMethods = Arrays.stream(methods).filter(m -> isConstructorMethodForName(m, payload.getConstructorMethod(), allowListClass)).filter(m -> m.getParameterCount() == payload.getConstructorSchema().getFieldsCount()).filter(m -> parametersCompatible(m.getParameters(), constructorRow)).collect(Collectors.toList());
    if (mappingConstructorMethods.size() == 0) {
        throw new RuntimeException("Could not find a matching constructor method. When using field names, make sure they " + "are available in the compiled Java class.");
    } else if (mappingConstructorMethods.size() != 1) {
        throw new RuntimeException("Expected to find a single mapping constructor method but found " + mappingConstructorMethods.size() + " Payload was " + payload);
    }
    return mappingConstructorMethods.get(0);
}
Also used : JsonProperty(com.fasterxml.jackson.annotation.JsonProperty) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Arrays(java.util.Arrays) Array(java.lang.reflect.Array) NoSuchSchemaException(org.apache.beam.sdk.schemas.NoSuchSchemaException) SchemaApi(org.apache.beam.model.pipeline.v1.SchemaApi) RowCoder(org.apache.beam.sdk.coders.RowCoder) SerializableFunction(org.apache.beam.sdk.transforms.SerializableFunction) Constructor(java.lang.reflect.Constructor) ArrayList(java.util.ArrayList) PTransform(org.apache.beam.sdk.transforms.PTransform) FunctionSpec(org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec) SchemaRegistry(org.apache.beam.sdk.schemas.SchemaRegistry) ByteString(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString) JavaFieldSchema(org.apache.beam.sdk.schemas.JavaFieldSchema) PInput(org.apache.beam.sdk.values.PInput) Row(org.apache.beam.sdk.values.Row) Method(java.lang.reflect.Method) Nullable(org.checkerframework.checker.nullness.qual.Nullable) InvalidProtocolBufferException(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.InvalidProtocolBufferException) Field(org.apache.beam.sdk.schemas.Schema.Field) TransformProvider(org.apache.beam.sdk.expansion.service.ExpansionService.TransformProvider) Collection(java.util.Collection) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Schema(org.apache.beam.sdk.schemas.Schema) TypeName(org.apache.beam.sdk.schemas.Schema.TypeName) JavaClassLookupPayload(org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload) InvocationTargetException(java.lang.reflect.InvocationTargetException) ExpansionMethods(org.apache.beam.model.pipeline.v1.ExternalTransforms.ExpansionMethods) ClassUtils(org.apache.beam.repackaged.core.org.apache.commons.lang3.ClassUtils) POutput(org.apache.beam.sdk.values.POutput) List(java.util.List) ParameterizedType(java.lang.reflect.ParameterizedType) Type(java.lang.reflect.Type) ReflectHelpers(org.apache.beam.sdk.util.common.ReflectHelpers) JsonCreator(com.fasterxml.jackson.annotation.JsonCreator) AutoValue(com.google.auto.value.AutoValue) Annotation(java.lang.annotation.Annotation) Pattern(java.util.regex.Pattern) SchemaTranslation(org.apache.beam.sdk.schemas.SchemaTranslation) Collections(java.util.Collections) BeamUrns.getUrn(org.apache.beam.runners.core.construction.BeamUrns.getUrn) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) Row(org.apache.beam.sdk.values.Row) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Method(java.lang.reflect.Method)

Example 2 with JavaClassLookupPayload

use of org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload in project beam by apache.

the class JavaClassLookupTransformProvider method findMappingConstructor.

private Constructor<PTransform<InputT, OutputT>> findMappingConstructor(Constructor<?>[] constructors, JavaClassLookupPayload payload) {
    Row constructorRow = decodeRow(payload.getConstructorSchema(), payload.getConstructorPayload());
    List<Constructor<?>> mappingConstructors = Arrays.stream(constructors).filter(c -> c.getParameterCount() == payload.getConstructorSchema().getFieldsCount()).filter(c -> parametersCompatible(c.getParameters(), constructorRow)).collect(Collectors.toList());
    if (mappingConstructors.size() == 0) {
        throw new RuntimeException("Could not find a matching constructor. When using field names, make sure they are " + "available in the compiled Java class.");
    } else if (mappingConstructors.size() != 1) {
        throw new RuntimeException("Expected to find a single mapping constructor but found " + mappingConstructors.size());
    }
    return (Constructor<PTransform<InputT, OutputT>>) mappingConstructors.get(0);
}
Also used : JsonProperty(com.fasterxml.jackson.annotation.JsonProperty) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Arrays(java.util.Arrays) Array(java.lang.reflect.Array) NoSuchSchemaException(org.apache.beam.sdk.schemas.NoSuchSchemaException) SchemaApi(org.apache.beam.model.pipeline.v1.SchemaApi) RowCoder(org.apache.beam.sdk.coders.RowCoder) SerializableFunction(org.apache.beam.sdk.transforms.SerializableFunction) Constructor(java.lang.reflect.Constructor) ArrayList(java.util.ArrayList) PTransform(org.apache.beam.sdk.transforms.PTransform) FunctionSpec(org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec) SchemaRegistry(org.apache.beam.sdk.schemas.SchemaRegistry) ByteString(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString) JavaFieldSchema(org.apache.beam.sdk.schemas.JavaFieldSchema) PInput(org.apache.beam.sdk.values.PInput) Row(org.apache.beam.sdk.values.Row) Method(java.lang.reflect.Method) Nullable(org.checkerframework.checker.nullness.qual.Nullable) InvalidProtocolBufferException(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.InvalidProtocolBufferException) Field(org.apache.beam.sdk.schemas.Schema.Field) TransformProvider(org.apache.beam.sdk.expansion.service.ExpansionService.TransformProvider) Collection(java.util.Collection) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Schema(org.apache.beam.sdk.schemas.Schema) TypeName(org.apache.beam.sdk.schemas.Schema.TypeName) JavaClassLookupPayload(org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload) InvocationTargetException(java.lang.reflect.InvocationTargetException) ExpansionMethods(org.apache.beam.model.pipeline.v1.ExternalTransforms.ExpansionMethods) ClassUtils(org.apache.beam.repackaged.core.org.apache.commons.lang3.ClassUtils) POutput(org.apache.beam.sdk.values.POutput) List(java.util.List) ParameterizedType(java.lang.reflect.ParameterizedType) Type(java.lang.reflect.Type) ReflectHelpers(org.apache.beam.sdk.util.common.ReflectHelpers) JsonCreator(com.fasterxml.jackson.annotation.JsonCreator) AutoValue(com.google.auto.value.AutoValue) Annotation(java.lang.annotation.Annotation) Pattern(java.util.regex.Pattern) SchemaTranslation(org.apache.beam.sdk.schemas.SchemaTranslation) Collections(java.util.Collections) BeamUrns.getUrn(org.apache.beam.runners.core.construction.BeamUrns.getUrn) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) Constructor(java.lang.reflect.Constructor) Row(org.apache.beam.sdk.values.Row)

Example 3 with JavaClassLookupPayload

use of org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload in project beam by apache.

the class JavaClassLookupTransformProvider method applyBuilderMethods.

private PTransform<PInput, POutput> applyBuilderMethods(PTransform<PInput, POutput> transform, JavaClassLookupPayload payload, AllowedClass allowListClass) {
    for (BuilderMethod builderMethod : payload.getBuilderMethodsList()) {
        Method method = getMethod(transform, builderMethod, allowListClass);
        try {
            Row builderMethodRow = decodeRow(builderMethod.getSchema(), builderMethod.getPayload());
            transform = (PTransform<PInput, POutput>) method.invoke(transform, getParameterValues(method.getParameters(), builderMethodRow, method.getGenericParameterTypes()));
        } catch (IllegalAccessException | InvocationTargetException e) {
            throw new IllegalArgumentException("Could not invoke the builder method " + builderMethod + " on transform " + transform + " with parameter schema " + builderMethod.getSchema(), e);
        }
    }
    return transform;
}
Also used : PInput(org.apache.beam.sdk.values.PInput) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) POutput(org.apache.beam.sdk.values.POutput) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Method(java.lang.reflect.Method) Row(org.apache.beam.sdk.values.Row) InvocationTargetException(java.lang.reflect.InvocationTargetException)

Example 4 with JavaClassLookupPayload

use of org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload in project beam by apache.

the class JavaClassLookupTransformProvider method getTransform.

@Override
public PTransform<PInput, POutput> getTransform(FunctionSpec spec) {
    JavaClassLookupPayload payload;
    try {
        payload = JavaClassLookupPayload.parseFrom(spec.getPayload());
    } catch (InvalidProtocolBufferException e) {
        throw new IllegalArgumentException("Invalid payload type for URN " + getUrn(ExpansionMethods.Enum.JAVA_CLASS_LOOKUP), e);
    }
    String className = payload.getClassName();
    try {
        AllowedClass allowlistClass = allowList.getAllowedClass(className);
        Class<PTransform<InputT, OutputT>> transformClass = (Class<PTransform<InputT, OutputT>>) ReflectHelpers.findClassLoader().loadClass(className);
        PTransform<PInput, POutput> transform;
        Row constructorRow = decodeRow(payload.getConstructorSchema(), payload.getConstructorPayload());
        if (payload.getConstructorMethod().isEmpty()) {
            Constructor<?>[] constructors = transformClass.getConstructors();
            Constructor<PTransform<InputT, OutputT>> constructor = findMappingConstructor(constructors, payload);
            Object[] parameterValues = getParameterValues(constructor.getParameters(), constructorRow, constructor.getGenericParameterTypes());
            transform = (PTransform<PInput, POutput>) constructor.newInstance(parameterValues);
        } else {
            Method[] methods = transformClass.getMethods();
            Method method = findMappingConstructorMethod(methods, payload, allowlistClass);
            Object[] parameterValues = getParameterValues(method.getParameters(), constructorRow, method.getGenericParameterTypes());
            transform = (PTransform<PInput, POutput>) method.invoke(null, /* static */
            parameterValues);
        }
        return applyBuilderMethods(transform, payload, allowlistClass);
    } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException("Could not find class " + className, e);
    } catch (InstantiationException | IllegalArgumentException | IllegalAccessException | InvocationTargetException e) {
        throw new IllegalArgumentException("Could not instantiate class " + className, e);
    }
}
Also used : JavaClassLookupPayload(org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload) ByteString(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString) POutput(org.apache.beam.sdk.values.POutput) PTransform(org.apache.beam.sdk.transforms.PTransform) PInput(org.apache.beam.sdk.values.PInput) Constructor(java.lang.reflect.Constructor) InvalidProtocolBufferException(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.InvalidProtocolBufferException) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Method(java.lang.reflect.Method) InvocationTargetException(java.lang.reflect.InvocationTargetException) Row(org.apache.beam.sdk.values.Row)

Aggregations

InvocationTargetException (java.lang.reflect.InvocationTargetException)4 Method (java.lang.reflect.Method)4 BuilderMethod (org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod)4 PInput (org.apache.beam.sdk.values.PInput)4 POutput (org.apache.beam.sdk.values.POutput)4 Constructor (java.lang.reflect.Constructor)3 JavaClassLookupPayload (org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload)3 PTransform (org.apache.beam.sdk.transforms.PTransform)3 JsonCreator (com.fasterxml.jackson.annotation.JsonCreator)2 JsonProperty (com.fasterxml.jackson.annotation.JsonProperty)2 AutoValue (com.google.auto.value.AutoValue)2 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)2 IOException (java.io.IOException)2 Annotation (java.lang.annotation.Annotation)2 Array (java.lang.reflect.Array)2 ParameterizedType (java.lang.reflect.ParameterizedType)2 Type (java.lang.reflect.Type)2 ArrayList (java.util.ArrayList)2 Arrays (java.util.Arrays)2 Collection (java.util.Collection)2