Search in sources :

Example 91 with PTransform

use of org.apache.beam.model.pipeline.v1.RunnerApi.PTransform in project beam by apache.

the class JavaClassLookupTransformProvider method getTransform.

@Override
public PTransform<PInput, POutput> getTransform(FunctionSpec spec) {
    JavaClassLookupPayload payload;
    try {
        payload = JavaClassLookupPayload.parseFrom(spec.getPayload());
    } catch (InvalidProtocolBufferException e) {
        throw new IllegalArgumentException("Invalid payload type for URN " + getUrn(ExpansionMethods.Enum.JAVA_CLASS_LOOKUP), e);
    }
    String className = payload.getClassName();
    try {
        AllowedClass allowlistClass = allowList.getAllowedClass(className);
        Class<PTransform<InputT, OutputT>> transformClass = (Class<PTransform<InputT, OutputT>>) ReflectHelpers.findClassLoader().loadClass(className);
        PTransform<PInput, POutput> transform;
        Row constructorRow = decodeRow(payload.getConstructorSchema(), payload.getConstructorPayload());
        if (payload.getConstructorMethod().isEmpty()) {
            Constructor<?>[] constructors = transformClass.getConstructors();
            Constructor<PTransform<InputT, OutputT>> constructor = findMappingConstructor(constructors, payload);
            Object[] parameterValues = getParameterValues(constructor.getParameters(), constructorRow, constructor.getGenericParameterTypes());
            transform = (PTransform<PInput, POutput>) constructor.newInstance(parameterValues);
        } else {
            Method[] methods = transformClass.getMethods();
            Method method = findMappingConstructorMethod(methods, payload, allowlistClass);
            Object[] parameterValues = getParameterValues(method.getParameters(), constructorRow, method.getGenericParameterTypes());
            transform = (PTransform<PInput, POutput>) method.invoke(null, /* static */
            parameterValues);
        }
        return applyBuilderMethods(transform, payload, allowlistClass);
    } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException("Could not find class " + className, e);
    } catch (InstantiationException | IllegalArgumentException | IllegalAccessException | InvocationTargetException e) {
        throw new IllegalArgumentException("Could not instantiate class " + className, e);
    }
}
Also used : JavaClassLookupPayload(org.apache.beam.model.pipeline.v1.ExternalTransforms.JavaClassLookupPayload) ByteString(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString) POutput(org.apache.beam.sdk.values.POutput) PTransform(org.apache.beam.sdk.transforms.PTransform) PInput(org.apache.beam.sdk.values.PInput) Constructor(java.lang.reflect.Constructor) InvalidProtocolBufferException(org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.InvalidProtocolBufferException) BuilderMethod(org.apache.beam.model.pipeline.v1.ExternalTransforms.BuilderMethod) Method(java.lang.reflect.Method) InvocationTargetException(java.lang.reflect.InvocationTargetException) Row(org.apache.beam.sdk.values.Row)

Aggregations

PTransform (org.apache.beam.model.pipeline.v1.RunnerApi.PTransform)58 Test (org.junit.Test)41 RunnerApi (org.apache.beam.model.pipeline.v1.RunnerApi)28 Components (org.apache.beam.model.pipeline.v1.RunnerApi.Components)23 PTransformNode (org.apache.beam.runners.core.construction.graph.PipelineNode.PTransformNode)22 PCollectionNode (org.apache.beam.runners.core.construction.graph.PipelineNode.PCollectionNode)21 PCollection (org.apache.beam.model.pipeline.v1.RunnerApi.PCollection)19 Map (java.util.Map)18 Environment (org.apache.beam.model.pipeline.v1.RunnerApi.Environment)18 ArrayList (java.util.ArrayList)16 ByteString (org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString)16 WindowedValue (org.apache.beam.sdk.util.WindowedValue)11 Collection (java.util.Collection)9 SdkComponents (org.apache.beam.runners.core.construction.SdkComponents)9 Collectors (java.util.stream.Collectors)8 ImmutableMap (org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableMap)8 IOException (java.io.IOException)7 HashSet (java.util.HashSet)7 FunctionSpec (org.apache.beam.model.pipeline.v1.RunnerApi.FunctionSpec)7 Pipeline (org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline)7