Search in sources :

Example 1 with TransformationScanProvider

use of org.apache.flink.table.planner.connectors.TransformationScanProvider in project flink by apache.

the class CommonExecTableSourceScan method translateToPlanInternal.

@Override
protected Transformation<RowData> translateToPlanInternal(PlannerBase planner, ExecNodeConfig config) {
    final StreamExecutionEnvironment env = planner.getExecEnv();
    final TransformationMetadata meta = createTransformationMeta(SOURCE_TRANSFORMATION, config);
    final InternalTypeInfo<RowData> outputTypeInfo = InternalTypeInfo.of((RowType) getOutputType());
    final ScanTableSource tableSource = tableSourceSpec.getScanTableSource(planner.getFlinkContext());
    ScanTableSource.ScanRuntimeProvider provider = tableSource.getScanRuntimeProvider(ScanRuntimeProviderContext.INSTANCE);
    if (provider instanceof SourceFunctionProvider) {
        final SourceFunctionProvider sourceFunctionProvider = (SourceFunctionProvider) provider;
        final SourceFunction<RowData> function = sourceFunctionProvider.createSourceFunction();
        final Transformation<RowData> transformation = createSourceFunctionTransformation(env, function, sourceFunctionProvider.isBounded(), meta.getName(), outputTypeInfo);
        return meta.fill(transformation);
    } else if (provider instanceof InputFormatProvider) {
        final InputFormat<RowData, ?> inputFormat = ((InputFormatProvider) provider).createInputFormat();
        final Transformation<RowData> transformation = createInputFormatTransformation(env, inputFormat, outputTypeInfo, meta.getName());
        return meta.fill(transformation);
    } else if (provider instanceof SourceProvider) {
        final Source<RowData, ?, ?> source = ((SourceProvider) provider).createSource();
        // TODO: Push down watermark strategy to source scan
        final Transformation<RowData> transformation = env.fromSource(source, WatermarkStrategy.noWatermarks(), meta.getName(), outputTypeInfo).getTransformation();
        return meta.fill(transformation);
    } else if (provider instanceof DataStreamScanProvider) {
        Transformation<RowData> transformation = ((DataStreamScanProvider) provider).produceDataStream(createProviderContext(), env).getTransformation();
        meta.fill(transformation);
        transformation.setOutputType(outputTypeInfo);
        return transformation;
    } else if (provider instanceof TransformationScanProvider) {
        final Transformation<RowData> transformation = ((TransformationScanProvider) provider).createTransformation(createProviderContext());
        meta.fill(transformation);
        transformation.setOutputType(outputTypeInfo);
        return transformation;
    } else {
        throw new UnsupportedOperationException(provider.getClass().getSimpleName() + " is unsupported now.");
    }
}
Also used : TransformationMetadata(org.apache.flink.table.planner.plan.nodes.exec.utils.TransformationMetadata) LegacySourceTransformation(org.apache.flink.streaming.api.transformations.LegacySourceTransformation) Transformation(org.apache.flink.api.dag.Transformation) TransformationScanProvider(org.apache.flink.table.planner.connectors.TransformationScanProvider) InputFormatProvider(org.apache.flink.table.connector.source.InputFormatProvider) SourceFunctionProvider(org.apache.flink.table.connector.source.SourceFunctionProvider) SourceProvider(org.apache.flink.table.connector.source.SourceProvider) ScanTableSource(org.apache.flink.table.connector.source.ScanTableSource) RowData(org.apache.flink.table.data.RowData) InputFormat(org.apache.flink.api.common.io.InputFormat) DataStreamScanProvider(org.apache.flink.table.connector.source.DataStreamScanProvider) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)

Aggregations

InputFormat (org.apache.flink.api.common.io.InputFormat)1 Transformation (org.apache.flink.api.dag.Transformation)1 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)1 LegacySourceTransformation (org.apache.flink.streaming.api.transformations.LegacySourceTransformation)1 DataStreamScanProvider (org.apache.flink.table.connector.source.DataStreamScanProvider)1 InputFormatProvider (org.apache.flink.table.connector.source.InputFormatProvider)1 ScanTableSource (org.apache.flink.table.connector.source.ScanTableSource)1 SourceFunctionProvider (org.apache.flink.table.connector.source.SourceFunctionProvider)1 SourceProvider (org.apache.flink.table.connector.source.SourceProvider)1 RowData (org.apache.flink.table.data.RowData)1 TransformationScanProvider (org.apache.flink.table.planner.connectors.TransformationScanProvider)1 TransformationMetadata (org.apache.flink.table.planner.plan.nodes.exec.utils.TransformationMetadata)1