Search in sources :

Example 1 with CollectDynamicSink

use of org.apache.flink.table.planner.connectors.CollectDynamicSink in project flink by apache.

the class StreamExecSink method translateToPlanInternal.

@SuppressWarnings("unchecked")
@Override
protected Transformation<Object> translateToPlanInternal(PlannerBase planner, ExecNodeConfig config) {
    final ExecEdge inputEdge = getInputEdges().get(0);
    final Transformation<RowData> inputTransform = (Transformation<RowData>) inputEdge.translateToPlan(planner);
    final RowType inputRowType = (RowType) inputEdge.getOutputType();
    final DynamicTableSink tableSink = tableSinkSpec.getTableSink(planner.getFlinkContext());
    final boolean isCollectSink = tableSink instanceof CollectDynamicSink;
    final List<Integer> rowtimeFieldIndices = new ArrayList<>();
    for (int i = 0; i < inputRowType.getFieldCount(); ++i) {
        if (TypeCheckUtils.isRowTime(inputRowType.getTypeAt(i))) {
            rowtimeFieldIndices.add(i);
        }
    }
    final int rowtimeFieldIndex;
    if (rowtimeFieldIndices.size() > 1 && !isCollectSink) {
        throw new TableException(String.format("The query contains more than one rowtime attribute column [%s] for writing into table '%s'.\n" + "Please select the column that should be used as the event-time timestamp " + "for the table sink by casting all other columns to regular TIMESTAMP or TIMESTAMP_LTZ.", rowtimeFieldIndices.stream().map(i -> inputRowType.getFieldNames().get(i)).collect(Collectors.joining(", ")), tableSinkSpec.getContextResolvedTable().getIdentifier().asSummaryString()));
    } else if (rowtimeFieldIndices.size() == 1) {
        rowtimeFieldIndex = rowtimeFieldIndices.get(0);
    } else {
        rowtimeFieldIndex = -1;
    }
    return createSinkTransformation(planner.getExecEnv(), config, inputTransform, tableSink, rowtimeFieldIndex, upsertMaterialize);
}
Also used : InputProperty(org.apache.flink.table.planner.plan.nodes.exec.InputProperty) JsonCreator(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonCreator) ChangelogMode(org.apache.flink.table.connector.ChangelogMode) CollectDynamicSink(org.apache.flink.table.planner.connectors.CollectDynamicSink) RowType(org.apache.flink.table.types.logical.RowType) JsonInclude(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonInclude) ExecNode(org.apache.flink.table.planner.plan.nodes.exec.ExecNode) ArrayList(java.util.ArrayList) DynamicTableSinkSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSinkSpec) FlinkVersion(org.apache.flink.FlinkVersion) TypeCheckUtils(org.apache.flink.table.runtime.typeutils.TypeCheckUtils) ExecNodeContext(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeContext) RowData(org.apache.flink.table.data.RowData) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) PlannerBase(org.apache.flink.table.planner.delegation.PlannerBase) ExecNodeMetadata(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeMetadata) ExecNodeConfig(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeConfig) TableException(org.apache.flink.table.api.TableException) CommonExecSink(org.apache.flink.table.planner.plan.nodes.exec.common.CommonExecSink) Collectors(java.util.stream.Collectors) JsonProperty(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonProperty) List(java.util.List) ExecEdge(org.apache.flink.table.planner.plan.nodes.exec.ExecEdge) LogicalType(org.apache.flink.table.types.logical.LogicalType) Transformation(org.apache.flink.api.dag.Transformation) Collections(java.util.Collections) Transformation(org.apache.flink.api.dag.Transformation) TableException(org.apache.flink.table.api.TableException) ExecEdge(org.apache.flink.table.planner.plan.nodes.exec.ExecEdge) ArrayList(java.util.ArrayList) RowType(org.apache.flink.table.types.logical.RowType) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) CollectDynamicSink(org.apache.flink.table.planner.connectors.CollectDynamicSink) RowData(org.apache.flink.table.data.RowData)

Aggregations

ArrayList (java.util.ArrayList)1 Collections (java.util.Collections)1 List (java.util.List)1 Collectors (java.util.stream.Collectors)1 FlinkVersion (org.apache.flink.FlinkVersion)1 Transformation (org.apache.flink.api.dag.Transformation)1 JsonCreator (org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonCreator)1 JsonInclude (org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonInclude)1 JsonProperty (org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonProperty)1 TableException (org.apache.flink.table.api.TableException)1 ChangelogMode (org.apache.flink.table.connector.ChangelogMode)1 DynamicTableSink (org.apache.flink.table.connector.sink.DynamicTableSink)1 RowData (org.apache.flink.table.data.RowData)1 CollectDynamicSink (org.apache.flink.table.planner.connectors.CollectDynamicSink)1 PlannerBase (org.apache.flink.table.planner.delegation.PlannerBase)1 ExecEdge (org.apache.flink.table.planner.plan.nodes.exec.ExecEdge)1 ExecNode (org.apache.flink.table.planner.plan.nodes.exec.ExecNode)1 ExecNodeConfig (org.apache.flink.table.planner.plan.nodes.exec.ExecNodeConfig)1 ExecNodeContext (org.apache.flink.table.planner.plan.nodes.exec.ExecNodeContext)1 ExecNodeMetadata (org.apache.flink.table.planner.plan.nodes.exec.ExecNodeMetadata)1