Search in sources :

Example 1 with RelHint

use of org.apache.calcite.rel.hint.RelHint in project flink by apache.

the class CatalogSourceTable method toRel.

@Override
public RelNode toRel(ToRelContext toRelContext) {
    final RelOptCluster cluster = toRelContext.getCluster();
    final List<RelHint> hints = toRelContext.getTableHints();
    final FlinkContext context = ShortcutUtils.unwrapContext(cluster);
    final FlinkRelBuilder relBuilder = FlinkRelBuilder.of(cluster, relOptSchema);
    // finalize catalog table with option hints
    final Map<String, String> hintedOptions = FlinkHints.getHintedOptions(hints);
    final ContextResolvedTable catalogTable = computeContextResolvedTable(context, hintedOptions);
    // create table source
    final DynamicTableSource tableSource = createDynamicTableSource(context, catalogTable.getResolvedTable());
    // prepare table source and convert to RelNode
    return DynamicSourceUtils.convertSourceToRel(!schemaTable.isStreamingMode(), context.getTableConfig().getConfiguration(), relBuilder, schemaTable.getContextResolvedTable(), schemaTable.getStatistic(), hints, tableSource);
}
Also used : RelOptCluster(org.apache.calcite.plan.RelOptCluster) FlinkRelBuilder(org.apache.flink.table.planner.calcite.FlinkRelBuilder) ContextResolvedTable(org.apache.flink.table.catalog.ContextResolvedTable) RelHint(org.apache.calcite.rel.hint.RelHint) FlinkContext(org.apache.flink.table.planner.calcite.FlinkContext) DynamicTableSource(org.apache.flink.table.connector.source.DynamicTableSource)

Example 2 with RelHint

use of org.apache.calcite.rel.hint.RelHint in project flink by apache.

the class SqlToOperationConverter method convertSqlInsert.

/**
 * Convert insert into statement.
 */
private Operation convertSqlInsert(RichSqlInsert insert) {
    // Get sink table name.
    List<String> targetTablePath = ((SqlIdentifier) insert.getTargetTableID()).names;
    // Get sink table hints.
    HintStrategyTable hintStrategyTable = flinkPlanner.config().getSqlToRelConverterConfig().getHintStrategyTable();
    List<RelHint> tableHints = SqlUtil.getRelHint(hintStrategyTable, insert.getTableHints());
    Map<String, String> dynamicOptions = FlinkHints.getHintedOptions(tableHints);
    UnresolvedIdentifier unresolvedIdentifier = UnresolvedIdentifier.of(targetTablePath);
    ObjectIdentifier identifier = catalogManager.qualifyIdentifier(unresolvedIdentifier);
    ContextResolvedTable contextResolvedTable = catalogManager.getTableOrError(identifier);
    PlannerQueryOperation query = (PlannerQueryOperation) convertValidatedSqlNodeOrFail(flinkPlanner, catalogManager, insert.getSource());
    return new SinkModifyOperation(contextResolvedTable, query, insert.getStaticPartitionKVs(), insert.isOverwrite(), dynamicOptions);
}
Also used : SinkModifyOperation(org.apache.flink.table.operations.SinkModifyOperation) UnresolvedIdentifier(org.apache.flink.table.catalog.UnresolvedIdentifier) ContextResolvedTable(org.apache.flink.table.catalog.ContextResolvedTable) SqlIdentifier(org.apache.calcite.sql.SqlIdentifier) RelHint(org.apache.calcite.rel.hint.RelHint) HintStrategyTable(org.apache.calcite.rel.hint.HintStrategyTable) ObjectIdentifier(org.apache.flink.table.catalog.ObjectIdentifier)

Example 3 with RelHint

use of org.apache.calcite.rel.hint.RelHint in project flink by apache.

the class DynamicSinkUtils method convertSinkToRel.

private static RelNode convertSinkToRel(FlinkRelBuilder relBuilder, RelNode input, Map<String, String> dynamicOptions, ContextResolvedTable contextResolvedTable, Map<String, String> staticPartitions, boolean isOverwrite, DynamicTableSink sink) {
    final DataTypeFactory dataTypeFactory = unwrapContext(relBuilder).getCatalogManager().getDataTypeFactory();
    final FlinkTypeFactory typeFactory = unwrapTypeFactory(relBuilder);
    final ResolvedSchema schema = contextResolvedTable.getResolvedSchema();
    final String tableDebugName = contextResolvedTable.getIdentifier().asSummaryString();
    List<SinkAbilitySpec> sinkAbilitySpecs = new ArrayList<>();
    // 1. prepare table sink
    prepareDynamicSink(tableDebugName, staticPartitions, isOverwrite, sink, contextResolvedTable.getResolvedTable(), sinkAbilitySpecs);
    sinkAbilitySpecs.forEach(spec -> spec.apply(sink));
    // 2. validate the query schema to the sink's table schema and apply cast if possible
    final RelNode query = validateSchemaAndApplyImplicitCast(input, schema, tableDebugName, dataTypeFactory, typeFactory);
    relBuilder.push(query);
    // 3. convert the sink's table schema to the consumed data type of the sink
    final List<Integer> metadataColumns = extractPersistedMetadataColumns(schema);
    if (!metadataColumns.isEmpty()) {
        pushMetadataProjection(relBuilder, typeFactory, schema, sink);
    }
    List<RelHint> hints = new ArrayList<>();
    if (!dynamicOptions.isEmpty()) {
        hints.add(RelHint.builder("OPTIONS").hintOptions(dynamicOptions).build());
    }
    final RelNode finalQuery = relBuilder.build();
    return LogicalSink.create(finalQuery, hints, contextResolvedTable, sink, staticPartitions, sinkAbilitySpecs.toArray(new SinkAbilitySpec[0]));
}
Also used : FlinkTypeFactory(org.apache.flink.table.planner.calcite.FlinkTypeFactory) RelNode(org.apache.calcite.rel.RelNode) ArrayList(java.util.ArrayList) DataTypeFactory(org.apache.flink.table.catalog.DataTypeFactory) SinkAbilitySpec(org.apache.flink.table.planner.plan.abilities.sink.SinkAbilitySpec) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) RelHint(org.apache.calcite.rel.hint.RelHint)

Aggregations

RelHint (org.apache.calcite.rel.hint.RelHint)3 ContextResolvedTable (org.apache.flink.table.catalog.ContextResolvedTable)2 ArrayList (java.util.ArrayList)1 RelOptCluster (org.apache.calcite.plan.RelOptCluster)1 RelNode (org.apache.calcite.rel.RelNode)1 HintStrategyTable (org.apache.calcite.rel.hint.HintStrategyTable)1 SqlIdentifier (org.apache.calcite.sql.SqlIdentifier)1 DataTypeFactory (org.apache.flink.table.catalog.DataTypeFactory)1 ObjectIdentifier (org.apache.flink.table.catalog.ObjectIdentifier)1 ResolvedSchema (org.apache.flink.table.catalog.ResolvedSchema)1 UnresolvedIdentifier (org.apache.flink.table.catalog.UnresolvedIdentifier)1 DynamicTableSource (org.apache.flink.table.connector.source.DynamicTableSource)1 SinkModifyOperation (org.apache.flink.table.operations.SinkModifyOperation)1 FlinkContext (org.apache.flink.table.planner.calcite.FlinkContext)1 FlinkRelBuilder (org.apache.flink.table.planner.calcite.FlinkRelBuilder)1 FlinkTypeFactory (org.apache.flink.table.planner.calcite.FlinkTypeFactory)1 SinkAbilitySpec (org.apache.flink.table.planner.plan.abilities.sink.SinkAbilitySpec)1