Search in sources :

Example 76 with ResolvedSchema

use of org.apache.flink.table.catalog.ResolvedSchema in project flink by apache.

the class FlinkCalciteCatalogReaderTest method testGetFlinkPreparingTableBase.

@Test
public void testGetFlinkPreparingTableBase() {
    // Mock CatalogSchemaTable.
    final ObjectIdentifier objectIdentifier = ObjectIdentifier.of("a", "b", "c");
    final ResolvedSchema schema = new ResolvedSchema(Collections.emptyList(), Collections.emptyList(), null);
    final CatalogTable catalogTable = ConnectorCatalogTable.source(new TestTableSource(true, TableSchema.fromResolvedSchema(schema)), true);
    final ResolvedCatalogTable resolvedCatalogTable = new ResolvedCatalogTable(catalogTable, schema);
    CatalogSchemaTable mockTable = new CatalogSchemaTable(ContextResolvedTable.permanent(objectIdentifier, CatalogManagerMocks.createEmptyCatalog(), resolvedCatalogTable), FlinkStatistic.UNKNOWN(), true);
    rootSchemaPlus.add(tableMockName, mockTable);
    Prepare.PreparingTable preparingTable = catalogReader.getTable(Collections.singletonList(tableMockName));
    assertTrue(preparingTable instanceof FlinkPreparingTableBase);
}
Also used : TestTableSource(org.apache.flink.table.planner.utils.TestTableSource) ResolvedCatalogTable(org.apache.flink.table.catalog.ResolvedCatalogTable) CatalogSchemaTable(org.apache.flink.table.planner.catalog.CatalogSchemaTable) Prepare(org.apache.calcite.prepare.Prepare) FlinkPreparingTableBase(org.apache.flink.table.planner.plan.schema.FlinkPreparingTableBase) ConnectorCatalogTable(org.apache.flink.table.catalog.ConnectorCatalogTable) CatalogTable(org.apache.flink.table.catalog.CatalogTable) ResolvedCatalogTable(org.apache.flink.table.catalog.ResolvedCatalogTable) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) ObjectIdentifier(org.apache.flink.table.catalog.ObjectIdentifier) Test(org.junit.Test)

Example 77 with ResolvedSchema

use of org.apache.flink.table.catalog.ResolvedSchema in project flink by apache.

the class DataTypeUtilsTest method testExpandRowType.

@Test
public void testExpandRowType() {
    DataType dataType = ROW(FIELD("f0", INT()), FIELD("f1", STRING()), FIELD("f2", TIMESTAMP(5).bridgedTo(Timestamp.class)), FIELD("f3", TIMESTAMP(3)));
    ResolvedSchema schema = DataTypeUtils.expandCompositeTypeToSchema(dataType);
    assertThat(schema).isEqualTo(ResolvedSchema.of(Column.physical("f0", INT()), Column.physical("f1", STRING()), Column.physical("f2", TIMESTAMP(5).bridgedTo(Timestamp.class)), Column.physical("f3", TIMESTAMP(3).bridgedTo(LocalDateTime.class))));
}
Also used : LocalDateTime(java.time.LocalDateTime) DataType(org.apache.flink.table.types.DataType) FieldsDataType(org.apache.flink.table.types.FieldsDataType) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Timestamp(java.sql.Timestamp) Test(org.junit.Test)

Example 78 with ResolvedSchema

use of org.apache.flink.table.catalog.ResolvedSchema in project flink by apache.

the class DataTypeUtilsTest method testExpandLegacyCompositeType.

@Test
public void testExpandLegacyCompositeType() {
    DataType dataType = TypeConversions.fromLegacyInfoToDataType(new TupleTypeInfo<>(Types.STRING, Types.INT, Types.SQL_TIMESTAMP));
    ResolvedSchema schema = DataTypeUtils.expandCompositeTypeToSchema(dataType);
    assertThat(schema).isEqualTo(ResolvedSchema.of(Column.physical("f0", STRING()), Column.physical("f1", INT()), Column.physical("f2", TIMESTAMP(3).bridgedTo(Timestamp.class))));
}
Also used : DataType(org.apache.flink.table.types.DataType) FieldsDataType(org.apache.flink.table.types.FieldsDataType) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Timestamp(java.sql.Timestamp) Test(org.junit.Test)

Example 79 with ResolvedSchema

use of org.apache.flink.table.catalog.ResolvedSchema in project flink by apache.

the class PushWatermarkIntoTableSourceScanRuleBase method hasSourceWatermarkDeclaration.

private boolean hasSourceWatermarkDeclaration(TableSourceTable table) {
    final ResolvedSchema schema = table.contextResolvedTable().getResolvedSchema();
    final List<WatermarkSpec> specs = schema.getWatermarkSpecs();
    // we only support one watermark spec for now
    if (specs.size() != 1) {
        return false;
    }
    final ResolvedExpression watermarkExpr = specs.get(0).getWatermarkExpression();
    final FunctionDefinition function = unwrapFunctionDefinition(watermarkExpr);
    return function == BuiltInFunctionDefinitions.SOURCE_WATERMARK;
}
Also used : WatermarkSpec(org.apache.flink.table.catalog.WatermarkSpec) SourceWatermarkSpec(org.apache.flink.table.planner.plan.abilities.source.SourceWatermarkSpec) ResolvedExpression(org.apache.flink.table.expressions.ResolvedExpression) ShortcutUtils.unwrapFunctionDefinition(org.apache.flink.table.planner.utils.ShortcutUtils.unwrapFunctionDefinition) FunctionDefinition(org.apache.flink.table.functions.FunctionDefinition) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema)

Example 80 with ResolvedSchema

use of org.apache.flink.table.catalog.ResolvedSchema in project flink by apache.

the class PushProjectIntoTableSourceScanRule method getPrimaryKeyProjections.

private List<RexNode> getPrimaryKeyProjections(LogicalTableScan scan) {
    final TableSourceTable source = scan.getTable().unwrap(TableSourceTable.class);
    final ResolvedSchema schema = source.contextResolvedTable().getResolvedSchema();
    if (!schema.getPrimaryKey().isPresent()) {
        return Collections.emptyList();
    }
    final FlinkTypeFactory typeFactory = unwrapTypeFactory(scan);
    final UniqueConstraint primaryKey = schema.getPrimaryKey().get();
    return primaryKey.getColumns().stream().map(columnName -> {
        final int idx = scan.getRowType().getFieldNames().indexOf(columnName);
        final Column column = schema.getColumn(idx).orElseThrow(() -> new TableException(String.format("Column at index %d not found.", idx)));
        return new RexInputRef(idx, typeFactory.createFieldTypeFromLogicalType(column.getDataType().getLogicalType()));
    }).collect(Collectors.toList());
}
Also used : IntStream(java.util.stream.IntStream) NestedProjectionUtil(org.apache.flink.table.planner.plan.utils.NestedProjectionUtil) Arrays(java.util.Arrays) ShortcutUtils.unwrapTypeFactory(org.apache.flink.table.planner.utils.ShortcutUtils.unwrapTypeFactory) SourceAbilityContext(org.apache.flink.table.planner.plan.abilities.source.SourceAbilityContext) Column(org.apache.flink.table.catalog.Column) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) RexNodeExtractor(org.apache.flink.table.planner.plan.utils.RexNodeExtractor) FlinkTypeFactory(org.apache.flink.table.planner.calcite.FlinkTypeFactory) RowType(org.apache.flink.table.types.logical.RowType) SupportsProjectionPushDown(org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown) ArrayList(java.util.ArrayList) RexNode(org.apache.calcite.rex.RexNode) NestedSchema(org.apache.flink.table.planner.plan.utils.NestedSchema) Projection(org.apache.flink.table.connector.Projection) ProjectRemoveRule(org.apache.calcite.rel.rules.ProjectRemoveRule) DynamicSourceUtils.createProducedType(org.apache.flink.table.planner.connectors.DynamicSourceUtils.createProducedType) RelDataType(org.apache.calcite.rel.type.RelDataType) DynamicTableSource(org.apache.flink.table.connector.source.DynamicTableSource) TableConfig(org.apache.flink.table.api.TableConfig) LogicalProject(org.apache.calcite.rel.logical.LogicalProject) ProjectPushDownSpec(org.apache.flink.table.planner.plan.abilities.source.ProjectPushDownSpec) TableException(org.apache.flink.table.api.TableException) ShortcutUtils.unwrapContext(org.apache.flink.table.planner.utils.ShortcutUtils.unwrapContext) RelRule(org.apache.calcite.plan.RelRule) NestedColumn(org.apache.flink.table.planner.plan.utils.NestedColumn) Collectors(java.util.stream.Collectors) DynamicSourceUtils.createRequiredMetadataKeys(org.apache.flink.table.planner.connectors.DynamicSourceUtils.createRequiredMetadataKeys) SourceAbilitySpec(org.apache.flink.table.planner.plan.abilities.source.SourceAbilitySpec) TableSourceTable(org.apache.flink.table.planner.plan.schema.TableSourceTable) RelOptRuleCall(org.apache.calcite.plan.RelOptRuleCall) RexInputRef(org.apache.calcite.rex.RexInputRef) Objects(java.util.Objects) DynamicSourceUtils(org.apache.flink.table.planner.connectors.DynamicSourceUtils) RelOptRule(org.apache.calcite.plan.RelOptRule) List(java.util.List) Stream(java.util.stream.Stream) UniqueConstraint(org.apache.flink.table.catalog.UniqueConstraint) SupportsReadingMetadata(org.apache.flink.table.connector.source.abilities.SupportsReadingMetadata) ReadingMetadataSpec(org.apache.flink.table.planner.plan.abilities.source.ReadingMetadataSpec) Internal(org.apache.flink.annotation.Internal) Collections(java.util.Collections) LogicalTableScan(org.apache.calcite.rel.logical.LogicalTableScan) TableException(org.apache.flink.table.api.TableException) FlinkTypeFactory(org.apache.flink.table.planner.calcite.FlinkTypeFactory) Column(org.apache.flink.table.catalog.Column) NestedColumn(org.apache.flink.table.planner.plan.utils.NestedColumn) UniqueConstraint(org.apache.flink.table.catalog.UniqueConstraint) RexInputRef(org.apache.calcite.rex.RexInputRef) TableSourceTable(org.apache.flink.table.planner.plan.schema.TableSourceTable) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema)

Aggregations

ResolvedSchema (org.apache.flink.table.catalog.ResolvedSchema)84 Test (org.junit.Test)50 DynamicTableSink (org.apache.flink.table.connector.sink.DynamicTableSink)20 DataType (org.apache.flink.table.types.DataType)20 RowData (org.apache.flink.table.data.RowData)17 ValidationException (org.apache.flink.table.api.ValidationException)14 ResolvedCatalogTable (org.apache.flink.table.catalog.ResolvedCatalogTable)14 List (java.util.List)11 SinkRuntimeProviderContext (org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext)11 DynamicTableSource (org.apache.flink.table.connector.source.DynamicTableSource)10 Column (org.apache.flink.table.catalog.Column)9 LogicalType (org.apache.flink.table.types.logical.LogicalType)9 RowType (org.apache.flink.table.types.logical.RowType)9 HashMap (java.util.HashMap)8 Collectors (java.util.stream.Collectors)8 RelDataType (org.apache.calcite.rel.type.RelDataType)8 Internal (org.apache.flink.annotation.Internal)8 HBaseWriteOptions (org.apache.flink.connector.hbase.options.HBaseWriteOptions)6 FlinkTypeFactory (org.apache.flink.table.planner.calcite.FlinkTypeFactory)6 Row (org.apache.flink.types.Row)6