Search in sources :

Example 1 with PlannerMocks

use of org.apache.flink.table.planner.utils.PlannerMocks in project flink by apache.

the class DynamicTableSourceSpecSerdeTest method testDynamicTableSourceSpecSerde.

@ParameterizedTest
@MethodSource("testDynamicTableSinkSpecSerde")
void testDynamicTableSourceSpecSerde(DynamicTableSourceSpec spec) throws IOException {
    PlannerMocks plannerMocks = PlannerMocks.create();
    CatalogManager catalogManager = plannerMocks.getCatalogManager();
    catalogManager.createTable(spec.getContextResolvedTable().getResolvedTable(), spec.getContextResolvedTable().getIdentifier(), false);
    SerdeContext serdeCtx = configuredSerdeContext(catalogManager, plannerMocks.getTableConfig());
    // Re-init the spec to be permanent with correct catalog
    spec = new DynamicTableSourceSpec(ContextResolvedTable.permanent(spec.getContextResolvedTable().getIdentifier(), catalogManager.getCatalog(catalogManager.getCurrentCatalog()).get(), spec.getContextResolvedTable().getResolvedTable()), spec.getSourceAbilities());
    String actualJson = toJson(serdeCtx, spec);
    DynamicTableSourceSpec actual = toObject(serdeCtx, actualJson, DynamicTableSourceSpec.class);
    assertThat(actual.getContextResolvedTable()).isEqualTo(spec.getContextResolvedTable());
    assertThat(actual.getSourceAbilities()).isEqualTo(spec.getSourceAbilities());
    assertThat(actual.getScanTableSource(plannerMocks.getPlannerContext().getFlinkContext())).isNotNull();
}
Also used : JsonSerdeTestUtil.configuredSerdeContext(org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeTestUtil.configuredSerdeContext) DynamicTableSourceSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSourceSpec) PlannerMocks(org.apache.flink.table.planner.utils.PlannerMocks) CatalogManager(org.apache.flink.table.catalog.CatalogManager) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 2 with PlannerMocks

use of org.apache.flink.table.planner.utils.PlannerMocks in project flink by apache.

the class DynamicTableSourceSpecSerdeTest method testDynamicTableSourceSpecSerdeWithEnrichmentOptions.

@Test
void testDynamicTableSourceSpecSerdeWithEnrichmentOptions() throws Exception {
    // Test model
    ObjectIdentifier identifier = ObjectIdentifier.of(DEFAULT_BUILTIN_CATALOG, DEFAULT_BUILTIN_DATABASE, "my_table");
    String formatPrefix = FactoryUtil.getFormatPrefix(FORMAT, TestFormatFactory.IDENTIFIER);
    Map<String, String> planOptions = new HashMap<>();
    planOptions.put(CONNECTOR.key(), TestDynamicTableFactory.IDENTIFIER);
    planOptions.put(TARGET.key(), "abc");
    planOptions.put(PASSWORD.key(), "abc");
    planOptions.put(FORMAT.key(), TestFormatFactory.IDENTIFIER);
    planOptions.put(formatPrefix + DELIMITER.key(), "|");
    Map<String, String> catalogOptions = new HashMap<>();
    catalogOptions.put(CONNECTOR.key(), TestDynamicTableFactory.IDENTIFIER);
    catalogOptions.put(TARGET.key(), "abc");
    catalogOptions.put(PASSWORD.key(), "xyz");
    catalogOptions.put(FORMAT.key(), TestFormatFactory.IDENTIFIER);
    catalogOptions.put(formatPrefix + DELIMITER.key(), ",");
    ResolvedCatalogTable planResolvedCatalogTable = tableWithOnlyPhysicalColumns(planOptions);
    ResolvedCatalogTable catalogResolvedCatalogTable = tableWithOnlyPhysicalColumns(catalogOptions);
    // Create planner mocks
    PlannerMocks plannerMocks = PlannerMocks.create(new Configuration().set(PLAN_RESTORE_CATALOG_OBJECTS, CatalogPlanRestore.ALL).set(PLAN_COMPILE_CATALOG_OBJECTS, CatalogPlanCompilation.ALL));
    CatalogManager catalogManager = plannerMocks.getCatalogManager();
    catalogManager.createTable(catalogResolvedCatalogTable, identifier, false);
    // Mock the context
    SerdeContext serdeCtx = configuredSerdeContext(catalogManager, plannerMocks.getTableConfig());
    DynamicTableSourceSpec planSpec = new DynamicTableSourceSpec(ContextResolvedTable.permanent(identifier, catalogManager.getCatalog(catalogManager.getCurrentCatalog()).get(), planResolvedCatalogTable), Collections.emptyList());
    String actualJson = toJson(serdeCtx, planSpec);
    DynamicTableSourceSpec actual = toObject(serdeCtx, actualJson, DynamicTableSourceSpec.class);
    assertThat(actual.getContextResolvedTable()).isEqualTo(planSpec.getContextResolvedTable());
    assertThat(actual.getSourceAbilities()).isNull();
    TestDynamicTableFactory.DynamicTableSourceMock dynamicTableSource = (TestDynamicTableFactory.DynamicTableSourceMock) actual.getScanTableSource(plannerMocks.getPlannerContext().getFlinkContext());
    assertThat(dynamicTableSource.password).isEqualTo("xyz");
    assertThat(((TestFormatFactory.DecodingFormatMock) dynamicTableSource.valueFormat).delimiter).isEqualTo(",");
}
Also used : Configuration(org.apache.flink.configuration.Configuration) HashMap(java.util.HashMap) JsonSerdeTestUtil.configuredSerdeContext(org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeTestUtil.configuredSerdeContext) DynamicTableSourceSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSourceSpec) TestDynamicTableFactory(org.apache.flink.table.factories.TestDynamicTableFactory) CatalogManager(org.apache.flink.table.catalog.CatalogManager) ResolvedCatalogTable(org.apache.flink.table.catalog.ResolvedCatalogTable) PlannerMocks(org.apache.flink.table.planner.utils.PlannerMocks) ObjectIdentifier(org.apache.flink.table.catalog.ObjectIdentifier) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 3 with PlannerMocks

use of org.apache.flink.table.planner.utils.PlannerMocks in project flink by apache.

the class DynamicTableSinkSpecSerdeTest method testDynamicTableSinkSpecSerde.

@ParameterizedTest
@MethodSource("testDynamicTableSinkSpecSerde")
void testDynamicTableSinkSpecSerde(DynamicTableSinkSpec spec) throws IOException {
    PlannerMocks plannerMocks = PlannerMocks.create();
    CatalogManager catalogManager = plannerMocks.getCatalogManager();
    catalogManager.createTable(spec.getContextResolvedTable().getResolvedTable(), spec.getContextResolvedTable().getIdentifier(), false);
    SerdeContext serdeCtx = configuredSerdeContext(catalogManager, plannerMocks.getTableConfig());
    // Re-init the spec to be permanent with correct catalog
    spec = new DynamicTableSinkSpec(ContextResolvedTable.permanent(spec.getContextResolvedTable().getIdentifier(), catalogManager.getCatalog(catalogManager.getCurrentCatalog()).get(), spec.getContextResolvedTable().getResolvedTable()), spec.getSinkAbilities());
    String actualJson = toJson(serdeCtx, spec);
    DynamicTableSinkSpec actual = toObject(serdeCtx, actualJson, DynamicTableSinkSpec.class);
    assertThat(actual.getContextResolvedTable()).isEqualTo(spec.getContextResolvedTable());
    assertThat(actual.getSinkAbilities()).isEqualTo(spec.getSinkAbilities());
    assertThat(actual.getTableSink(plannerMocks.getPlannerContext().getFlinkContext())).isNotNull();
}
Also used : JsonSerdeTestUtil.configuredSerdeContext(org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeTestUtil.configuredSerdeContext) DynamicTableSinkSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSinkSpec) PlannerMocks(org.apache.flink.table.planner.utils.PlannerMocks) CatalogManager(org.apache.flink.table.catalog.CatalogManager) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 4 with PlannerMocks

use of org.apache.flink.table.planner.utils.PlannerMocks in project flink by apache.

the class DynamicTableSinkSpecSerdeTest method testDynamicTableSinkSpecSerdeWithEnrichmentOptions.

@Test
void testDynamicTableSinkSpecSerdeWithEnrichmentOptions() throws Exception {
    // Test model
    ObjectIdentifier identifier = ObjectIdentifier.of(DEFAULT_BUILTIN_CATALOG, DEFAULT_BUILTIN_DATABASE, "my_table");
    String formatPrefix = FactoryUtil.getFormatPrefix(FORMAT, TestFormatFactory.IDENTIFIER);
    Map<String, String> planOptions = new HashMap<>();
    planOptions.put(CONNECTOR.key(), TestDynamicTableFactory.IDENTIFIER);
    planOptions.put(TARGET.key(), "abc");
    planOptions.put(BUFFER_SIZE.key(), "1000");
    planOptions.put(FORMAT.key(), TestFormatFactory.IDENTIFIER);
    planOptions.put(formatPrefix + DELIMITER.key(), "|");
    Map<String, String> catalogOptions = new HashMap<>();
    catalogOptions.put(CONNECTOR.key(), TestDynamicTableFactory.IDENTIFIER);
    catalogOptions.put(TARGET.key(), "xyz");
    catalogOptions.put(BUFFER_SIZE.key(), "2000");
    catalogOptions.put(FORMAT.key(), TestFormatFactory.IDENTIFIER);
    catalogOptions.put(formatPrefix + DELIMITER.key(), ",");
    ResolvedCatalogTable planResolvedCatalogTable = tableWithOnlyPhysicalColumns(planOptions);
    ResolvedCatalogTable catalogResolvedCatalogTable = tableWithOnlyPhysicalColumns(catalogOptions);
    // Create planner mocks
    PlannerMocks plannerMocks = PlannerMocks.create(new Configuration().set(PLAN_RESTORE_CATALOG_OBJECTS, CatalogPlanRestore.ALL).set(PLAN_COMPILE_CATALOG_OBJECTS, CatalogPlanCompilation.ALL));
    CatalogManager catalogManager = plannerMocks.getCatalogManager();
    catalogManager.createTable(catalogResolvedCatalogTable, identifier, false);
    // Mock the context
    SerdeContext serdeCtx = configuredSerdeContext(catalogManager, plannerMocks.getTableConfig());
    DynamicTableSinkSpec planSpec = new DynamicTableSinkSpec(ContextResolvedTable.permanent(identifier, catalogManager.getCatalog(catalogManager.getCurrentCatalog()).get(), planResolvedCatalogTable), Collections.emptyList());
    String actualJson = toJson(serdeCtx, planSpec);
    DynamicTableSinkSpec actual = toObject(serdeCtx, actualJson, DynamicTableSinkSpec.class);
    assertThat(actual.getContextResolvedTable()).isEqualTo(planSpec.getContextResolvedTable());
    assertThat(actual.getSinkAbilities()).isNull();
    TestDynamicTableFactory.DynamicTableSinkMock dynamicTableSink = (TestDynamicTableFactory.DynamicTableSinkMock) actual.getTableSink(plannerMocks.getPlannerContext().getFlinkContext());
    assertThat(dynamicTableSink.target).isEqualTo("abc");
    assertThat(dynamicTableSink.bufferSize).isEqualTo(2000);
    assertThat(((TestFormatFactory.EncodingFormatMock) dynamicTableSink.valueFormat).delimiter).isEqualTo(",");
}
Also used : Configuration(org.apache.flink.configuration.Configuration) HashMap(java.util.HashMap) JsonSerdeTestUtil.configuredSerdeContext(org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeTestUtil.configuredSerdeContext) DynamicTableSinkSpec(org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSinkSpec) TestDynamicTableFactory(org.apache.flink.table.factories.TestDynamicTableFactory) CatalogManager(org.apache.flink.table.catalog.CatalogManager) ResolvedCatalogTable(org.apache.flink.table.catalog.ResolvedCatalogTable) PlannerMocks(org.apache.flink.table.planner.utils.PlannerMocks) ObjectIdentifier(org.apache.flink.table.catalog.ObjectIdentifier) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Aggregations

CatalogManager (org.apache.flink.table.catalog.CatalogManager)4 JsonSerdeTestUtil.configuredSerdeContext (org.apache.flink.table.planner.plan.nodes.exec.serde.JsonSerdeTestUtil.configuredSerdeContext)4 PlannerMocks (org.apache.flink.table.planner.utils.PlannerMocks)4 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)4 HashMap (java.util.HashMap)2 Configuration (org.apache.flink.configuration.Configuration)2 ObjectIdentifier (org.apache.flink.table.catalog.ObjectIdentifier)2 ResolvedCatalogTable (org.apache.flink.table.catalog.ResolvedCatalogTable)2 TestDynamicTableFactory (org.apache.flink.table.factories.TestDynamicTableFactory)2 DynamicTableSinkSpec (org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSinkSpec)2 DynamicTableSourceSpec (org.apache.flink.table.planner.plan.nodes.exec.spec.DynamicTableSourceSpec)2 Test (org.junit.jupiter.api.Test)2 MethodSource (org.junit.jupiter.params.provider.MethodSource)2