Search in sources :

Example 1 with SinkRuntimeProviderContext

use of org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext in project flink by apache.

the class KinesisFirehoseDynamicTableFactoryTest method testGoodTableSink.

@Test
public void testGoodTableSink() {
    ResolvedSchema sinkSchema = defaultSinkSchema();
    Map<String, String> sinkOptions = defaultTableOptions().build();
    // Construct actual DynamicTableSink using FactoryUtil
    KinesisFirehoseDynamicSink actualSink = (KinesisFirehoseDynamicSink) createTableSink(sinkSchema, sinkOptions);
    // Construct expected DynamicTableSink using factory under test
    KinesisFirehoseDynamicSink expectedSink = new KinesisFirehoseDynamicSink.KinesisDataFirehoseDynamicSinkBuilder().setConsumedDataType(sinkSchema.toPhysicalRowDataType()).setDeliveryStream(DELIVERY_STREAM_NAME).setFirehoseClientProperties(defaultSinkProperties()).setEncodingFormat(new TestFormatFactory.EncodingFormatMock(",")).build();
    Assertions.assertThat(actualSink).isEqualTo(expectedSink);
    // verify the produced sink
    DynamicTableSink.SinkRuntimeProvider sinkFunctionProvider = actualSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    Sink<RowData> sinkFunction = ((SinkV2Provider) sinkFunctionProvider).createSink();
    Assertions.assertThat(sinkFunction).isInstanceOf(KinesisFirehoseSink.class);
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) RowData(org.apache.flink.table.data.RowData) SinkV2Provider(org.apache.flink.table.connector.sink.SinkV2Provider) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 2 with SinkRuntimeProviderContext

use of org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext in project flink by apache.

the class KinesisFirehoseDynamicTableFactoryTest method testGoodTableSinkWithSinkOptions.

@Test
public void testGoodTableSinkWithSinkOptions() {
    ResolvedSchema sinkSchema = defaultSinkSchema();
    Map<String, String> sinkOptions = defaultTableOptionsWithSinkOptions().build();
    // Construct actual DynamicTableSink using FactoryUtil
    KinesisFirehoseDynamicSink actualSink = (KinesisFirehoseDynamicSink) createTableSink(sinkSchema, sinkOptions);
    // Construct expected DynamicTableSink using factory under test
    KinesisFirehoseDynamicSink expectedSink = getDefaultSinkBuilder().setConsumedDataType(sinkSchema.toPhysicalRowDataType()).setDeliveryStream(DELIVERY_STREAM_NAME).setFirehoseClientProperties(defaultSinkProperties()).setEncodingFormat(new TestFormatFactory.EncodingFormatMock(",")).build();
    Assertions.assertThat(actualSink).isEqualTo(expectedSink);
    // verify the produced sink
    DynamicTableSink.SinkRuntimeProvider sinkFunctionProvider = actualSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    Sink<RowData> sinkFunction = ((SinkV2Provider) sinkFunctionProvider).createSink();
    Assertions.assertThat(sinkFunction).isInstanceOf(KinesisFirehoseSink.class);
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) RowData(org.apache.flink.table.data.RowData) SinkV2Provider(org.apache.flink.table.connector.sink.SinkV2Provider) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 3 with SinkRuntimeProviderContext

use of org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext in project flink by apache.

the class KinesisDynamicTableSinkFactoryTest method testGoodTableSinkForNonPartitionedTable.

@Test
public void testGoodTableSinkForNonPartitionedTable() {
    ResolvedSchema sinkSchema = defaultSinkSchema();
    Map<String, String> sinkOptions = defaultTableOptions().build();
    // Construct actual DynamicTableSink using FactoryUtil
    KinesisDynamicSink actualSink = (KinesisDynamicSink) createTableSink(sinkSchema, sinkOptions);
    // Construct expected DynamicTableSink using factory under test
    KinesisDynamicSink expectedSink = (KinesisDynamicSink) new KinesisDynamicSink.KinesisDynamicTableSinkBuilder().setConsumedDataType(sinkSchema.toPhysicalRowDataType()).setStream(STREAM_NAME).setKinesisClientProperties(defaultProducerProperties()).setEncodingFormat(new TestFormatFactory.EncodingFormatMock(",")).setPartitioner(new RandomKinesisPartitionKeyGenerator<>()).build();
    Assertions.assertThat(actualSink).isEqualTo(expectedSink);
    // verify the produced sink
    DynamicTableSink.SinkRuntimeProvider sinkFunctionProvider = actualSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    Sink<RowData> sinkFunction = ((SinkV2Provider) sinkFunctionProvider).createSink();
    Assertions.assertThat(sinkFunction).isInstanceOf(KinesisDataStreamsSink.class);
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) RowData(org.apache.flink.table.data.RowData) SinkV2Provider(org.apache.flink.table.connector.sink.SinkV2Provider) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 4 with SinkRuntimeProviderContext

use of org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext in project flink by apache.

the class KinesisDynamicTableSinkFactoryTest method testGoodTableSinkForNonPartitionedTableWithSinkOptions.

@Test
public void testGoodTableSinkForNonPartitionedTableWithSinkOptions() {
    ResolvedSchema sinkSchema = defaultSinkSchema();
    Map<String, String> sinkOptions = defaultTableOptionsWithSinkOptions().build();
    // Construct actual DynamicTableSink using FactoryUtil
    KinesisDynamicSink actualSink = (KinesisDynamicSink) createTableSink(sinkSchema, sinkOptions);
    // Construct expected DynamicTableSink using factory under test
    KinesisDynamicSink expectedSink = (KinesisDynamicSink) getDefaultSinkBuilder().setConsumedDataType(sinkSchema.toPhysicalRowDataType()).setStream(STREAM_NAME).setKinesisClientProperties(defaultProducerProperties()).setEncodingFormat(new TestFormatFactory.EncodingFormatMock(",")).setPartitioner(new RandomKinesisPartitionKeyGenerator<>()).build();
    Assertions.assertThat(actualSink).isEqualTo(expectedSink);
    // verify the produced sink
    DynamicTableSink.SinkRuntimeProvider sinkFunctionProvider = actualSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    Sink<RowData> sinkFunction = ((SinkV2Provider) sinkFunctionProvider).createSink();
    Assertions.assertThat(sinkFunction).isInstanceOf(KinesisDataStreamsSink.class);
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) RowData(org.apache.flink.table.data.RowData) SinkV2Provider(org.apache.flink.table.connector.sink.SinkV2Provider) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 5 with SinkRuntimeProviderContext

use of org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext in project flink by apache.

the class KinesisDynamicTableSinkFactoryTest method testGoodTableSinkForPartitionedTable.

@Test
public void testGoodTableSinkForPartitionedTable() {
    ResolvedSchema sinkSchema = defaultSinkSchema();
    DataType physicalDataType = sinkSchema.toPhysicalRowDataType();
    Map<String, String> sinkOptions = defaultTableOptions().build();
    List<String> sinkPartitionKeys = Arrays.asList("name", "curr_id");
    // Construct actual DynamicTableSink using FactoryUtil
    KinesisDynamicSink actualSink = (KinesisDynamicSink) createTableSink(sinkSchema, sinkPartitionKeys, sinkOptions);
    // Construct expected DynamicTableSink using factory under test
    KinesisDynamicSink expectedSink = (KinesisDynamicSink) new KinesisDynamicSink.KinesisDynamicTableSinkBuilder().setConsumedDataType(physicalDataType).setStream(STREAM_NAME).setKinesisClientProperties(defaultProducerProperties()).setEncodingFormat(new TestFormatFactory.EncodingFormatMock(",")).setPartitioner(new RowDataFieldsKinesisPartitionKeyGenerator((RowType) physicalDataType.getLogicalType(), sinkPartitionKeys)).build();
    // verify that the constructed DynamicTableSink is as expected
    Assertions.assertThat(actualSink).isEqualTo(expectedSink);
    // verify the produced sink
    DynamicTableSink.SinkRuntimeProvider sinkFunctionProvider = actualSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    Sink<RowData> sinkFunction = ((SinkV2Provider) sinkFunctionProvider).createSink();
    Assertions.assertThat(sinkFunction).isInstanceOf(KinesisDataStreamsSink.class);
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) RowType(org.apache.flink.table.types.logical.RowType) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) RowData(org.apache.flink.table.data.RowData) DataType(org.apache.flink.table.types.DataType) SinkV2Provider(org.apache.flink.table.connector.sink.SinkV2Provider) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Aggregations

SinkRuntimeProviderContext (org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext)27 DynamicTableSink (org.apache.flink.table.connector.sink.DynamicTableSink)23 RowData (org.apache.flink.table.data.RowData)19 Test (org.junit.Test)17 ResolvedSchema (org.apache.flink.table.catalog.ResolvedSchema)12 SinkV2Provider (org.apache.flink.table.connector.sink.SinkV2Provider)11 TestDynamicTableFactory (org.apache.flink.table.factories.TestDynamicTableFactory)7 HashMap (java.util.HashMap)5 RowType (org.apache.flink.table.types.logical.RowType)5 SinkFunctionProvider (org.apache.flink.table.connector.sink.SinkFunctionProvider)4 SerializationSchema (org.apache.flink.api.common.serialization.SerializationSchema)3 AvroRowDataSerializationSchema (org.apache.flink.formats.avro.AvroRowDataSerializationSchema)3 ConfluentRegistryAvroSerializationSchema (org.apache.flink.formats.avro.registry.confluent.ConfluentRegistryAvroSerializationSchema)3 DynamicTableSource (org.apache.flink.table.connector.source.DynamicTableSource)3 BinaryRowData (org.apache.flink.table.data.binary.BinaryRowData)3 EncodingFormatMock (org.apache.flink.table.factories.TestFormatFactory.EncodingFormatMock)3 Test (org.junit.jupiter.api.Test)3 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)3 Transformation (org.apache.flink.api.dag.Transformation)2 Configuration (org.apache.flink.configuration.Configuration)2