Search in sources :

Example 1 with HBaseDynamicTableSink

use of org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink in project flink by apache.

the class HBaseDynamicTableFactoryTest method testParallelismOptions.

@Test
public void testParallelismOptions() {
    Map<String, String> options = getAllOptions();
    options.put("sink.parallelism", "2");
    ResolvedSchema schema = ResolvedSchema.of(Column.physical(ROWKEY, STRING()));
    DynamicTableSink sink = createTableSink(schema, options);
    assertTrue(sink instanceof HBaseDynamicTableSink);
    HBaseDynamicTableSink hbaseSink = (HBaseDynamicTableSink) sink;
    SinkFunctionProvider provider = (SinkFunctionProvider) hbaseSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    assertEquals(2, (long) provider.getParallelism().get());
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) HBaseDynamicTableSink(org.apache.flink.connector.hbase2.sink.HBaseDynamicTableSink) HBaseDynamicTableSink(org.apache.flink.connector.hbase2.sink.HBaseDynamicTableSink) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) SinkFunctionProvider(org.apache.flink.table.connector.sink.SinkFunctionProvider) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 2 with HBaseDynamicTableSink

use of org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink in project flink by apache.

the class HBaseDynamicTableFactoryTest method testDisabledBufferFlushOptions.

@Test
public void testDisabledBufferFlushOptions() {
    Map<String, String> options = getAllOptions();
    options.put("sink.buffer-flush.max-size", "0");
    options.put("sink.buffer-flush.max-rows", "0");
    options.put("sink.buffer-flush.interval", "0");
    ResolvedSchema schema = ResolvedSchema.of(Column.physical(ROWKEY, STRING()));
    DynamicTableSink sink = createTableSink(schema, options);
    HBaseWriteOptions expected = HBaseWriteOptions.builder().setBufferFlushMaxRows(0).setBufferFlushIntervalMillis(0).setBufferFlushMaxSizeInBytes(0).build();
    HBaseWriteOptions actual = ((HBaseDynamicTableSink) sink).getWriteOptions();
    assertEquals(expected, actual);
}
Also used : HBaseDynamicTableSink(org.apache.flink.connector.hbase2.sink.HBaseDynamicTableSink) HBaseDynamicTableSink(org.apache.flink.connector.hbase2.sink.HBaseDynamicTableSink) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) HBaseWriteOptions(org.apache.flink.connector.hbase.options.HBaseWriteOptions) Test(org.junit.Test)

Example 3 with HBaseDynamicTableSink

use of org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink in project flink by apache.

the class HBaseDynamicTableFactoryTest method testDisabledBufferFlushOptions.

@Test
public void testDisabledBufferFlushOptions() {
    Map<String, String> options = getAllOptions();
    options.put("sink.buffer-flush.max-size", "0");
    options.put("sink.buffer-flush.max-rows", "0");
    options.put("sink.buffer-flush.interval", "0");
    ResolvedSchema schema = ResolvedSchema.of(Column.physical(ROWKEY, STRING()));
    DynamicTableSink sink = createTableSink(schema, options);
    HBaseWriteOptions expected = HBaseWriteOptions.builder().setBufferFlushMaxRows(0).setBufferFlushIntervalMillis(0).setBufferFlushMaxSizeInBytes(0).build();
    HBaseWriteOptions actual = ((HBaseDynamicTableSink) sink).getWriteOptions();
    assertEquals(expected, actual);
}
Also used : HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) HBaseWriteOptions(org.apache.flink.connector.hbase.options.HBaseWriteOptions) Test(org.junit.Test)

Example 4 with HBaseDynamicTableSink

use of org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink in project flink by apache.

the class HBaseDynamicTableFactoryTest method testParallelismOptions.

@Test
public void testParallelismOptions() {
    Map<String, String> options = getAllOptions();
    options.put("sink.parallelism", "2");
    ResolvedSchema schema = ResolvedSchema.of(Column.physical(ROWKEY, STRING()));
    DynamicTableSink sink = createTableSink(schema, options);
    assertTrue(sink instanceof HBaseDynamicTableSink);
    HBaseDynamicTableSink hbaseSink = (HBaseDynamicTableSink) sink;
    SinkFunctionProvider provider = (SinkFunctionProvider) hbaseSink.getSinkRuntimeProvider(new SinkRuntimeProviderContext(false));
    assertEquals(2, (long) provider.getParallelism().get());
}
Also used : SinkRuntimeProviderContext(org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext) HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) SinkFunctionProvider(org.apache.flink.table.connector.sink.SinkFunctionProvider) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) Test(org.junit.Test)

Example 5 with HBaseDynamicTableSink

use of org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink in project flink by apache.

the class HBaseDynamicTableFactoryTest method testBufferFlushOptions.

@Test
public void testBufferFlushOptions() {
    Map<String, String> options = getAllOptions();
    options.put("sink.buffer-flush.max-size", "10mb");
    options.put("sink.buffer-flush.max-rows", "100");
    options.put("sink.buffer-flush.interval", "10s");
    ResolvedSchema schema = ResolvedSchema.of(Column.physical(ROWKEY, STRING()));
    DynamicTableSink sink = createTableSink(schema, options);
    HBaseWriteOptions expected = HBaseWriteOptions.builder().setBufferFlushMaxRows(100).setBufferFlushIntervalMillis(10 * 1000).setBufferFlushMaxSizeInBytes(10 * 1024 * 1024).build();
    HBaseWriteOptions actual = ((HBaseDynamicTableSink) sink).getWriteOptions();
    assertEquals(expected, actual);
}
Also used : HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) HBaseDynamicTableSink(org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink) DynamicTableSink(org.apache.flink.table.connector.sink.DynamicTableSink) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) HBaseWriteOptions(org.apache.flink.connector.hbase.options.HBaseWriteOptions) Test(org.junit.Test)

Aggregations

HBaseWriteOptions (org.apache.flink.connector.hbase.options.HBaseWriteOptions)8 ResolvedSchema (org.apache.flink.table.catalog.ResolvedSchema)8 DynamicTableSink (org.apache.flink.table.connector.sink.DynamicTableSink)8 Test (org.junit.Test)8 HBaseDynamicTableSink (org.apache.flink.connector.hbase1.sink.HBaseDynamicTableSink)5 HBaseDynamicTableSink (org.apache.flink.connector.hbase2.sink.HBaseDynamicTableSink)5 HBaseTableSchema (org.apache.flink.connector.hbase.util.HBaseTableSchema)4 ReadableConfig (org.apache.flink.configuration.ReadableConfig)2 HBaseConnectorOptionsUtil.getHBaseConfiguration (org.apache.flink.connector.hbase.table.HBaseConnectorOptionsUtil.getHBaseConfiguration)2 HBaseConnectorOptionsUtil.getHBaseWriteOptions (org.apache.flink.connector.hbase.table.HBaseConnectorOptionsUtil.getHBaseWriteOptions)2 SinkFunctionProvider (org.apache.flink.table.connector.sink.SinkFunctionProvider)2 TableFactoryHelper (org.apache.flink.table.factories.FactoryUtil.TableFactoryHelper)2 FactoryUtil.createTableFactoryHelper (org.apache.flink.table.factories.FactoryUtil.createTableFactoryHelper)2 SinkRuntimeProviderContext (org.apache.flink.table.runtime.connector.sink.SinkRuntimeProviderContext)2 Configuration (org.apache.hadoop.conf.Configuration)2