Search in sources :

Example 6 with JdbcNumericBetweenParametersProvider

use of org.apache.flink.connector.jdbc.split.JdbcNumericBetweenParametersProvider in project flink by apache.

the class JdbcInputFormatTest method testJdbcInputFormatWithoutParallelismAndNumericColumnSplitting.

@Test
public void testJdbcInputFormatWithoutParallelismAndNumericColumnSplitting() throws IOException {
    final long min = TEST_DATA[0].id;
    final long max = TEST_DATA[TEST_DATA.length - 1].id;
    // generate a single split
    final long fetchSize = max + 1;
    JdbcParameterValuesProvider pramProvider = new JdbcNumericBetweenParametersProvider(min, max).ofBatchSize(fetchSize);
    jdbcInputFormat = JdbcInputFormat.buildJdbcInputFormat().setDrivername(DERBY_EBOOKSHOP_DB.getDriverClass()).setDBUrl(DERBY_EBOOKSHOP_DB.getUrl()).setQuery(SELECT_ALL_BOOKS_SPLIT_BY_ID).setRowTypeInfo(ROW_TYPE_INFO).setParametersProvider(pramProvider).setResultSetType(ResultSet.TYPE_SCROLL_INSENSITIVE).finish();
    jdbcInputFormat.openInputFormat();
    InputSplit[] splits = jdbcInputFormat.createInputSplits(1);
    // assert that a single split was generated
    Assert.assertEquals(1, splits.length);
    int recordCount = 0;
    Row row = new Row(5);
    for (InputSplit split : splits) {
        jdbcInputFormat.open(split);
        while (!jdbcInputFormat.reachedEnd()) {
            Row next = jdbcInputFormat.nextRecord(row);
            assertEquals(TEST_DATA[recordCount], next);
            recordCount++;
        }
        jdbcInputFormat.close();
    }
    jdbcInputFormat.closeInputFormat();
    Assert.assertEquals(TEST_DATA.length, recordCount);
}
Also used : Row(org.apache.flink.types.Row) JdbcNumericBetweenParametersProvider(org.apache.flink.connector.jdbc.split.JdbcNumericBetweenParametersProvider) InputSplit(org.apache.flink.core.io.InputSplit) JdbcParameterValuesProvider(org.apache.flink.connector.jdbc.split.JdbcParameterValuesProvider) Test(org.junit.Test)

Aggregations

JdbcNumericBetweenParametersProvider (org.apache.flink.connector.jdbc.split.JdbcNumericBetweenParametersProvider)6 JdbcParameterValuesProvider (org.apache.flink.connector.jdbc.split.JdbcParameterValuesProvider)4 InputSplit (org.apache.flink.core.io.InputSplit)4 Test (org.junit.Test)4 Row (org.apache.flink.types.Row)3 GenericRowData (org.apache.flink.table.data.GenericRowData)2 RowData (org.apache.flink.table.data.RowData)2 Connection (java.sql.Connection)1 PreparedStatement (java.sql.PreparedStatement)1 ResultSet (java.sql.ResultSet)1 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)1 JdbcConnectionOptions (org.apache.flink.connector.jdbc.JdbcConnectionOptions)1 JdbcInputFormat (org.apache.flink.connector.jdbc.JdbcInputFormat)1 JdbcDialect (org.apache.flink.connector.jdbc.dialect.JdbcDialect)1 SimpleJdbcConnectionProvider (org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider)1 RowType (org.apache.flink.table.types.logical.RowType)1