Search in sources :

Example 1 with JdbcMapper

use of org.apache.storm.jdbc.mapper.JdbcMapper in project storm by apache.

the class UserPersistanceTopology method getTopology.

@Override
public StormTopology getTopology() {
    JdbcLookupBolt departmentLookupBolt = new JdbcLookupBolt(connectionProvider, SELECT_QUERY, this.jdbcLookupMapper);
    //must specify column schema when providing custom query.
    List<Column> schemaColumns = Lists.newArrayList(new Column("create_date", Types.DATE), new Column("dept_name", Types.VARCHAR), new Column("user_id", Types.INTEGER), new Column("user_name", Types.VARCHAR));
    JdbcMapper mapper = new SimpleJdbcMapper(schemaColumns);
    JdbcInsertBolt userPersistanceBolt = new JdbcInsertBolt(connectionProvider, mapper).withInsertQuery("insert into user (create_date, dept_name, user_id, user_name) values (?,?,?,?)");
    // userSpout ==> jdbcBolt
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout(USER_SPOUT, this.userSpout, 1);
    builder.setBolt(LOOKUP_BOLT, departmentLookupBolt, 1).shuffleGrouping(USER_SPOUT);
    builder.setBolt(PERSISTANCE_BOLT, userPersistanceBolt, 1).shuffleGrouping(LOOKUP_BOLT);
    return builder.createTopology();
}
Also used : SimpleJdbcMapper(org.apache.storm.jdbc.mapper.SimpleJdbcMapper) JdbcInsertBolt(org.apache.storm.jdbc.bolt.JdbcInsertBolt) Column(org.apache.storm.jdbc.common.Column) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) SimpleJdbcMapper(org.apache.storm.jdbc.mapper.SimpleJdbcMapper) JdbcMapper(org.apache.storm.jdbc.mapper.JdbcMapper) JdbcLookupBolt(org.apache.storm.jdbc.bolt.JdbcLookupBolt)

Example 2 with JdbcMapper

use of org.apache.storm.jdbc.mapper.JdbcMapper in project storm by apache.

the class JdbcInsertBoltTest method testValidation.

@Test
public void testValidation() {
    ConnectionProvider provider = new HikariCPConnectionProvider(new HashMap<String, Object>());
    JdbcMapper mapper = new SimpleJdbcMapper(Lists.newArrayList(new Column("test", 0)));
    expectIllegaArgs(null, mapper);
    expectIllegaArgs(provider, null);
    try {
        JdbcInsertBolt bolt = new JdbcInsertBolt(provider, mapper);
        bolt.withInsertQuery("test");
        bolt.withTableName("test");
        Assert.fail("Should have thrown IllegalArgumentException.");
    } catch (IllegalArgumentException ne) {
    //expected
    }
    try {
        JdbcInsertBolt bolt = new JdbcInsertBolt(provider, mapper);
        bolt.withTableName("test");
        bolt.withInsertQuery("test");
        Assert.fail("Should have thrown IllegalArgumentException.");
    } catch (IllegalArgumentException ne) {
    //expected
    }
}
Also used : SimpleJdbcMapper(org.apache.storm.jdbc.mapper.SimpleJdbcMapper) Column(org.apache.storm.jdbc.common.Column) SimpleJdbcMapper(org.apache.storm.jdbc.mapper.SimpleJdbcMapper) JdbcMapper(org.apache.storm.jdbc.mapper.JdbcMapper) HikariCPConnectionProvider(org.apache.storm.jdbc.common.HikariCPConnectionProvider) ConnectionProvider(org.apache.storm.jdbc.common.ConnectionProvider) HikariCPConnectionProvider(org.apache.storm.jdbc.common.HikariCPConnectionProvider) Test(org.junit.Test)

Aggregations

Column (org.apache.storm.jdbc.common.Column)2 JdbcMapper (org.apache.storm.jdbc.mapper.JdbcMapper)2 SimpleJdbcMapper (org.apache.storm.jdbc.mapper.SimpleJdbcMapper)2 JdbcInsertBolt (org.apache.storm.jdbc.bolt.JdbcInsertBolt)1 JdbcLookupBolt (org.apache.storm.jdbc.bolt.JdbcLookupBolt)1 ConnectionProvider (org.apache.storm.jdbc.common.ConnectionProvider)1 HikariCPConnectionProvider (org.apache.storm.jdbc.common.HikariCPConnectionProvider)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1 Test (org.junit.Test)1