Search in sources :

Example 11 with Command

use of org.apache.gobblin.source.extractor.extract.Command in project incubator-gobblin by apache.

the class JdbcExtractor method getSourceCount.

@Override
public long getSourceCount(String schema, String entity, WorkUnit workUnit, List<Predicate> predicateList) throws RecordCountException {
    this.log.info("Get source record count using JDBC");
    long count = 0;
    try {
        List<Command> cmds = this.getCountMetadata(schema, entity, workUnit, predicateList);
        CommandOutput<?, ?> response = this.executeSql(cmds);
        count = this.getCount(response);
        this.log.info("Source record count:" + count);
        return count;
    } catch (Exception e) {
        throw new RecordCountException("Failed to get source record count using JDBC; error - " + e.getMessage(), e);
    }
}
Also used : RecordCountException(org.apache.gobblin.source.extractor.exception.RecordCountException) Command(org.apache.gobblin.source.extractor.extract.Command) ParseException(java.text.ParseException) RecordCountException(org.apache.gobblin.source.extractor.exception.RecordCountException) DataRecordException(org.apache.gobblin.source.extractor.DataRecordException) SqlParseException(org.apache.calcite.sql.parser.SqlParseException) SchemaException(org.apache.gobblin.source.extractor.exception.SchemaException) SQLException(java.sql.SQLException) IOException(java.io.IOException) HighWatermarkException(org.apache.gobblin.source.extractor.exception.HighWatermarkException)

Aggregations

Command (org.apache.gobblin.source.extractor.extract.Command)11 IOException (java.io.IOException)10 SchemaException (org.apache.gobblin.source.extractor.exception.SchemaException)10 DataRecordException (org.apache.gobblin.source.extractor.DataRecordException)8 HighWatermarkException (org.apache.gobblin.source.extractor.exception.HighWatermarkException)8 RecordCountException (org.apache.gobblin.source.extractor.exception.RecordCountException)8 SQLException (java.sql.SQLException)5 ParseException (java.text.ParseException)5 SqlParseException (org.apache.calcite.sql.parser.SqlParseException)5 RestApiConnectionException (org.apache.gobblin.source.extractor.exception.RestApiConnectionException)4 RestApiProcessingException (org.apache.gobblin.source.extractor.exception.RestApiProcessingException)4 JsonElement (com.google.gson.JsonElement)3 JsonArray (com.google.gson.JsonArray)2 JsonObject (com.google.gson.JsonObject)2 PreparedStatement (java.sql.PreparedStatement)2 ResultSet (java.sql.ResultSet)2 Schema (org.apache.gobblin.source.extractor.schema.Schema)2 JdbcCommandType (org.apache.gobblin.source.jdbc.JdbcCommand.JdbcCommandType)2 Splitter (com.google.common.base.Splitter)1 Statement (java.sql.Statement)1