Search in sources :

Example 6 with HighWatermarkException

use of org.apache.gobblin.source.extractor.exception.HighWatermarkException in project incubator-gobblin by apache.

the class JdbcExtractor method getMaxWatermark.

@Override
public long getMaxWatermark(String schema, String entity, String watermarkColumn, List<Predicate> predicateList, String watermarkSourceFormat) throws HighWatermarkException {
    this.log.info("Get high watermark using JDBC");
    long calculatedHighWatermark = ConfigurationKeys.DEFAULT_WATERMARK_VALUE;
    try {
        List<Command> cmds = this.getHighWatermarkMetadata(schema, entity, watermarkColumn, predicateList);
        CommandOutput<?, ?> response = this.executeSql(cmds);
        calculatedHighWatermark = this.getHighWatermark(response, watermarkColumn, watermarkSourceFormat);
        return calculatedHighWatermark;
    } catch (Exception e) {
        throw new HighWatermarkException("Failed to get high watermark using JDBC; error - " + e.getMessage(), e);
    }
}
Also used : Command(org.apache.gobblin.source.extractor.extract.Command) HighWatermarkException(org.apache.gobblin.source.extractor.exception.HighWatermarkException) ParseException(java.text.ParseException) RecordCountException(org.apache.gobblin.source.extractor.exception.RecordCountException) DataRecordException(org.apache.gobblin.source.extractor.DataRecordException) SqlParseException(org.apache.calcite.sql.parser.SqlParseException) SchemaException(org.apache.gobblin.source.extractor.exception.SchemaException) SQLException(java.sql.SQLException) IOException(java.io.IOException) HighWatermarkException(org.apache.gobblin.source.extractor.exception.HighWatermarkException)

Aggregations

IOException (java.io.IOException)6 DataRecordException (org.apache.gobblin.source.extractor.DataRecordException)6 HighWatermarkException (org.apache.gobblin.source.extractor.exception.HighWatermarkException)6 RecordCountException (org.apache.gobblin.source.extractor.exception.RecordCountException)6 SchemaException (org.apache.gobblin.source.extractor.exception.SchemaException)6 ParseException (java.text.ParseException)4 RestApiConnectionException (org.apache.gobblin.source.extractor.exception.RestApiConnectionException)3 AsyncApiException (com.sforce.async.AsyncApiException)2 SQLException (java.sql.SQLException)2 SimpleDateFormat (java.text.SimpleDateFormat)2 Date (java.util.Date)2 Iterator (java.util.Iterator)2 SqlParseException (org.apache.calcite.sql.parser.SqlParseException)2 RestApiClientException (org.apache.gobblin.source.extractor.exception.RestApiClientException)2 Command (org.apache.gobblin.source.extractor.extract.Command)2 JsonArray (com.google.gson.JsonArray)1 JsonElement (com.google.gson.JsonElement)1 JsonObject (com.google.gson.JsonObject)1 ResultSet (java.sql.ResultSet)1 ListIterator (java.util.ListIterator)1