Search in sources :

Example 6 with SchemaException

use of org.apache.gobblin.source.extractor.exception.SchemaException in project incubator-gobblin by apache.

the class TeradataExtractor method getSchema.

@Override
public JsonArray getSchema(CommandOutput<?, ?> response) throws SchemaException, IOException {
    log.debug("Extract schema from resultset");
    ResultSet resultset = null;
    Iterator<ResultSet> itr = (Iterator<ResultSet>) response.getResults().values().iterator();
    if (itr.hasNext()) {
        resultset = itr.next();
    } else {
        throw new SchemaException("Failed to get schema from Teradata - empty schema resultset");
    }
    JsonArray fieldJsonArray = new JsonArray();
    try {
        Schema schema = new Schema();
        ResultSetMetaData rsmd = resultset.getMetaData();
        String columnName, columnTypeName;
        for (int i = 1; i <= rsmd.getColumnCount(); i++) {
            columnName = rsmd.getColumnName(i);
            columnTypeName = rsmd.getColumnTypeName(i);
            schema.setColumnName(columnName);
            List<String> mapSymbols = null;
            JsonObject newDataType = this.convertDataType(columnName, columnTypeName, ELEMENT_DATA_TYPE, mapSymbols);
            schema.setDataType(newDataType);
            schema.setLength(rsmd.getColumnDisplaySize(i));
            schema.setPrecision(rsmd.getPrecision(i));
            schema.setScale(rsmd.getScale(i));
            schema.setNullable(rsmd.isNullable(i) == ResultSetMetaData.columnNullable);
            schema.setComment(rsmd.getColumnLabel(i));
            String jsonStr = gson.toJson(schema);
            JsonObject obj = gson.fromJson(jsonStr, JsonObject.class).getAsJsonObject();
            fieldJsonArray.add(obj);
        }
    } catch (Exception e) {
        throw new SchemaException("Failed to get schema from Teradaa; error - " + e.getMessage(), e);
    }
    return fieldJsonArray;
}
Also used : JsonArray(com.google.gson.JsonArray) ResultSetMetaData(java.sql.ResultSetMetaData) SchemaException(org.apache.gobblin.source.extractor.exception.SchemaException) Schema(org.apache.gobblin.source.extractor.schema.Schema) ResultSet(java.sql.ResultSet) Iterator(java.util.Iterator) JsonObject(com.google.gson.JsonObject) SchemaException(org.apache.gobblin.source.extractor.exception.SchemaException) IOException(java.io.IOException) HighWatermarkException(org.apache.gobblin.source.extractor.exception.HighWatermarkException) RecordCountException(org.apache.gobblin.source.extractor.exception.RecordCountException) DataRecordException(org.apache.gobblin.source.extractor.DataRecordException)

Aggregations

IOException (java.io.IOException)6 SchemaException (org.apache.gobblin.source.extractor.exception.SchemaException)6 JsonArray (com.google.gson.JsonArray)5 JsonObject (com.google.gson.JsonObject)5 Schema (org.apache.gobblin.source.extractor.schema.Schema)5 DataRecordException (org.apache.gobblin.source.extractor.DataRecordException)4 HighWatermarkException (org.apache.gobblin.source.extractor.exception.HighWatermarkException)4 RecordCountException (org.apache.gobblin.source.extractor.exception.RecordCountException)4 Iterator (java.util.Iterator)3 JsonElement (com.google.gson.JsonElement)2 ResultSet (java.sql.ResultSet)2 ParseException (java.text.ParseException)2 RestApiConnectionException (org.apache.gobblin.source.extractor.exception.RestApiConnectionException)2 Command (org.apache.gobblin.source.extractor.extract.Command)2 Splitter (com.google.common.base.Splitter)1 AsyncApiException (com.sforce.async.AsyncApiException)1 ResultSetMetaData (java.sql.ResultSetMetaData)1 SQLException (java.sql.SQLException)1 ArrayList (java.util.ArrayList)1 ListIterator (java.util.ListIterator)1