Search in sources :

Example 1 with BadExternalFileException

use of bio.terra.service.tabulardata.exception.BadExternalFileException in project jade-data-repo by DataBiosphere.

the class BigQueryPdao method createSoftDeleteExternalTable.

public void createSoftDeleteExternalTable(Dataset dataset, String path, String tableName, String suffix) throws InterruptedException {
    BigQueryProject bigQueryProject = bigQueryProjectForDataset(dataset);
    String extTableName = externalTableName(tableName, suffix);
    TableId tableId = TableId.of(prefixName(dataset.getName()), extTableName);
    Schema schema = Schema.of(Field.of(PDAO_ROW_ID_COLUMN, LegacySQLTypeName.STRING));
    ExternalTableDefinition tableDef = ExternalTableDefinition.of(path, schema, FormatOptions.csv());
    TableInfo tableInfo = TableInfo.of(tableId, tableDef);
    bigQueryProject.getBigQuery().create(tableInfo);
    // validate that the external table has data
    String sql = new ST(validateExtTableTemplate).add("rowId", PDAO_ROW_ID_COLUMN).add("project", bigQueryProject.getProjectId()).add("dataset", tableId.getDataset()).add("table", tableId.getTable()).render();
    TableResult result = bigQueryProject.query(sql);
    if (result.getTotalRows() == 0L) {
        // either the file at the path is empty or it doesn't exist. error out and let the cleanup begin
        String msg = String.format("No rows found at %s. Likely it is from a bad path or empty file(s).", path);
        throw new BadExternalFileException(msg);
    }
}
Also used : TableId(com.google.cloud.bigquery.TableId) ST(org.stringtemplate.v4.ST) ExternalTableDefinition(com.google.cloud.bigquery.ExternalTableDefinition) TableResult(com.google.cloud.bigquery.TableResult) BadExternalFileException(bio.terra.service.tabulardata.exception.BadExternalFileException) Schema(com.google.cloud.bigquery.Schema) TableInfo(com.google.cloud.bigquery.TableInfo)

Aggregations

BadExternalFileException (bio.terra.service.tabulardata.exception.BadExternalFileException)1 ExternalTableDefinition (com.google.cloud.bigquery.ExternalTableDefinition)1 Schema (com.google.cloud.bigquery.Schema)1 TableId (com.google.cloud.bigquery.TableId)1 TableInfo (com.google.cloud.bigquery.TableInfo)1 TableResult (com.google.cloud.bigquery.TableResult)1 ST (org.stringtemplate.v4.ST)1