Search in sources :

Example 6 with AutoCloseableHiveLock

use of org.apache.gobblin.hive.AutoCloseableHiveLock in project incubator-gobblin by apache.

the class HiveMetaStoreBasedRegister method ensureHiveDbExistence.

/**
 * If databse existed on Hive side will return false;
 * Or will create the table thru. RPC and return retVal from remote MetaStore.
 * @param hiveDbName is the hive databases to be checked for existence
 */
private boolean ensureHiveDbExistence(String hiveDbName, IMetaStoreClient client) throws IOException {
    try (AutoCloseableHiveLock lock = this.locks.getDbLock(hiveDbName)) {
        Database db = new Database();
        db.setName(hiveDbName);
        try {
            try (Timer.Context context = this.metricContext.timer(GET_HIVE_DATABASE).time()) {
                client.getDatabase(db.getName());
            }
            return false;
        } catch (NoSuchObjectException nsoe) {
        // proceed with create
        } catch (TException te) {
            throw new IOException(te);
        }
        Preconditions.checkState(this.hiveDbRootDir.isPresent(), "Missing required property " + HiveRegProps.HIVE_DB_ROOT_DIR);
        db.setLocationUri(new Path(this.hiveDbRootDir.get(), hiveDbName + HIVE_DB_EXTENSION).toString());
        try {
            try (Timer.Context context = this.metricContext.timer(CREATE_HIVE_DATABASE).time()) {
                client.createDatabase(db);
            }
            log.info("Created database " + hiveDbName);
            HiveMetaStoreEventHelper.submitSuccessfulDBCreation(this.eventSubmitter, hiveDbName);
            return true;
        } catch (AlreadyExistsException e) {
            return false;
        } catch (TException e) {
            HiveMetaStoreEventHelper.submitFailedDBCreation(this.eventSubmitter, hiveDbName, e);
            throw new IOException("Unable to create Hive database " + hiveDbName, e);
        }
    }
}
Also used : TException(org.apache.thrift.TException) Path(org.apache.hadoop.fs.Path) Timer(com.codahale.metrics.Timer) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) Database(org.apache.hadoop.hive.metastore.api.Database) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) IOException(java.io.IOException) AutoCloseableHiveLock(org.apache.gobblin.hive.AutoCloseableHiveLock)

Aggregations

AutoCloseableHiveLock (org.apache.gobblin.hive.AutoCloseableHiveLock)6 Timer (com.codahale.metrics.Timer)5 IOException (java.io.IOException)3 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)3 TException (org.apache.thrift.TException)3 HivePartition (org.apache.gobblin.hive.HivePartition)2 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)2 Partition (org.apache.hadoop.hive.metastore.api.Partition)2 Lock (java.util.concurrent.locks.Lock)1 ReadWriteLock (java.util.concurrent.locks.ReadWriteLock)1 ReentrantReadWriteLock (java.util.concurrent.locks.ReentrantReadWriteLock)1 HiveLock (org.apache.gobblin.hive.HiveLock)1 MetricContext (org.apache.gobblin.metrics.MetricContext)1 SchemaRegistryException (org.apache.gobblin.metrics.kafka.SchemaRegistryException)1 Path (org.apache.hadoop.fs.Path)1 Database (org.apache.hadoop.hive.metastore.api.Database)1 Snapshot (org.apache.iceberg.Snapshot)1 Transaction (org.apache.iceberg.Transaction)1 UpdateProperties (org.apache.iceberg.UpdateProperties)1 TableIdentifier (org.apache.iceberg.catalog.TableIdentifier)1