Search in sources :

Example 41 with IMetaStoreClient

use of org.apache.hadoop.hive.metastore.IMetaStoreClient in project cdap by caskdata.

the class BaseHiveExploreService method getMetaStoreClient.

private IMetaStoreClient getMetaStoreClient() throws ExploreException {
    if (metastoreClientLocal.get() == null) {
        try {
            IMetaStoreClient client = new HiveMetaStoreClient(getHiveConf());
            Supplier<IMetaStoreClient> supplier = Suppliers.ofInstance(client);
            metastoreClientLocal.set(supplier);
            // We use GC of the supplier as a signal for us to know that a thread is gone
            // The supplier is set into the thread local, which will get GC'ed when the thread is gone.
            // Since we use a weak reference key to the supplier that points to the client
            // (in the metastoreClientReferences map), it won't block GC of the supplier instance.
            // We can use the weak reference, which is retrieved through polling the ReferenceQueue,
            // to get back the client and call close() on it.
            metastoreClientReferences.put(new WeakReference<>(supplier, metastoreClientReferenceQueue), client);
        } catch (MetaException e) {
            throw new ExploreException("Error initializing Hive Metastore client", e);
        }
    }
    return metastoreClientLocal.get().get();
}
Also used : HiveMetaStoreClient(org.apache.hadoop.hive.metastore.HiveMetaStoreClient) IMetaStoreClient(org.apache.hadoop.hive.metastore.IMetaStoreClient) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) ExploreException(co.cask.cdap.explore.service.ExploreException)

Aggregations

IMetaStoreClient (org.apache.hadoop.hive.metastore.IMetaStoreClient)41 TException (org.apache.thrift.TException)12 IOException (java.io.IOException)11 Path (org.apache.hadoop.fs.Path)11 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)11 HiveConf (org.apache.hadoop.hive.conf.HiveConf)10 HiveMetaStoreClient (org.apache.hadoop.hive.metastore.HiveMetaStoreClient)10 Table (org.apache.hadoop.hive.metastore.api.Table)10 Test (org.junit.Test)10 FileStatus (org.apache.hadoop.fs.FileStatus)9 FileSystem (org.apache.hadoop.fs.FileSystem)9 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)8 CompactionRequest (org.apache.hadoop.hive.metastore.api.CompactionRequest)8 TxnStore (org.apache.hadoop.hive.metastore.txn.TxnStore)8 HiveEndPoint (org.apache.hive.hcatalog.streaming.HiveEndPoint)8 HiveSQLException (org.apache.hive.service.cli.HiveSQLException)7 ArrayList (java.util.ArrayList)6 DelimitedInputWriter (org.apache.hive.hcatalog.streaming.DelimitedInputWriter)6 StreamingConnection (org.apache.hive.hcatalog.streaming.StreamingConnection)6 Table (org.apache.hadoop.hive.ql.metadata.Table)5