Search in sources :

Example 1 with DatabaseInfo

use of alluxio.master.table.DatabaseInfo in project alluxio by Alluxio.

the class HiveDatabase method getDatabaseInfo.

@Override
public DatabaseInfo getDatabaseInfo() throws IOException {
    try (CloseableResource<IMetaStoreClient> client = mClientPool.acquireClientResource()) {
        Database hiveDb = client.get().getDatabase(mHiveDbName);
        alluxio.grpc.table.PrincipalType type = alluxio.grpc.table.PrincipalType.USER;
        if (Objects.equals(hiveDb.getOwnerType(), PrincipalType.ROLE)) {
            type = alluxio.grpc.table.PrincipalType.ROLE;
        }
        return new DatabaseInfo(hiveDb.getLocationUri(), hiveDb.getOwnerName(), type, hiveDb.getDescription(), hiveDb.getParameters());
    } catch (TException e) {
        throw new IOException("Failed to get hive database " + mHiveDbName + ". " + e.getMessage(), e);
    }
}
Also used : TException(org.apache.thrift.TException) DatabaseInfo(alluxio.master.table.DatabaseInfo) UnderDatabase(alluxio.table.common.udb.UnderDatabase) Database(org.apache.hadoop.hive.metastore.api.Database) IOException(java.io.IOException) IMetaStoreClient(org.apache.hadoop.hive.metastore.IMetaStoreClient)

Example 2 with DatabaseInfo

use of alluxio.master.table.DatabaseInfo in project alluxio by Alluxio.

the class GlueDatabase method getDatabaseInfo.

@Override
public DatabaseInfo getDatabaseInfo() throws IOException {
    try {
        GetDatabaseRequest dbRequest = new GetDatabaseRequest().withCatalogId(mGlueConfiguration.get(Property.CATALOG_ID)).withName(mGlueDbName);
        GetDatabaseResult dbResult = mGlueClient.getDatabase(dbRequest);
        Database glueDatabase = dbResult.getDatabase();
        // Glue database location, description and parameters could be null
        String glueDbLocation = glueDatabase.getLocationUri() == null ? "" : glueDatabase.getLocationUri();
        String glueDbDescription = glueDatabase.getDescription() == null ? "" : glueDatabase.getDescription();
        Map<String, String> glueParameters = new HashMap<>();
        if (glueDatabase.getParameters() != null) {
            glueParameters.putAll(glueDatabase.getParameters());
        }
        return new DatabaseInfo(glueDbLocation, mOwnerName, mOwnerType, glueDbDescription, glueParameters);
    } catch (EntityNotFoundException e) {
        throw new IOException("Cannot find glue database: " + mGlueDbName + "Catalog ID: " + mGlueConfiguration.get(Property.CATALOG_ID) + ". " + e.getMessage(), e);
    }
}
Also used : GetDatabaseRequest(com.amazonaws.services.glue.model.GetDatabaseRequest) HashMap(java.util.HashMap) DatabaseInfo(alluxio.master.table.DatabaseInfo) UnderDatabase(alluxio.table.common.udb.UnderDatabase) Database(com.amazonaws.services.glue.model.Database) GetDatabaseResult(com.amazonaws.services.glue.model.GetDatabaseResult) EntityNotFoundException(com.amazonaws.services.glue.model.EntityNotFoundException) IOException(java.io.IOException)

Example 3 with DatabaseInfo

use of alluxio.master.table.DatabaseInfo in project alluxio by Alluxio.

the class TableMasterJournalIntegrationTest method journalSync.

@Test
public void journalSync() throws Exception {
    LocalAlluxioCluster mCluster = sClusterResource.get();
    TableMaster tableMaster = mCluster.getLocalAlluxioMaster().getMasterProcess().getMaster(TableMaster.class);
    genTable(1, 2, true);
    tableMaster.attachDatabase(TestUdbFactory.TYPE, "connect", DB_NAME, DB_NAME, Collections.emptyMap(), false);
    checkDb(tableMaster, DB_NAME, TestDatabase.sTestDbInfo);
    DatabaseInfo oldInfo = TestDatabase.sTestDbInfo;
    DatabaseInfo newInfo = new DatabaseInfo("test2://test2", "newowner", PrincipalType.ROLE, "newcomment", ImmutableMap.of("key", "value"));
    checkTable(tableMaster, DB_NAME, 1, 2);
    checkTable(tableMaster, DB_NAME, 1, 2);
    assertEquals(TestDatabase.getTableName(0), tableMaster.getAllTables(DB_NAME).get(0));
    assertEquals(1, tableMaster.getAllTables(DB_NAME).size());
    assertEquals(2, tableMaster.getTable(DB_NAME, TestDatabase.getTableName(0)).getPartitions().size());
    // Update Udb, the table should stay the same, until we sync
    genTable(2, 3, true);
    tableMaster.syncDatabase(DB_NAME);
    checkTable(tableMaster, DB_NAME, 2, 3);
    // Drop a table to create a 'remove_table' entry
    genTable(1, 10, true);
    tableMaster.syncDatabase(DB_NAME);
    checkTable(tableMaster, DB_NAME, 1, 10);
    restartMaster();
    TableMaster tableMasterRestart = mCluster.getLocalAlluxioMaster().getMasterProcess().getMaster(TableMaster.class);
    TestDatabase.sTestDbInfo = newInfo;
    checkDb(tableMasterRestart, DB_NAME, oldInfo);
    tableMasterRestart.syncDatabase(DB_NAME);
    checkDb(tableMasterRestart, DB_NAME, newInfo);
    checkTable(tableMasterRestart, DB_NAME, 1, 10);
}
Also used : LocalAlluxioCluster(alluxio.master.LocalAlluxioCluster) DatabaseInfo(alluxio.master.table.DatabaseInfo) TableMaster(alluxio.master.table.TableMaster) Test(org.junit.Test)

Aggregations

DatabaseInfo (alluxio.master.table.DatabaseInfo)3 UnderDatabase (alluxio.table.common.udb.UnderDatabase)2 IOException (java.io.IOException)2 LocalAlluxioCluster (alluxio.master.LocalAlluxioCluster)1 TableMaster (alluxio.master.table.TableMaster)1 Database (com.amazonaws.services.glue.model.Database)1 EntityNotFoundException (com.amazonaws.services.glue.model.EntityNotFoundException)1 GetDatabaseRequest (com.amazonaws.services.glue.model.GetDatabaseRequest)1 GetDatabaseResult (com.amazonaws.services.glue.model.GetDatabaseResult)1 HashMap (java.util.HashMap)1 IMetaStoreClient (org.apache.hadoop.hive.metastore.IMetaStoreClient)1 Database (org.apache.hadoop.hive.metastore.api.Database)1 TException (org.apache.thrift.TException)1 Test (org.junit.Test)1