Search in sources :

Example 1 with GetAllWriteEventInfoRequest

use of org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest in project hive by apache.

the class CommitTxnHandler method getAllWriteEventInfo.

private List<WriteEventInfo> getAllWriteEventInfo(Context withinContext) throws Exception {
    String contextDbName = StringUtils.normalizeIdentifier(withinContext.replScope.getDbName());
    GetAllWriteEventInfoRequest request = new GetAllWriteEventInfoRequest(eventMessage.getTxnId());
    request.setDbName(contextDbName);
    List<WriteEventInfo> writeEventInfoList = withinContext.db.getMSC().getAllWriteEventInfo(request);
    return ((writeEventInfoList == null) ? null : new ArrayList<>(Collections2.filter(writeEventInfoList, writeEventInfo -> {
        assert (writeEventInfo != null);
        // it should be skipped.
        return (ReplUtils.tableIncludedInReplScope(withinContext.replScope, writeEventInfo.getTable()) && ReplUtils.tableIncludedInReplScope(withinContext.oldReplScope, writeEventInfo.getTable()) && !withinContext.getTablesForBootstrap().contains(writeEventInfo.getTable().toLowerCase()));
    })));
}
Also used : LoginException(javax.security.auth.login.LoginException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) DumpMetaData(org.apache.hadoop.hive.ql.parse.repl.load.DumpMetaData) Collections2(com.google.common.collect.Collections2) GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) SemanticException(org.apache.hadoop.hive.ql.parse.SemanticException) DumpType(org.apache.hadoop.hive.ql.parse.repl.DumpType) StringUtils(org.apache.hadoop.hive.metastore.utils.StringUtils) ArrayList(java.util.ArrayList) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) Lists(com.google.common.collect.Lists) EximUtil(org.apache.hadoop.hive.ql.parse.EximUtil) Path(org.apache.hadoop.fs.Path) ReplChangeManager(org.apache.hadoop.hive.metastore.ReplChangeManager) HiveUtils(org.apache.hadoop.hive.ql.metadata.HiveUtils) HiveConf(org.apache.hadoop.hive.conf.HiveConf) Table(org.apache.hadoop.hive.ql.metadata.Table) IOException(java.io.IOException) HiveFatalException(org.apache.hadoop.hive.ql.metadata.HiveFatalException) File(java.io.File) Partition(org.apache.hadoop.hive.ql.metadata.Partition) List(java.util.List) NotificationEvent(org.apache.hadoop.hive.metastore.api.NotificationEvent) ReplUtils(org.apache.hadoop.hive.ql.exec.repl.util.ReplUtils) CommitTxnMessage(org.apache.hadoop.hive.metastore.messaging.CommitTxnMessage) GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) ArrayList(java.util.ArrayList)

Example 2 with GetAllWriteEventInfoRequest

use of org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest in project hive by apache.

the class TestGetAllWriteEventInfo method testGetByWrongTable.

@Test
public void testGetByWrongTable() throws Exception {
    GetAllWriteEventInfoRequest req = new GetAllWriteEventInfoRequest();
    req.setTxnId(TXN_ID);
    req.setDbName(DB_NAME);
    req.setTableName("wrong_table");
    List<WriteEventInfo> writeEventInfoList = client.getAllWriteEventInfo(req);
    Assert.assertTrue(writeEventInfoList.isEmpty());
}
Also used : GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 3 with GetAllWriteEventInfoRequest

use of org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest in project hive by apache.

the class TestGetAllWriteEventInfo method testGetByTxnId.

@Test
public void testGetByTxnId() throws Exception {
    GetAllWriteEventInfoRequest req = new GetAllWriteEventInfoRequest();
    req.setTxnId(TXN_ID);
    List<WriteEventInfo> writeEventInfoList = client.getAllWriteEventInfo(req);
    Assert.assertEquals(1, writeEventInfoList.size());
    WriteEventInfo writeEventInfo = writeEventInfoList.get(0);
    Assert.assertEquals(TXN_ID, writeEventInfo.getWriteId());
    Assert.assertEquals(DB_NAME, writeEventInfo.getDatabase());
    Assert.assertEquals(TABLE_NAME, writeEventInfo.getTable());
}
Also used : GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 4 with GetAllWriteEventInfoRequest

use of org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest in project hive by apache.

the class TestGetAllWriteEventInfo method testGetByWrongDB.

@Test
public void testGetByWrongDB() throws Exception {
    GetAllWriteEventInfoRequest req = new GetAllWriteEventInfoRequest();
    req.setTxnId(TXN_ID);
    req.setDbName("wrong_db");
    List<WriteEventInfo> writeEventInfoList = client.getAllWriteEventInfo(req);
    Assert.assertTrue(writeEventInfoList.isEmpty());
}
Also used : GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 5 with GetAllWriteEventInfoRequest

use of org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest in project hive by apache.

the class TestGetAllWriteEventInfo method testGetByTxnIdAndTableName.

@Test
public void testGetByTxnIdAndTableName() throws Exception {
    GetAllWriteEventInfoRequest req = new GetAllWriteEventInfoRequest();
    req.setTxnId(TXN_ID);
    req.setDbName(DB_NAME);
    req.setTableName(TABLE_NAME);
    List<WriteEventInfo> writeEventInfoList = client.getAllWriteEventInfo(req);
    Assert.assertEquals(1, writeEventInfoList.size());
    WriteEventInfo writeEventInfo = writeEventInfoList.get(0);
    Assert.assertEquals(TXN_ID, writeEventInfo.getWriteId());
    Assert.assertEquals(DB_NAME, writeEventInfo.getDatabase());
    Assert.assertEquals(TABLE_NAME, writeEventInfo.getTable());
}
Also used : GetAllWriteEventInfoRequest(org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest) WriteEventInfo(org.apache.hadoop.hive.metastore.api.WriteEventInfo) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Aggregations

GetAllWriteEventInfoRequest (org.apache.hadoop.hive.metastore.api.GetAllWriteEventInfoRequest)6 WriteEventInfo (org.apache.hadoop.hive.metastore.api.WriteEventInfo)6 MetastoreCheckinTest (org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)5 Test (org.junit.Test)5 Collections2 (com.google.common.collect.Collections2)1 Lists (com.google.common.collect.Lists)1 File (java.io.File)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 LoginException (javax.security.auth.login.LoginException)1 Path (org.apache.hadoop.fs.Path)1 HiveConf (org.apache.hadoop.hive.conf.HiveConf)1 ReplChangeManager (org.apache.hadoop.hive.metastore.ReplChangeManager)1 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)1 NotificationEvent (org.apache.hadoop.hive.metastore.api.NotificationEvent)1 CommitTxnMessage (org.apache.hadoop.hive.metastore.messaging.CommitTxnMessage)1 StringUtils (org.apache.hadoop.hive.metastore.utils.StringUtils)1 ReplUtils (org.apache.hadoop.hive.ql.exec.repl.util.ReplUtils)1 HiveFatalException (org.apache.hadoop.hive.ql.metadata.HiveFatalException)1