Search in sources :

Example 1 with AllTableConstraintsRequest

use of org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest in project hive by apache.

the class Hive method getTableConstraints.

public SQLAllTableConstraints getTableConstraints(String dbName, String tblName, long tableId) throws HiveException, NoSuchObjectException {
    try {
        ValidWriteIdList validWriteIdList = getValidWriteIdList(dbName, tblName);
        AllTableConstraintsRequest request = new AllTableConstraintsRequest(dbName, tblName, getDefaultCatalog(conf));
        request.setTableId(tableId);
        request.setValidWriteIdList(validWriteIdList != null ? validWriteIdList.writeToString() : null);
        return getMSC().getAllTableConstraints(request);
    } catch (NoSuchObjectException e) {
        throw e;
    } catch (Exception e) {
        throw new HiveException(e);
    }
}
Also used : ValidWriteIdList(org.apache.hadoop.hive.common.ValidWriteIdList) AllTableConstraintsRequest(org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) LockException(org.apache.hadoop.hive.ql.lockmgr.LockException) InvalidOperationException(org.apache.hadoop.hive.metastore.api.InvalidOperationException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) ExecutionException(java.util.concurrent.ExecutionException) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) HiveMetaException(org.apache.hadoop.hive.metastore.HiveMetaException) SemanticException(org.apache.hadoop.hive.ql.parse.SemanticException) TApplicationException(org.apache.thrift.TApplicationException) TException(org.apache.thrift.TException) SerDeException(org.apache.hadoop.hive.serde2.SerDeException) FileNotFoundException(java.io.FileNotFoundException) JDODataStoreException(javax.jdo.JDODataStoreException)

Example 2 with AllTableConstraintsRequest

use of org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest in project hive by apache.

the class TestGetAllTableConstraints method fewPresentWithMultipleConstraints.

/**
 * Test where only some of the constraints are present along with multiple values for a single constraint
 * @throws TException
 */
@Test
public void fewPresentWithMultipleConstraints() throws TException {
    Table table = testTables[0];
    SQLAllTableConstraints expected = new SQLAllTableConstraints();
    // Set col1 as primary key Constraint in default catalog and database
    List<SQLPrimaryKey> pk = new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName("col1_pk").build(metaStore.getConf());
    client.addPrimaryKey(pk);
    expected.setPrimaryKeys(pk);
    // Set col2 with Unique Constraint in default catalog and database
    List<SQLUniqueConstraint> uc = new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName("col2_unique").build(metaStore.getConf());
    client.addUniqueConstraint(uc);
    expected.setUniqueConstraints(uc);
    // Set col3 with default Constraint in default catalog and database
    List<SQLDefaultConstraint> dv = new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName("col3_default").setDefaultVal(false).build(metaStore.getConf());
    client.addDefaultConstraint(dv);
    expected.setDefaultConstraints(dv);
    // Set col2 with not null constraint in default catalog and database;
    SQLNotNullConstraint nnCol2 = new SQLNotNullConstraint(table.getCatName(), table.getDbName(), table.getTableName(), "col2", "col2_not_null", true, true, true);
    SQLNotNullConstraint nnCol3 = new SQLNotNullConstraint(table.getCatName(), table.getDbName(), table.getTableName(), "col3", "col3_not_null", true, true, true);
    List<SQLNotNullConstraint> nn = new ArrayList<>();
    nn.add(nnCol2);
    nn.add(nnCol3);
    client.addNotNullConstraint(nn);
    expected.setNotNullConstraints(nn);
    expected.setForeignKeys(new ArrayList<>());
    expected.setCheckConstraints(new ArrayList<>());
    // Fetch all constraints for the table in default catalog and database
    AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName(), table.getCatName());
    SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
    Assert.assertEquals(expected, fetched);
}
Also used : SQLPrimaryKeyBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder) SQLUniqueConstraint(org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint) SQLPrimaryKey(org.apache.hadoop.hive.metastore.api.SQLPrimaryKey) Table(org.apache.hadoop.hive.metastore.api.Table) ArrayList(java.util.ArrayList) SQLDefaultConstraint(org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint) SQLNotNullConstraint(org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint) SQLDefaultConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder) SQLAllTableConstraints(org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints) SQLUniqueConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder) AllTableConstraintsRequest(org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 3 with AllTableConstraintsRequest

use of org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest in project hive by apache.

the class Hive method getTableConstraints.

public TableConstraintsInfo getTableConstraints(String dbName, String tblName, boolean fetchReliable, boolean fetchEnabled, long tableId) throws HiveException {
    PerfLogger perfLogger = SessionState.getPerfLogger();
    perfLogger.perfLogBegin(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS);
    try {
        ValidWriteIdList validWriteIdList = getValidWriteIdList(dbName, tblName);
        AllTableConstraintsRequest request = new AllTableConstraintsRequest(dbName, tblName, getDefaultCatalog(conf));
        request.setValidWriteIdList(validWriteIdList != null ? validWriteIdList.writeToString() : null);
        request.setTableId(tableId);
        SQLAllTableConstraints tableConstraints = getMSC().getAllTableConstraints(request);
        if (fetchReliable && tableConstraints != null) {
            if (CollectionUtils.isNotEmpty(tableConstraints.getPrimaryKeys())) {
                tableConstraints.setPrimaryKeys(tableConstraints.getPrimaryKeys().stream().filter(SQLPrimaryKey::isRely_cstr).collect(Collectors.toList()));
            }
            if (CollectionUtils.isNotEmpty(tableConstraints.getForeignKeys())) {
                tableConstraints.setForeignKeys(tableConstraints.getForeignKeys().stream().filter(SQLForeignKey::isRely_cstr).collect(Collectors.toList()));
            }
            if (CollectionUtils.isNotEmpty(tableConstraints.getUniqueConstraints())) {
                tableConstraints.setUniqueConstraints(tableConstraints.getUniqueConstraints().stream().filter(SQLUniqueConstraint::isRely_cstr).collect(Collectors.toList()));
            }
            if (CollectionUtils.isNotEmpty(tableConstraints.getNotNullConstraints())) {
                tableConstraints.setNotNullConstraints(tableConstraints.getNotNullConstraints().stream().filter(SQLNotNullConstraint::isRely_cstr).collect(Collectors.toList()));
            }
        }
        if (fetchEnabled && tableConstraints != null) {
            if (CollectionUtils.isNotEmpty(tableConstraints.getCheckConstraints())) {
                tableConstraints.setCheckConstraints(tableConstraints.getCheckConstraints().stream().filter(SQLCheckConstraint::isEnable_cstr).collect(Collectors.toList()));
            }
            if (CollectionUtils.isNotEmpty(tableConstraints.getDefaultConstraints())) {
                tableConstraints.setDefaultConstraints(tableConstraints.getDefaultConstraints().stream().filter(SQLDefaultConstraint::isEnable_cstr).collect(Collectors.toList()));
            }
        }
        return new TableConstraintsInfo(new PrimaryKeyInfo(tableConstraints.getPrimaryKeys(), tblName, dbName), new ForeignKeyInfo(tableConstraints.getForeignKeys(), tblName, dbName), new UniqueConstraint(tableConstraints.getUniqueConstraints(), tblName, dbName), new DefaultConstraint(tableConstraints.getDefaultConstraints(), tblName, dbName), new CheckConstraint(tableConstraints.getCheckConstraints()), new NotNullConstraint(tableConstraints.getNotNullConstraints(), tblName, dbName));
    } catch (Exception e) {
        throw new HiveException(e);
    } finally {
        perfLogger.perfLogEnd(CLASS_NAME, PerfLogger.HIVE_GET_TABLE_CONSTRAINTS, "HS2-cache");
    }
}
Also used : SQLUniqueConstraint(org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint) SQLPrimaryKey(org.apache.hadoop.hive.metastore.api.SQLPrimaryKey) SQLCheckConstraint(org.apache.hadoop.hive.metastore.api.SQLCheckConstraint) SQLForeignKey(org.apache.hadoop.hive.metastore.api.SQLForeignKey) PerfLogger(org.apache.hadoop.hive.ql.log.PerfLogger) SQLUniqueConstraint(org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) LockException(org.apache.hadoop.hive.ql.lockmgr.LockException) InvalidOperationException(org.apache.hadoop.hive.metastore.api.InvalidOperationException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) ExecutionException(java.util.concurrent.ExecutionException) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) HiveMetaException(org.apache.hadoop.hive.metastore.HiveMetaException) SemanticException(org.apache.hadoop.hive.ql.parse.SemanticException) TApplicationException(org.apache.thrift.TApplicationException) TException(org.apache.thrift.TException) SerDeException(org.apache.hadoop.hive.serde2.SerDeException) FileNotFoundException(java.io.FileNotFoundException) JDODataStoreException(javax.jdo.JDODataStoreException) SQLDefaultConstraint(org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint) SQLNotNullConstraint(org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint) SQLDefaultConstraint(org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint) SQLAllTableConstraints(org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints) ValidWriteIdList(org.apache.hadoop.hive.common.ValidWriteIdList) AllTableConstraintsRequest(org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest) SQLCheckConstraint(org.apache.hadoop.hive.metastore.api.SQLCheckConstraint) SQLNotNullConstraint(org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint)

Example 4 with AllTableConstraintsRequest

use of org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest in project hive by apache.

the class TestGetAllTableConstraints method allConstraintsPresent.

/**
 * Test to verify all constraint are present in table
 * @throws TException
 */
@Test
public void allConstraintsPresent() throws TException {
    Table table = testTables[0];
    Table parentTable = testTables[1];
    SQLAllTableConstraints expected = new SQLAllTableConstraints();
    // Set col1 as primary key Constraint in default catalog and database
    List<SQLPrimaryKey> pk = new SQLPrimaryKeyBuilder().onTable(table).addColumn("col1").setConstraintName("col1_pk").build(metaStore.getConf());
    client.addPrimaryKey(pk);
    expected.setPrimaryKeys(pk);
    // Set col2 with Unique Constraint in default catalog and database
    String uniqueConstraintName = "col2_unique";
    List<SQLUniqueConstraint> uc = new SQLUniqueConstraintBuilder().onTable(table).addColumn("col2").setConstraintName("col2_unique").build(metaStore.getConf());
    client.addUniqueConstraint(uc);
    expected.setUniqueConstraints(uc);
    // Set col3 with default Constraint in default catalog and database
    List<SQLDefaultConstraint> dv = new SQLDefaultConstraintBuilder().onTable(table).addColumn("col3").setConstraintName("col3_default").setDefaultVal(false).build(metaStore.getConf());
    client.addDefaultConstraint(dv);
    expected.setDefaultConstraints(dv);
    // Set col3 with not null constraint in default catalog and database;
    List<SQLNotNullConstraint> nn = new SQLNotNullConstraintBuilder().onTable(table).addColumn("col3").setConstraintName("col3_not_null").build(metaStore.getConf());
    client.addNotNullConstraint(nn);
    expected.setNotNullConstraints(nn);
    // Set col2 with not check constraint in default catalog and database;
    List<SQLCheckConstraint> cc = new SQLCheckConstraintBuilder().onTable(table).addColumn("col2").setConstraintName("col2_check").setCheckExpression("= 5").build(metaStore.getConf());
    client.addCheckConstraint(cc);
    expected.setCheckConstraints(cc);
    // Set col1 of parent table to PK and Set Col4 of table to FK
    List<SQLPrimaryKey> parentPk = new SQLPrimaryKeyBuilder().onTable(parentTable).addColumn("col1").setConstraintName("parentpk").build(metaStore.getConf());
    client.addPrimaryKey(parentPk);
    String fkConstraintName = "fk";
    List<SQLForeignKey> fk = new SQLForeignKeyBuilder().fromPrimaryKey(parentPk).onTable(table).setConstraintName(fkConstraintName).addColumn("col4").build(metaStore.getConf());
    client.addForeignKey(fk);
    expected.setForeignKeys(fk);
    // Fetch all constraints for the table in default catalog and database
    AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName(), table.getCatName());
    SQLAllTableConstraints actual = client.getAllTableConstraints(request);
    Assert.assertEquals(expected, actual);
}
Also used : SQLPrimaryKeyBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLPrimaryKeyBuilder) SQLUniqueConstraint(org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint) SQLForeignKeyBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLForeignKeyBuilder) SQLPrimaryKey(org.apache.hadoop.hive.metastore.api.SQLPrimaryKey) SQLCheckConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLCheckConstraintBuilder) Table(org.apache.hadoop.hive.metastore.api.Table) SQLCheckConstraint(org.apache.hadoop.hive.metastore.api.SQLCheckConstraint) SQLForeignKey(org.apache.hadoop.hive.metastore.api.SQLForeignKey) SQLDefaultConstraint(org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint) SQLNotNullConstraint(org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint) SQLDefaultConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLDefaultConstraintBuilder) SQLNotNullConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLNotNullConstraintBuilder) SQLAllTableConstraints(org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints) SQLUniqueConstraintBuilder(org.apache.hadoop.hive.metastore.client.builder.SQLUniqueConstraintBuilder) AllTableConstraintsRequest(org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Example 5 with AllTableConstraintsRequest

use of org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest in project hive by apache.

the class TestGetAllTableConstraints method noConstraints.

/**
 * Test where no constraint is present in the table
 * @throws TException
 */
@Test
public void noConstraints() throws TException {
    Table table = testTables[0];
    SQLAllTableConstraints constraints = new SQLAllTableConstraints();
    constraints.setPrimaryKeys(new ArrayList<>());
    constraints.setForeignKeys(new ArrayList<>());
    constraints.setNotNullConstraints(new ArrayList<>());
    constraints.setCheckConstraints(new ArrayList<>());
    constraints.setDefaultConstraints(new ArrayList<>());
    constraints.setUniqueConstraints(new ArrayList<>());
    AllTableConstraintsRequest request = new AllTableConstraintsRequest(table.getDbName(), table.getTableName(), table.getCatName());
    SQLAllTableConstraints fetched = client.getAllTableConstraints(request);
    Assert.assertEquals(constraints, fetched);
}
Also used : Table(org.apache.hadoop.hive.metastore.api.Table) SQLAllTableConstraints(org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints) AllTableConstraintsRequest(org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest) Test(org.junit.Test) MetastoreCheckinTest(org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)

Aggregations

AllTableConstraintsRequest (org.apache.hadoop.hive.metastore.api.AllTableConstraintsRequest)5 SQLAllTableConstraints (org.apache.hadoop.hive.metastore.api.SQLAllTableConstraints)4 MetastoreCheckinTest (org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest)3 SQLDefaultConstraint (org.apache.hadoop.hive.metastore.api.SQLDefaultConstraint)3 SQLNotNullConstraint (org.apache.hadoop.hive.metastore.api.SQLNotNullConstraint)3 SQLPrimaryKey (org.apache.hadoop.hive.metastore.api.SQLPrimaryKey)3 SQLUniqueConstraint (org.apache.hadoop.hive.metastore.api.SQLUniqueConstraint)3 Table (org.apache.hadoop.hive.metastore.api.Table)3 Test (org.junit.Test)3 FileNotFoundException (java.io.FileNotFoundException)2 IOException (java.io.IOException)2 UnknownHostException (java.net.UnknownHostException)2 ExecutionException (java.util.concurrent.ExecutionException)2 JDODataStoreException (javax.jdo.JDODataStoreException)2 ValidWriteIdList (org.apache.hadoop.hive.common.ValidWriteIdList)2 HiveMetaException (org.apache.hadoop.hive.metastore.HiveMetaException)2 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)2 InvalidOperationException (org.apache.hadoop.hive.metastore.api.InvalidOperationException)2 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)2 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)2