Search in sources :

Example 51 with MetaException

use of org.apache.hadoop.hive.metastore.api.MetaException in project hive by apache.

the class RevokePrivAuthUtils method authorizeAndGetRevokePrivileges.

public static List<HiveObjectPrivilege> authorizeAndGetRevokePrivileges(List<HivePrincipal> principals, List<HivePrivilege> hivePrivileges, HivePrivilegeObject hivePrivObject, boolean grantOption, IMetaStoreClient mClient, String userName) throws HiveAuthzPluginException, HiveAccessControlException {
    List<HiveObjectPrivilege> matchingPrivs = new ArrayList<HiveObjectPrivilege>();
    StringBuilder errMsg = new StringBuilder();
    for (HivePrincipal principal : principals) {
        // get metastore/thrift privilege object for this principal and object, not looking at
        // privileges obtained indirectly via roles
        List<HiveObjectPrivilege> msObjPrivs;
        try {
            msObjPrivs = mClient.list_privileges(principal.getName(), AuthorizationUtils.getThriftPrincipalType(principal.getType()), SQLAuthorizationUtils.getThriftHiveObjectRef(hivePrivObject));
        } catch (MetaException e) {
            throw new HiveAuthzPluginException(e);
        } catch (TException e) {
            throw new HiveAuthzPluginException(e);
        }
        // the resulting privileges need to be filtered on privilege type and
        // username
        // create a Map to capture object privileges corresponding to privilege
        // type
        Map<String, HiveObjectPrivilege> priv2privObj = new HashMap<String, HiveObjectPrivilege>();
        for (HiveObjectPrivilege msObjPriv : msObjPrivs) {
            PrivilegeGrantInfo grantInfo = msObjPriv.getGrantInfo();
            // check if the grantor matches current user
            if (grantInfo.getGrantor() != null && grantInfo.getGrantor().equals(userName) && grantInfo.getGrantorType() == PrincipalType.USER) {
                // add to the map
                priv2privObj.put(grantInfo.getPrivilege(), msObjPriv);
            }
        // else skip this one
        }
        // find the privileges that we are looking for
        for (HivePrivilege hivePrivilege : hivePrivileges) {
            HiveObjectPrivilege matchedPriv = priv2privObj.get(hivePrivilege.getName());
            if (matchedPriv != null) {
                matchingPrivs.add(matchedPriv);
            } else {
                errMsg.append("Cannot find privilege ").append(hivePrivilege).append(" for ").append(principal).append(" on ").append(hivePrivObject).append(" granted by ").append(userName).append(System.getProperty("line.separator"));
            }
        }
    }
    if (errMsg.length() != 0) {
        throw new HiveAccessControlException(errMsg.toString());
    }
    return matchingPrivs;
}
Also used : TException(org.apache.thrift.TException) HashMap(java.util.HashMap) PrivilegeGrantInfo(org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo) HivePrivilege(org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrivilege) ArrayList(java.util.ArrayList) HiveAuthzPluginException(org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException) HiveObjectPrivilege(org.apache.hadoop.hive.metastore.api.HiveObjectPrivilege) HiveAccessControlException(org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException) HivePrincipal(org.apache.hadoop.hive.ql.security.authorization.plugin.HivePrincipal) MetaException(org.apache.hadoop.hive.metastore.api.MetaException)

Example 52 with MetaException

use of org.apache.hadoop.hive.metastore.api.MetaException in project metacat by Netflix.

the class HiveConnectorFactory method createThriftClient.

private IMetacatHiveClient createThriftClient() throws MetaException {
    final HiveMetastoreClientFactory factory = new HiveMetastoreClientFactory(null, (int) HiveConnectorUtil.toTime(configuration.getOrDefault(HiveConfigConstants.HIVE_METASTORE_TIMEOUT, "20s"), TimeUnit.SECONDS, TimeUnit.MILLISECONDS));
    final String metastoreUri = configuration.get(HiveConfigConstants.THRIFT_URI);
    URI uri = null;
    try {
        uri = new URI(metastoreUri);
    } catch (Exception e) {
        final String message = String.format("Invalid thrift uri %s", metastoreUri);
        log.info(message);
        throw new IllegalArgumentException(message, e);
    }
    return new MetacatHiveClient(uri, factory);
}
Also used : HiveMetastoreClientFactory(com.netflix.metacat.connector.hive.client.thrift.HiveMetastoreClientFactory) MetacatHiveClient(com.netflix.metacat.connector.hive.client.thrift.MetacatHiveClient) URI(java.net.URI) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) TException(org.apache.thrift.TException)

Example 53 with MetaException

use of org.apache.hadoop.hive.metastore.api.MetaException in project metacat by Netflix.

the class HiveConnectorTableService method list.

/**
     * {@inheritDoc}.
     */
@Override
public List<TableInfo> list(@Nonnull @NonNull final ConnectorContext requestContext, @Nonnull @NonNull final QualifiedName name, @Nullable final QualifiedName prefix, @Nullable final Sort sort, @Nullable final Pageable pageable) {
    try {
        final List<TableInfo> tableInfos = Lists.newArrayList();
        for (String tableName : metacatHiveClient.getAllTables(name.getDatabaseName())) {
            final QualifiedName qualifiedName = QualifiedName.ofDatabase(name.getCatalogName(), tableName);
            if (!qualifiedName.toString().startsWith(prefix.toString())) {
                continue;
            }
            final Table table = metacatHiveClient.getTableByName(name.getDatabaseName(), tableName);
            tableInfos.add(hiveMetacatConverters.toTableInfo(name, table));
        }
        //supporting sort by name only
        if (sort != null) {
            ConnectorUtils.sort(tableInfos, sort, Comparator.comparing(p -> p.getName().getTableName()));
        }
        return ConnectorUtils.paginate(tableInfos, pageable);
    } catch (MetaException exception) {
        throw new DatabaseNotFoundException(name, exception);
    } catch (TException exception) {
        throw new ConnectorException(String.format("Failed list hive table %s", name), exception);
    }
}
Also used : MetaException(org.apache.hadoop.hive.metastore.api.MetaException) SerDeInfo(org.apache.hadoop.hive.metastore.api.SerDeInfo) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) Inject(javax.inject.Inject) Strings(com.google.common.base.Strings) ConnectorTableService(com.netflix.metacat.common.server.connectors.ConnectorTableService) FieldInfo(com.netflix.metacat.common.server.connectors.model.FieldInfo) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) Lists(com.google.common.collect.Lists) ImmutableList(com.google.common.collect.ImmutableList) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) Map(java.util.Map) Path(org.apache.hadoop.fs.Path) ConnectorContext(com.netflix.metacat.common.server.connectors.ConnectorContext) StorageInfo(com.netflix.metacat.common.server.connectors.model.StorageInfo) Named(javax.inject.Named) HiveConnectorInfoConverter(com.netflix.metacat.connector.hive.converters.HiveConnectorInfoConverter) StorageDescriptor(org.apache.hadoop.hive.metastore.api.StorageDescriptor) Nonnull(javax.annotation.Nonnull) Nullable(javax.annotation.Nullable) ImmutableMap(com.google.common.collect.ImmutableMap) NonNull(lombok.NonNull) Pageable(com.netflix.metacat.common.dto.Pageable) TException(org.apache.thrift.TException) QualifiedName(com.netflix.metacat.common.QualifiedName) InvalidObjectException(org.apache.hadoop.hive.metastore.api.InvalidObjectException) TableNotFoundException(com.netflix.metacat.common.server.connectors.exception.TableNotFoundException) Maps(com.google.common.collect.Maps) Table(org.apache.hadoop.hive.metastore.api.Table) FieldSchema(org.apache.hadoop.hive.metastore.api.FieldSchema) List(java.util.List) TableInfo(com.netflix.metacat.common.server.connectors.model.TableInfo) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) TableType(org.apache.hadoop.hive.metastore.TableType) ConnectorUtils(com.netflix.metacat.common.server.connectors.ConnectorUtils) Comparator(java.util.Comparator) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) Sort(com.netflix.metacat.common.dto.Sort) TException(org.apache.thrift.TException) Table(org.apache.hadoop.hive.metastore.api.Table) QualifiedName(com.netflix.metacat.common.QualifiedName) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) TableInfo(com.netflix.metacat.common.server.connectors.model.TableInfo) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)

Example 54 with MetaException

use of org.apache.hadoop.hive.metastore.api.MetaException in project metacat by Netflix.

the class HiveConnectorDatabaseService method listNames.

/**
     * {@inheritDoc}.
     */
@Override
public List<QualifiedName> listNames(@Nonnull @NonNull final ConnectorContext requestContext, @Nonnull @NonNull final QualifiedName name, @Nullable final QualifiedName prefix, @Nullable final Sort sort, @Nullable final Pageable pageable) {
    try {
        final List<QualifiedName> qualifiedNames = Lists.newArrayList();
        final String databaseFilter = (prefix != null) ? prefix.getDatabaseName() : null;
        for (String databaseName : metacatHiveClient.getAllDatabases()) {
            final QualifiedName qualifiedName = QualifiedName.ofDatabase(name.getCatalogName(), databaseName);
            if (databaseFilter != null && !databaseName.startsWith(databaseFilter)) {
                continue;
            }
            qualifiedNames.add(qualifiedName);
        }
        //supporting sort by qualified name only
        if (sort != null) {
            ConnectorUtils.sort(qualifiedNames, sort, Comparator.comparing(QualifiedName::toString));
        }
        return ConnectorUtils.paginate(qualifiedNames, pageable);
    } catch (MetaException exception) {
        throw new InvalidMetaException(name, exception);
    } catch (TException exception) {
        throw new ConnectorException(String.format("Failed listName hive database %s", name), exception);
    }
}
Also used : TException(org.apache.thrift.TException) QualifiedName(com.netflix.metacat.common.QualifiedName) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)

Example 55 with MetaException

use of org.apache.hadoop.hive.metastore.api.MetaException in project metacat by Netflix.

the class HiveConnectorTableService method create.

/**
     * Create a table.
     *
     * @param requestContext The request context
     * @param tableInfo      The resource metadata
     */
@Override
public void create(@Nonnull @NonNull final ConnectorContext requestContext, @Nonnull @NonNull final TableInfo tableInfo) {
    final QualifiedName tableName = tableInfo.getName();
    try {
        final Table table = hiveMetacatConverters.fromTableInfo(tableInfo);
        updateTable(requestContext, table, tableInfo);
        metacatHiveClient.createTable(table);
    } catch (AlreadyExistsException exception) {
        throw new TableAlreadyExistsException(tableName, exception);
    } catch (MetaException exception) {
        throw new InvalidMetaException(tableName, exception);
    } catch (NoSuchObjectException | InvalidObjectException exception) {
        throw new DatabaseNotFoundException(QualifiedName.ofDatabase(tableName.getCatalogName(), tableName.getDatabaseName()), exception);
    } catch (TException exception) {
        throw new ConnectorException(String.format("Failed create hive table %s", tableName), exception);
    }
}
Also used : TException(org.apache.thrift.TException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) Table(org.apache.hadoop.hive.metastore.api.Table) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) QualifiedName(com.netflix.metacat.common.QualifiedName) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) InvalidObjectException(org.apache.hadoop.hive.metastore.api.InvalidObjectException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)

Aggregations

MetaException (org.apache.hadoop.hive.metastore.api.MetaException)318 IOException (java.io.IOException)123 ArrayList (java.util.ArrayList)95 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)74 TException (org.apache.thrift.TException)67 Table (org.apache.hadoop.hive.metastore.api.Table)59 Partition (org.apache.hadoop.hive.metastore.api.Partition)57 SQLException (java.sql.SQLException)55 InvalidObjectException (org.apache.hadoop.hive.metastore.api.InvalidObjectException)53 Path (org.apache.hadoop.fs.Path)45 Connection (java.sql.Connection)36 InvalidOperationException (org.apache.hadoop.hive.metastore.api.InvalidOperationException)34 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)32 Statement (java.sql.Statement)31 Test (org.junit.Test)30 List (java.util.List)25 Database (org.apache.hadoop.hive.metastore.api.Database)25 FieldSchema (org.apache.hadoop.hive.metastore.api.FieldSchema)25 ResultSet (java.sql.ResultSet)22 UnknownDBException (org.apache.hadoop.hive.metastore.api.UnknownDBException)22