Search in sources :

Example 11 with InvalidMetaException

use of com.netflix.metacat.common.server.connectors.exception.InvalidMetaException in project metacat by Netflix.

the class PolarisConnectorDatabaseService method update.

/**
 * {@inheritDoc}.
 */
@Override
public void update(final ConnectorRequestContext context, final DatabaseInfo databaseInfo) {
    final QualifiedName name = databaseInfo.getName();
    try {
        final PolarisDatabaseEntity db = polarisStoreService.getDatabase(name.getDatabaseName()).orElseThrow(() -> new DatabaseNotFoundException(name));
        // currently db objects have no mutable fields so this is noop
        polarisStoreService.saveDatabase(db.toBuilder().build());
    } catch (DatabaseNotFoundException exception) {
        log.error(String.format("Not found exception for polaris database %s", name), exception);
        throw exception;
    } catch (DataIntegrityViolationException exception) {
        throw new InvalidMetaException(name, exception);
    } catch (Exception exception) {
        throw new ConnectorException(String.format("Failed updating polaris database %s", databaseInfo.getName()), exception);
    }
}
Also used : QualifiedName(com.netflix.metacat.common.QualifiedName) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) PolarisDatabaseEntity(com.netflix.metacat.connector.polaris.store.entities.PolarisDatabaseEntity) DatabaseAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.DatabaseAlreadyExistsException) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException)

Example 12 with InvalidMetaException

use of com.netflix.metacat.common.server.connectors.exception.InvalidMetaException in project metacat by Netflix.

the class PolarisTableMapper method toEntity.

/**
 * {@inheritDoc}.
 */
@Override
public PolarisTableEntity toEntity(final TableInfo info) {
    final Map<String, String> metadata = info.getMetadata();
    if (MapUtils.isEmpty(metadata)) {
        final String message = String.format("No parameters defined for iceberg table %s", info.getName());
        throw new InvalidMetaException(info.getName(), message, null);
    }
    final String location = metadata.get(DirectSqlTable.PARAM_METADATA_LOCATION);
    if (StringUtils.isEmpty(location)) {
        final String message = String.format("No metadata location defined for iceberg table %s", info.getName());
        throw new InvalidMetaException(info.getName(), message, null);
    }
    final PolarisTableEntity tableEntity = PolarisTableEntity.builder().dbName(info.getName().getDatabaseName()).tblName(info.getName().getTableName()).metadataLocation(location).build();
    return tableEntity;
}
Also used : PolarisTableEntity(com.netflix.metacat.connector.polaris.store.entities.PolarisTableEntity) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)

Example 13 with InvalidMetaException

use of com.netflix.metacat.common.server.connectors.exception.InvalidMetaException in project metacat by Netflix.

the class PolarisConnectorTableService method create.

/**
 * {@inheritDoc}.
 */
@Override
public void create(final ConnectorRequestContext requestContext, final TableInfo tableInfo) {
    final QualifiedName name = tableInfo.getName();
    // check exists then create in non-transactional optimistic manner
    if (exists(requestContext, name)) {
        throw new TableAlreadyExistsException(name);
    }
    try {
        final PolarisTableEntity entity = polarisTableMapper.toEntity(tableInfo);
        polarisStoreService.createTable(entity.getDbName(), entity.getTblName(), entity.getMetadataLocation());
    } catch (DataIntegrityViolationException | InvalidMetaException exception) {
        throw new InvalidMetaException(name, exception);
    } catch (Exception exception) {
        final String msg = String.format("Failed creating polaris table %s", name);
        log.error(msg, exception);
        throw new ConnectorException(msg, exception);
    }
}
Also used : TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) QualifiedName(com.netflix.metacat.common.QualifiedName) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) PolarisTableEntity(com.netflix.metacat.connector.polaris.store.entities.PolarisTableEntity) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) TablePreconditionFailedException(com.netflix.metacat.common.server.connectors.exception.TablePreconditionFailedException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException) TableNotFoundException(com.netflix.metacat.common.server.connectors.exception.TableNotFoundException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException)

Example 14 with InvalidMetaException

use of com.netflix.metacat.common.server.connectors.exception.InvalidMetaException in project metacat by Netflix.

the class PolarisConnectorTableService method rename.

/**
 * {@inheritDoc}.
 */
@Override
public void rename(final ConnectorRequestContext context, final QualifiedName oldName, final QualifiedName newName) {
    // check exists then rename in non-transactional optimistic manner
    if (exists(context, newName)) {
        throw new TableAlreadyExistsException(newName);
    }
    try {
        final PolarisTableEntity table = polarisStoreService.getTable(oldName.getDatabaseName(), oldName.getTableName()).orElseThrow(() -> new TableNotFoundException(oldName));
        polarisStoreService.saveTable(table.toBuilder().tblName(newName.getTableName()).build());
    } catch (TableNotFoundException exception) {
        log.error(String.format("Not found exception for polaris table %s", oldName), exception);
        throw exception;
    } catch (DataIntegrityViolationException exception) {
        throw new InvalidMetaException(oldName, exception);
    } catch (Exception exception) {
        final String msg = String.format("Failed renaming polaris table %s", oldName);
        log.error(msg, exception);
        throw new ConnectorException(msg, exception);
    }
}
Also used : TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) TableNotFoundException(com.netflix.metacat.common.server.connectors.exception.TableNotFoundException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) PolarisTableEntity(com.netflix.metacat.connector.polaris.store.entities.PolarisTableEntity) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) TablePreconditionFailedException(com.netflix.metacat.common.server.connectors.exception.TablePreconditionFailedException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException) TableNotFoundException(com.netflix.metacat.common.server.connectors.exception.TableNotFoundException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) DataIntegrityViolationException(org.springframework.dao.DataIntegrityViolationException)

Example 15 with InvalidMetaException

use of com.netflix.metacat.common.server.connectors.exception.InvalidMetaException in project metacat by Netflix.

the class RequestWrapper method processRequest.

/**
 * Request wrapper to to process request.
 *
 * @param name                name
 * @param resourceRequestName request name
 * @param requestTags         tags that needs to be added to the registry
 * @param supplier            supplier
 * @param <R>                 response
 * @return response of supplier
 */
public <R> R processRequest(final QualifiedName name, final String resourceRequestName, final Map<String, String> requestTags, final Supplier<R> supplier) {
    final long start = registry.clock().wallTime();
    final Map<String, String> tags = new HashMap<>(name.parts());
    if (requestTags != null) {
        tags.putAll(requestTags);
    }
    tags.put("request", resourceRequestName);
    tags.put("scheme", MetacatContextManager.getContext().getScheme());
    registry.counter(requestCounterId.withTags(tags)).increment();
    try {
        // check rate limit in try-catch block in case ratelimiter throws exception.
        // those exceptions can be tracked correctly, by the existing finally block that logs metrics.
        checkRequestRateLimit(name, resourceRequestName, tags);
        log.info("### Calling method: {} for {}", resourceRequestName, name);
        return supplier.get();
    } catch (UnsupportedOperationException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        log.error(e.getMessage(), e);
        throw new MetacatNotSupportedException("Catalog does not support the operation. " + e.getMessage());
    } catch (DatabaseAlreadyExistsException | TableAlreadyExistsException | PartitionAlreadyExistsException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        log.error(e.getMessage(), e);
        throw new MetacatAlreadyExistsException(e.getMessage());
    } catch (NotFoundException | MetacatNotFoundException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        log.error(e.getMessage(), e);
        throw new MetacatNotFoundException(String.format("Unable to locate for %s. Details: %s", name, e.getMessage()));
    } catch (InvalidMetaException | IllegalArgumentException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        log.error(e.getMessage(), e);
        throw new MetacatBadRequestException(String.format("%s.%s", e.getMessage(), e.getCause() == null ? "" : e.getCause().getMessage()));
    } catch (TablePreconditionFailedException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        log.error(e.getMessage(), e);
        throw new MetacatPreconditionFailedException(String.format("%s.%s", e.getMessage(), e.getCause() == null ? "" : e.getCause().getMessage()));
    } catch (ConnectorException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        final String message = String.format("%s.%s -- %s failed for %s", e.getMessage(), e.getCause() == null ? "" : e.getCause().getMessage(), resourceRequestName, name);
        log.error(message, e);
        for (Throwable ex : Throwables.getCausalChain(e)) {
            if (ex.getMessage().contains("too many connections") || ex.getMessage().contains("Timeout: Pool empty")) {
                throw new MetacatTooManyRequestsException(ex.getMessage());
            }
        }
        throw new MetacatException(message, e);
    } catch (UserMetadataServiceException e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        final String message = String.format("%s.%s -- %s usermetadata operation failed for %s", e.getMessage(), e.getCause() == null ? "" : e.getCause().getMessage(), resourceRequestName, name);
        throw new MetacatUserMetadataException(message);
    } catch (Exception e) {
        collectRequestExceptionMetrics(tags, e.getClass().getSimpleName());
        final String message = String.format("%s.%s -- %s failed for %s", e.getMessage(), e.getCause() == null ? "" : e.getCause().getMessage(), resourceRequestName, name);
        log.error(message, e);
        if (e instanceof MetacatException) {
            throw e;
        } else {
            throw new MetacatException(message, e);
        }
    } finally {
        final long duration = registry.clock().wallTime() - start;
        log.info("### Time taken to complete {} for {} is {} ms", resourceRequestName, name, duration);
        tryAddTableTypeTag(tags, name);
        this.registry.timer(requestTimerId.withTags(tags)).record(duration, TimeUnit.MILLISECONDS);
    }
}
Also used : UserMetadataServiceException(com.netflix.metacat.common.server.usermetadata.UserMetadataServiceException) HashMap(java.util.HashMap) MetacatException(com.netflix.metacat.common.exception.MetacatException) NotFoundException(com.netflix.metacat.common.server.connectors.exception.NotFoundException) MetacatNotFoundException(com.netflix.metacat.common.exception.MetacatNotFoundException) MetacatNotFoundException(com.netflix.metacat.common.exception.MetacatNotFoundException) MetacatPreconditionFailedException(com.netflix.metacat.common.exception.MetacatPreconditionFailedException) PartitionAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.PartitionAlreadyExistsException) MetacatUserMetadataException(com.netflix.metacat.common.exception.MetacatUserMetadataException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) DatabaseAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.DatabaseAlreadyExistsException) TablePreconditionFailedException(com.netflix.metacat.common.server.connectors.exception.TablePreconditionFailedException) MetacatTooManyRequestsException(com.netflix.metacat.common.exception.MetacatTooManyRequestsException) MetacatNotSupportedException(com.netflix.metacat.common.exception.MetacatNotSupportedException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) MetacatException(com.netflix.metacat.common.exception.MetacatException) DatabaseAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.DatabaseAlreadyExistsException) TablePreconditionFailedException(com.netflix.metacat.common.server.connectors.exception.TablePreconditionFailedException) InvalidMetaException(com.netflix.metacat.common.server.connectors.exception.InvalidMetaException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) MetacatBadRequestException(com.netflix.metacat.common.exception.MetacatBadRequestException) MetacatNotSupportedException(com.netflix.metacat.common.exception.MetacatNotSupportedException) UserMetadataServiceException(com.netflix.metacat.common.server.usermetadata.UserMetadataServiceException) MetacatUserMetadataException(com.netflix.metacat.common.exception.MetacatUserMetadataException) NotFoundException(com.netflix.metacat.common.server.connectors.exception.NotFoundException) MetacatPreconditionFailedException(com.netflix.metacat.common.exception.MetacatPreconditionFailedException) MetacatAlreadyExistsException(com.netflix.metacat.common.exception.MetacatAlreadyExistsException) MetacatNotFoundException(com.netflix.metacat.common.exception.MetacatNotFoundException) PartitionAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.PartitionAlreadyExistsException) TableAlreadyExistsException(com.netflix.metacat.common.server.connectors.exception.TableAlreadyExistsException) MetacatTooManyRequestsException(com.netflix.metacat.common.exception.MetacatTooManyRequestsException) ConnectorException(com.netflix.metacat.common.server.connectors.exception.ConnectorException) MetacatAlreadyExistsException(com.netflix.metacat.common.exception.MetacatAlreadyExistsException) MetacatBadRequestException(com.netflix.metacat.common.exception.MetacatBadRequestException)

Aggregations

InvalidMetaException (com.netflix.metacat.common.server.connectors.exception.InvalidMetaException)33 ConnectorException (com.netflix.metacat.common.server.connectors.exception.ConnectorException)28 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)21 TException (org.apache.thrift.TException)21 QualifiedName (com.netflix.metacat.common.QualifiedName)18 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)17 TableNotFoundException (com.netflix.metacat.common.server.connectors.exception.TableNotFoundException)16 InvalidObjectException (org.apache.hadoop.hive.metastore.api.InvalidObjectException)14 Table (org.apache.hadoop.hive.metastore.api.Table)13 Partition (org.apache.hadoop.hive.metastore.api.Partition)9 Pageable (com.netflix.metacat.common.dto.Pageable)8 DatabaseNotFoundException (com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException)8 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)7 Lists (com.google.common.collect.Lists)6 DatabaseAlreadyExistsException (com.netflix.metacat.common.server.connectors.exception.DatabaseAlreadyExistsException)6 TablePreconditionFailedException (com.netflix.metacat.common.server.connectors.exception.TablePreconditionFailedException)6 TableInfo (com.netflix.metacat.common.server.connectors.model.TableInfo)6 ArrayList (java.util.ArrayList)6 Sort (com.netflix.metacat.common.dto.Sort)5 ConnectorContext (com.netflix.metacat.common.server.connectors.ConnectorContext)5