Search in sources :

Example 6 with PartitionNotExistException

use of org.apache.flink.table.catalog.exceptions.PartitionNotExistException in project flink by apache.

the class GenericInMemoryCatalog method alterPartitionStatistics.

@Override
public void alterPartitionStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogTableStatistics partitionStatistics, boolean ignoreIfNotExists) throws PartitionNotExistException {
    checkNotNull(tablePath);
    checkNotNull(partitionSpec);
    checkNotNull(partitionStatistics);
    if (partitionExists(tablePath, partitionSpec)) {
        partitionStats.get(tablePath).put(partitionSpec, partitionStatistics.copy());
    } else if (!ignoreIfNotExists) {
        throw new PartitionNotExistException(getName(), tablePath, partitionSpec);
    }
}
Also used : PartitionNotExistException(org.apache.flink.table.catalog.exceptions.PartitionNotExistException)

Example 7 with PartitionNotExistException

use of org.apache.flink.table.catalog.exceptions.PartitionNotExistException in project flink by apache.

the class GenericInMemoryCatalog method alterPartitionColumnStatistics.

@Override
public void alterPartitionColumnStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogColumnStatistics columnStatistics, boolean ignoreIfNotExists) throws PartitionNotExistException {
    checkNotNull(tablePath);
    checkNotNull(partitionSpec);
    checkNotNull(columnStatistics);
    if (partitionExists(tablePath, partitionSpec)) {
        partitionColumnStats.get(tablePath).put(partitionSpec, columnStatistics.copy());
    } else if (!ignoreIfNotExists) {
        throw new PartitionNotExistException(getName(), tablePath, partitionSpec);
    }
}
Also used : PartitionNotExistException(org.apache.flink.table.catalog.exceptions.PartitionNotExistException)

Example 8 with PartitionNotExistException

use of org.apache.flink.table.catalog.exceptions.PartitionNotExistException in project flink by apache.

the class GenericInMemoryCatalog method dropPartition.

@Override
public void dropPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, boolean ignoreIfNotExists) throws PartitionNotExistException, CatalogException {
    checkNotNull(tablePath);
    checkNotNull(partitionSpec);
    if (partitionExists(tablePath, partitionSpec)) {
        partitions.get(tablePath).remove(partitionSpec);
        partitionStats.get(tablePath).remove(partitionSpec);
        partitionColumnStats.get(tablePath).remove(partitionSpec);
    } else if (!ignoreIfNotExists) {
        throw new PartitionNotExistException(getName(), tablePath, partitionSpec);
    }
}
Also used : PartitionNotExistException(org.apache.flink.table.catalog.exceptions.PartitionNotExistException)

Example 9 with PartitionNotExistException

use of org.apache.flink.table.catalog.exceptions.PartitionNotExistException in project flink by apache.

the class PushPartitionIntoTableSourceScanRule method getPartitionStats.

private Optional<TableStats> getPartitionStats(Catalog catalog, ObjectPath tablePath, Map<String, String> partition) {
    try {
        CatalogPartitionSpec spec = new CatalogPartitionSpec(partition);
        CatalogTableStatistics partitionStat = catalog.getPartitionStatistics(tablePath, spec);
        CatalogColumnStatistics partitionColStat = catalog.getPartitionColumnStatistics(tablePath, spec);
        TableStats stats = CatalogTableStatisticsConverter.convertToTableStats(partitionStat, partitionColStat);
        return Optional.of(stats);
    } catch (PartitionNotExistException e) {
        return Optional.empty();
    }
}
Also used : PartitionNotExistException(org.apache.flink.table.catalog.exceptions.PartitionNotExistException) TableStats(org.apache.flink.table.plan.stats.TableStats) CatalogPartitionSpec(org.apache.flink.table.catalog.CatalogPartitionSpec) CatalogColumnStatistics(org.apache.flink.table.catalog.stats.CatalogColumnStatistics) CatalogTableStatistics(org.apache.flink.table.catalog.stats.CatalogTableStatistics)

Example 10 with PartitionNotExistException

use of org.apache.flink.table.catalog.exceptions.PartitionNotExistException in project flink by apache.

the class HiveCatalog method alterPartition.

@Override
public void alterPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogPartition newPartition, boolean ignoreIfNotExists) throws PartitionNotExistException, CatalogException {
    checkNotNull(tablePath, "Table path cannot be null");
    checkNotNull(partitionSpec, "CatalogPartitionSpec cannot be null");
    checkNotNull(newPartition, "New partition cannot be null");
    // the target doesn't exist
    try {
        Table hiveTable = getHiveTable(tablePath);
        boolean isHiveTable = isHiveTable(hiveTable.getParameters());
        if (!isHiveTable) {
            throw new CatalogException("Currently only supports partition for hive tables");
        }
        Partition hivePartition = getHivePartition(hiveTable, partitionSpec);
        if (hivePartition == null) {
            if (ignoreIfNotExists) {
                return;
            }
            throw new PartitionNotExistException(getName(), tablePath, partitionSpec);
        }
        AlterTableOp op = HiveTableUtil.extractAlterTableOp(newPartition.getProperties());
        if (op == null) {
            throw new CatalogException(ALTER_TABLE_OP + " is missing for alter table operation");
        }
        alterTableViaProperties(op, null, null, hivePartition.getParameters(), newPartition.getProperties(), hivePartition.getSd());
        client.alter_partition(tablePath.getDatabaseName(), tablePath.getObjectName(), hivePartition);
    } catch (NoSuchObjectException e) {
        if (!ignoreIfNotExists) {
            throw new PartitionNotExistException(getName(), tablePath, partitionSpec, e);
        }
    } catch (InvalidOperationException | MetaException | TableNotExistException | PartitionSpecInvalidException e) {
        throw new PartitionNotExistException(getName(), tablePath, partitionSpec, e);
    } catch (TException e) {
        throw new CatalogException(String.format("Failed to alter existing partition with new partition %s of table %s", partitionSpec, tablePath), e);
    }
}
Also used : TException(org.apache.thrift.TException) Partition(org.apache.hadoop.hive.metastore.api.Partition) CatalogPartition(org.apache.flink.table.catalog.CatalogPartition) CatalogTable(org.apache.flink.table.catalog.CatalogTable) SqlCreateHiveTable(org.apache.flink.sql.parser.hive.ddl.SqlCreateHiveTable) Table(org.apache.hadoop.hive.metastore.api.Table) CatalogBaseTable(org.apache.flink.table.catalog.CatalogBaseTable) TableNotExistException(org.apache.flink.table.catalog.exceptions.TableNotExistException) CatalogException(org.apache.flink.table.catalog.exceptions.CatalogException) AlterTableOp(org.apache.flink.sql.parser.hive.ddl.SqlAlterHiveTable.AlterTableOp) InvalidOperationException(org.apache.hadoop.hive.metastore.api.InvalidOperationException) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) PartitionNotExistException(org.apache.flink.table.catalog.exceptions.PartitionNotExistException) PartitionSpecInvalidException(org.apache.flink.table.catalog.exceptions.PartitionSpecInvalidException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException)

Aggregations

PartitionNotExistException (org.apache.flink.table.catalog.exceptions.PartitionNotExistException)13 CatalogException (org.apache.flink.table.catalog.exceptions.CatalogException)7 PartitionSpecInvalidException (org.apache.flink.table.catalog.exceptions.PartitionSpecInvalidException)6 TableNotExistException (org.apache.flink.table.catalog.exceptions.TableNotExistException)6 TException (org.apache.thrift.TException)6 CatalogPartition (org.apache.flink.table.catalog.CatalogPartition)5 Partition (org.apache.hadoop.hive.metastore.api.Partition)5 SqlCreateHiveTable (org.apache.flink.sql.parser.hive.ddl.SqlCreateHiveTable)4 CatalogBaseTable (org.apache.flink.table.catalog.CatalogBaseTable)4 CatalogTable (org.apache.flink.table.catalog.CatalogTable)4 Table (org.apache.hadoop.hive.metastore.api.Table)4 CatalogColumnStatistics (org.apache.flink.table.catalog.stats.CatalogColumnStatistics)3 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)3 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)3 CatalogTableStatistics (org.apache.flink.table.catalog.stats.CatalogTableStatistics)2 ArrayList (java.util.ArrayList)1 List (java.util.List)1 AlterTableOp (org.apache.flink.sql.parser.hive.ddl.SqlAlterHiveTable.AlterTableOp)1 CatalogPartitionImpl (org.apache.flink.table.catalog.CatalogPartitionImpl)1 CatalogPartitionSpec (org.apache.flink.table.catalog.CatalogPartitionSpec)1