Search in sources :

Example 31 with CatalogPartitionSpec

use of org.apache.flink.table.catalog.CatalogPartitionSpec in project flink by apache.

the class SqlToOperationConverter method convertShowPartitions.

/**
 * Convert SHOW PARTITIONS statement.
 */
private Operation convertShowPartitions(SqlShowPartitions sqlShowPartitions) {
    UnresolvedIdentifier unresolvedIdentifier = UnresolvedIdentifier.of(sqlShowPartitions.fullTableName());
    ObjectIdentifier tableIdentifier = catalogManager.qualifyIdentifier(unresolvedIdentifier);
    LinkedHashMap<String, String> partitionKVs = sqlShowPartitions.getPartitionKVs();
    if (partitionKVs != null) {
        CatalogPartitionSpec partitionSpec = new CatalogPartitionSpec(partitionKVs);
        return new ShowPartitionsOperation(tableIdentifier, partitionSpec);
    }
    return new ShowPartitionsOperation(tableIdentifier, null);
}
Also used : UnresolvedIdentifier(org.apache.flink.table.catalog.UnresolvedIdentifier) ShowPartitionsOperation(org.apache.flink.table.operations.ShowPartitionsOperation) CatalogPartitionSpec(org.apache.flink.table.catalog.CatalogPartitionSpec) ObjectIdentifier(org.apache.flink.table.catalog.ObjectIdentifier)

Example 32 with CatalogPartitionSpec

use of org.apache.flink.table.catalog.CatalogPartitionSpec in project flink by apache.

the class TestValuesCatalog method listPartitionsByFilter.

@Override
public List<CatalogPartitionSpec> listPartitionsByFilter(ObjectPath tablePath, List<Expression> filters) throws TableNotExistException, TableNotPartitionedException, CatalogException {
    if (!supportListPartitionByFilter) {
        throw new UnsupportedOperationException("TestValuesCatalog doesn't support list partition by filters");
    }
    List<CatalogPartitionSpec> partitions = listPartitions(tablePath);
    if (partitions.isEmpty()) {
        return partitions;
    }
    CatalogBaseTable table = this.getTable(tablePath);
    TableSchema schema = table.getSchema();
    List<ResolvedExpression> resolvedExpressions = filters.stream().map(filter -> {
        if (filter instanceof ResolvedExpression) {
            return (ResolvedExpression) filter;
        }
        throw new UnsupportedOperationException(String.format("TestValuesCatalog only works with resolved expressions. Get unresolved expression: %s", filter));
    }).collect(Collectors.toList());
    return partitions.stream().filter(partition -> {
        Function<String, Comparable<?>> getter = getValueGetter(partition.getPartitionSpec(), schema);
        return FilterUtils.isRetainedAfterApplyingFilterPredicates(resolvedExpressions, getter);
    }).collect(Collectors.toList());
}
Also used : DataType(org.apache.flink.table.types.DataType) TableNotExistException(org.apache.flink.table.catalog.exceptions.TableNotExistException) FilterUtils(org.apache.flink.table.planner.utils.FilterUtils) IntType(org.apache.flink.table.types.logical.IntType) TableException(org.apache.flink.table.api.TableException) TableSchema(org.apache.flink.table.api.TableSchema) VarCharType(org.apache.flink.table.types.logical.VarCharType) CatalogBaseTable(org.apache.flink.table.catalog.CatalogBaseTable) Expression(org.apache.flink.table.expressions.Expression) ObjectPath(org.apache.flink.table.catalog.ObjectPath) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) CharType(org.apache.flink.table.types.logical.CharType) CatalogPartitionSpec(org.apache.flink.table.catalog.CatalogPartitionSpec) List(java.util.List) TableNotPartitionedException(org.apache.flink.table.catalog.exceptions.TableNotPartitionedException) DoubleType(org.apache.flink.table.types.logical.DoubleType) LogicalType(org.apache.flink.table.types.logical.LogicalType) BooleanType(org.apache.flink.table.types.logical.BooleanType) ResolvedExpression(org.apache.flink.table.expressions.ResolvedExpression) Map(java.util.Map) Optional(java.util.Optional) GenericInMemoryCatalog(org.apache.flink.table.catalog.GenericInMemoryCatalog) CatalogException(org.apache.flink.table.catalog.exceptions.CatalogException) Function(java.util.function.Function) CatalogBaseTable(org.apache.flink.table.catalog.CatalogBaseTable) TableSchema(org.apache.flink.table.api.TableSchema) ResolvedExpression(org.apache.flink.table.expressions.ResolvedExpression) CatalogPartitionSpec(org.apache.flink.table.catalog.CatalogPartitionSpec)

Aggregations

CatalogPartitionSpec (org.apache.flink.table.catalog.CatalogPartitionSpec)32 HashMap (java.util.HashMap)20 LinkedHashMap (java.util.LinkedHashMap)15 ArrayList (java.util.ArrayList)11 Map (java.util.Map)11 ObjectPath (org.apache.flink.table.catalog.ObjectPath)11 List (java.util.List)10 CatalogTable (org.apache.flink.table.catalog.CatalogTable)10 Path (org.apache.flink.core.fs.Path)8 CatalogBaseTable (org.apache.flink.table.catalog.CatalogBaseTable)8 CatalogPartition (org.apache.flink.table.catalog.CatalogPartition)7 Test (org.junit.Test)7 HashSet (java.util.HashSet)6 SqlCreateHiveTable (org.apache.flink.sql.parser.hive.ddl.SqlCreateHiveTable)6 CatalogPartitionImpl (org.apache.flink.table.catalog.CatalogPartitionImpl)6 ObjectIdentifier (org.apache.flink.table.catalog.ObjectIdentifier)6 ValidationException (org.apache.flink.table.api.ValidationException)5 RowData (org.apache.flink.table.data.RowData)5 Partition (org.apache.hadoop.hive.metastore.api.Partition)5 Set (java.util.Set)4