Search in sources :

Example 1 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class InformationSchemaPageSource method addTablesRecords.

private void addTablesRecords(QualifiedTablePrefix prefix) {
    Set<SchemaTableName> tables = listTables(session, metadata, accessControl, prefix);
    Set<SchemaTableName> views = listViews(session, metadata, accessControl, prefix);
    for (SchemaTableName name : union(tables, views)) {
        // if table and view names overlap, the view wins
        String type = views.contains(name) ? "VIEW" : "BASE TABLE";
        addRecord(prefix.getCatalogName(), name.getSchemaName(), name.getTableName(), type, null);
        if (isLimitExhausted()) {
            return;
        }
    }
}
Also used : SchemaTableName(io.trino.spi.connector.SchemaTableName)

Example 2 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class SystemPageSourceProvider method createPageSource.

@Override
public ConnectorPageSource createPageSource(ConnectorTransactionHandle transaction, ConnectorSession session, ConnectorSplit split, ConnectorTableHandle table, List<ColumnHandle> columns, DynamicFilter dynamicFilter) {
    requireNonNull(columns, "columns is null");
    SystemTransactionHandle systemTransaction = (SystemTransactionHandle) transaction;
    SystemSplit systemSplit = (SystemSplit) split;
    SchemaTableName tableName = ((SystemTableHandle) table).getSchemaTableName();
    SystemTable systemTable = tables.getSystemTable(session, tableName).orElseThrow(() -> new TrinoException(NOT_FOUND, format("Table '%s' not found", tableName)));
    List<ColumnMetadata> tableColumns = systemTable.getTableMetadata().getColumns();
    Map<String, Integer> columnsByName = new HashMap<>();
    for (int i = 0; i < tableColumns.size(); i++) {
        ColumnMetadata column = tableColumns.get(i);
        if (columnsByName.put(column.getName(), i) != null) {
            throw new TrinoException(GENERIC_INTERNAL_ERROR, "Duplicate column name: " + column.getName());
        }
    }
    ImmutableList.Builder<Integer> userToSystemFieldIndex = ImmutableList.builder();
    for (ColumnHandle column : columns) {
        String columnName = ((SystemColumnHandle) column).getColumnName();
        Integer index = columnsByName.get(columnName);
        if (index == null) {
            throw new TrinoException(GENERIC_INTERNAL_ERROR, format("Column does not exist: %s.%s", tableName, columnName));
        }
        userToSystemFieldIndex.add(index);
    }
    TupleDomain<ColumnHandle> constraint = systemSplit.getConstraint();
    if (constraint.isNone()) {
        return new EmptyPageSource();
    }
    TupleDomain<Integer> newConstraint = systemSplit.getConstraint().transformKeys(columnHandle -> columnsByName.get(((SystemColumnHandle) columnHandle).getColumnName()));
    try {
        return new MappedPageSource(systemTable.pageSource(systemTransaction.getConnectorTransactionHandle(), session, newConstraint), userToSystemFieldIndex.build());
    } catch (UnsupportedOperationException e) {
        return new RecordPageSource(new MappedRecordSet(toRecordSet(systemTransaction.getConnectorTransactionHandle(), systemTable, session, newConstraint), userToSystemFieldIndex.build()));
    }
}
Also used : ColumnHandle(io.trino.spi.connector.ColumnHandle) ColumnMetadata(io.trino.spi.connector.ColumnMetadata) HashMap(java.util.HashMap) ImmutableList(com.google.common.collect.ImmutableList) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) SchemaTableName(io.trino.spi.connector.SchemaTableName) RecordPageSource(io.trino.spi.connector.RecordPageSource) MappedPageSource(io.trino.split.MappedPageSource) EmptyPageSource(io.trino.spi.connector.EmptyPageSource) MappedRecordSet(io.trino.split.MappedRecordSet) TrinoException(io.trino.spi.TrinoException) SystemTable(io.trino.spi.connector.SystemTable)

Example 3 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class SystemTablesMetadata method listTableColumns.

@Override
public Map<SchemaTableName, List<ColumnMetadata>> listTableColumns(ConnectorSession session, SchemaTablePrefix prefix) {
    requireNonNull(prefix, "prefix is null");
    if (prefix.getTable().isPresent()) {
        // if table is concrete we just use tables.getSystemTable to support tables which are not listable
        SchemaTableName tableName = prefix.toSchemaTableName();
        return tables.getSystemTable(session, tableName).map(systemTable -> singletonMap(tableName, systemTable.getTableMetadata().getColumns())).orElseGet(ImmutableMap::of);
    }
    ImmutableMap.Builder<SchemaTableName, List<ColumnMetadata>> builder = ImmutableMap.builder();
    for (SystemTable table : tables.listSystemTables(session)) {
        ConnectorTableMetadata tableMetadata = table.getTableMetadata();
        if (prefix.matches(tableMetadata.getTable())) {
            builder.put(tableMetadata.getTable(), tableMetadata.getColumns());
        }
    }
    return builder.buildOrThrow();
}
Also used : Constraint(io.trino.spi.connector.Constraint) ColumnMetadata(io.trino.spi.connector.ColumnMetadata) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) SystemColumnHandle.toSystemColumnHandles(io.trino.connector.system.SystemColumnHandle.toSystemColumnHandles) ConnectorTableHandle(io.trino.spi.connector.ConnectorTableHandle) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) ColumnHandle(io.trino.spi.connector.ColumnHandle) Collections.singletonMap(java.util.Collections.singletonMap) JdbcTable(io.trino.connector.system.jdbc.JdbcTable) MetadataUtil.findColumnMetadata(io.trino.metadata.MetadataUtil.findColumnMetadata) NOT_FOUND(io.trino.spi.StandardErrorCode.NOT_FOUND) ConstraintApplicationResult(io.trino.spi.connector.ConstraintApplicationResult) ImmutableMap(com.google.common.collect.ImmutableMap) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) TrinoException(io.trino.spi.TrinoException) ConnectorSession(io.trino.spi.connector.ConnectorSession) TupleDomain(io.trino.spi.predicate.TupleDomain) Sets(com.google.common.collect.Sets) SchemaTableName(io.trino.spi.connector.SchemaTableName) String.format(java.lang.String.format) List(java.util.List) ConnectorTableProperties(io.trino.spi.connector.ConnectorTableProperties) SchemaTablePrefix(io.trino.spi.connector.SchemaTablePrefix) Optional(java.util.Optional) ConnectorMetadata(io.trino.spi.connector.ConnectorMetadata) SystemTable(io.trino.spi.connector.SystemTable) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) List(java.util.List) SystemTable(io.trino.spi.connector.SystemTable) SchemaTableName(io.trino.spi.connector.SchemaTableName) ImmutableMap(com.google.common.collect.ImmutableMap) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata)

Example 4 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class ColumnJdbcTable method applyFilter.

@Override
public TupleDomain<ColumnHandle> applyFilter(ConnectorSession connectorSession, Constraint constraint) {
    TupleDomain<ColumnHandle> tupleDomain = constraint.getSummary();
    if (tupleDomain.isNone() || constraint.predicate().isEmpty()) {
        return tupleDomain;
    }
    Predicate<Map<ColumnHandle, NullableValue>> predicate = constraint.predicate().get();
    Set<ColumnHandle> predicateColumns = constraint.getPredicateColumns().orElseThrow(() -> new VerifyException("columns not present for a predicate"));
    boolean hasSchemaPredicate = predicateColumns.contains(TABLE_SCHEMA_COLUMN);
    boolean hasTablePredicate = predicateColumns.contains(TABLE_NAME_COLUMN);
    if (!hasSchemaPredicate && !hasTablePredicate) {
        // No filter on schema name and table name at all.
        return tupleDomain;
    }
    Session session = ((FullConnectorSession) connectorSession).getSession();
    Optional<String> catalogFilter = tryGetSingleVarcharValue(tupleDomain, TABLE_CATALOG_COLUMN);
    Optional<String> schemaFilter = tryGetSingleVarcharValue(tupleDomain, TABLE_SCHEMA_COLUMN);
    Optional<String> tableFilter = tryGetSingleVarcharValue(tupleDomain, TABLE_NAME_COLUMN);
    if (schemaFilter.isPresent() && tableFilter.isPresent()) {
        // No need to narrow down the domain.
        return tupleDomain;
    }
    List<String> catalogs = listCatalogs(session, metadata, accessControl, catalogFilter).keySet().stream().filter(catalogName -> predicate.test(ImmutableMap.of(TABLE_CATALOG_COLUMN, toNullableValue(catalogName)))).collect(toImmutableList());
    List<CatalogSchemaName> schemas = catalogs.stream().flatMap(catalogName -> listSchemas(session, metadata, accessControl, catalogName, schemaFilter).stream().filter(schemaName -> !hasSchemaPredicate || predicate.test(ImmutableMap.of(TABLE_CATALOG_COLUMN, toNullableValue(catalogName), TABLE_SCHEMA_COLUMN, toNullableValue(schemaName)))).map(schemaName -> new CatalogSchemaName(catalogName, schemaName))).collect(toImmutableList());
    if (!hasTablePredicate) {
        return TupleDomain.withColumnDomains(ImmutableMap.<ColumnHandle, Domain>builder().put(TABLE_CATALOG_COLUMN, schemas.stream().map(CatalogSchemaName::getCatalogName).collect(toVarcharDomain()).simplify(MAX_DOMAIN_SIZE)).put(TABLE_SCHEMA_COLUMN, schemas.stream().map(CatalogSchemaName::getSchemaName).collect(toVarcharDomain()).simplify(MAX_DOMAIN_SIZE)).buildOrThrow());
    }
    List<CatalogSchemaTableName> tables = schemas.stream().flatMap(schema -> {
        QualifiedTablePrefix tablePrefix = tableFilter.isPresent() ? new QualifiedTablePrefix(schema.getCatalogName(), schema.getSchemaName(), tableFilter.get()) : new QualifiedTablePrefix(schema.getCatalogName(), schema.getSchemaName());
        return listTables(session, metadata, accessControl, tablePrefix).stream().filter(schemaTableName -> predicate.test(ImmutableMap.of(TABLE_CATALOG_COLUMN, toNullableValue(schema.getCatalogName()), TABLE_SCHEMA_COLUMN, toNullableValue(schemaTableName.getSchemaName()), TABLE_NAME_COLUMN, toNullableValue(schemaTableName.getTableName())))).map(schemaTableName -> new CatalogSchemaTableName(schema.getCatalogName(), schemaTableName.getSchemaName(), schemaTableName.getTableName()));
    }).collect(toImmutableList());
    return TupleDomain.withColumnDomains(ImmutableMap.<ColumnHandle, Domain>builder().put(TABLE_CATALOG_COLUMN, tables.stream().map(CatalogSchemaTableName::getCatalogName).collect(toVarcharDomain()).simplify(MAX_DOMAIN_SIZE)).put(TABLE_SCHEMA_COLUMN, tables.stream().map(catalogSchemaTableName -> catalogSchemaTableName.getSchemaTableName().getSchemaName()).collect(toVarcharDomain()).simplify(MAX_DOMAIN_SIZE)).put(TABLE_NAME_COLUMN, tables.stream().map(catalogSchemaTableName -> catalogSchemaTableName.getSchemaTableName().getTableName()).collect(toVarcharDomain()).simplify(MAX_DOMAIN_SIZE)).buildOrThrow());
}
Also used : FilterUtil.tryGetSingleVarcharValue(io.trino.connector.system.jdbc.FilterUtil.tryGetSingleVarcharValue) TableMetadataBuilder.tableMetadataBuilder(io.trino.metadata.MetadataUtil.TableMetadataBuilder.tableMetadataBuilder) TimestampWithTimeZoneType(io.trino.spi.type.TimestampWithTimeZoneType) Slices(io.airlift.slice.Slices) Map(java.util.Map) CatalogSchemaName(io.trino.spi.connector.CatalogSchemaName) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice) Collector(java.util.stream.Collector) INTEGER(io.trino.spi.type.IntegerType.INTEGER) ENGLISH(java.util.Locale.ENGLISH) SMALLINT(io.trino.spi.type.SmallintType.SMALLINT) MetadataListing.listCatalogs(io.trino.metadata.MetadataListing.listCatalogs) ImmutableMap(com.google.common.collect.ImmutableMap) TypeUtils.getDisplayLabel(io.trino.type.TypeUtils.getDisplayLabel) Predicate(java.util.function.Predicate) Domain(io.trino.spi.predicate.Domain) Collection(java.util.Collection) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) Set(java.util.Set) ArrayType(io.trino.spi.type.ArrayType) Math.min(java.lang.Math.min) Collectors(java.util.stream.Collectors) SchemaTableName(io.trino.spi.connector.SchemaTableName) ZoneId(java.time.ZoneId) List(java.util.List) AccessControl(io.trino.security.AccessControl) BIGINT(io.trino.spi.type.BigintType.BIGINT) CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) Entry(java.util.Map.Entry) Optional(java.util.Optional) DecimalType(io.trino.spi.type.DecimalType) DATE(io.trino.spi.type.DateType.DATE) REAL(io.trino.spi.type.RealType.REAL) Session(io.trino.Session) TimeWithTimeZoneType(io.trino.spi.type.TimeWithTimeZoneType) Types(java.sql.Types) Constraint(io.trino.spi.connector.Constraint) TimeType(io.trino.spi.type.TimeType) FullConnectorSession(io.trino.FullConnectorSession) NullableValue(io.trino.spi.predicate.NullableValue) ColumnMetadata(io.trino.spi.connector.ColumnMetadata) Type(io.trino.spi.type.Type) BOOLEAN(io.trino.spi.type.BooleanType.BOOLEAN) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata) DatabaseMetaData(java.sql.DatabaseMetaData) TimestampType(io.trino.spi.type.TimestampType) Inject(javax.inject.Inject) VarcharType(io.trino.spi.type.VarcharType) Objects.requireNonNull(java.util.Objects.requireNonNull) ColumnHandle(io.trino.spi.connector.ColumnHandle) ImmutableSet.toImmutableSet(com.google.common.collect.ImmutableSet.toImmutableSet) VARBINARY(io.trino.spi.type.VarbinaryType.VARBINARY) QualifiedTablePrefix(io.trino.metadata.QualifiedTablePrefix) MetadataListing.listTables(io.trino.metadata.MetadataListing.listTables) VerifyException(com.google.common.base.VerifyException) MetadataListing.listSchemas(io.trino.metadata.MetadataListing.listSchemas) RecordCursor(io.trino.spi.connector.RecordCursor) MetadataListing.listTableColumns(io.trino.metadata.MetadataListing.listTableColumns) ConnectorSession(io.trino.spi.connector.ConnectorSession) TupleDomain(io.trino.spi.predicate.TupleDomain) InMemoryRecordSet(io.trino.spi.connector.InMemoryRecordSet) Builder(io.trino.spi.connector.InMemoryRecordSet.Builder) DOUBLE(io.trino.spi.type.DoubleType.DOUBLE) SystemColumnHandle(io.trino.connector.system.SystemColumnHandle) CharType(io.trino.spi.type.CharType) SystemSessionProperties.isOmitDateTimeTypePrecision(io.trino.SystemSessionProperties.isOmitDateTimeTypePrecision) Metadata(io.trino.metadata.Metadata) FilterUtil.tablePrefix(io.trino.connector.system.jdbc.FilterUtil.tablePrefix) TINYINT(io.trino.spi.type.TinyintType.TINYINT) ConnectorTransactionHandle(io.trino.spi.connector.ConnectorTransactionHandle) ColumnHandle(io.trino.spi.connector.ColumnHandle) SystemColumnHandle(io.trino.connector.system.SystemColumnHandle) CatalogSchemaTableName(io.trino.spi.connector.CatalogSchemaTableName) QualifiedTablePrefix(io.trino.metadata.QualifiedTablePrefix) VerifyException(com.google.common.base.VerifyException) CatalogSchemaName(io.trino.spi.connector.CatalogSchemaName) Domain(io.trino.spi.predicate.Domain) TupleDomain(io.trino.spi.predicate.TupleDomain) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) Session(io.trino.Session) FullConnectorSession(io.trino.FullConnectorSession) ConnectorSession(io.trino.spi.connector.ConnectorSession) FullConnectorSession(io.trino.FullConnectorSession)

Example 5 with SchemaTableName

use of io.trino.spi.connector.SchemaTableName in project trino by trinodb.

the class TableCommentSystemTable method getComment.

private Optional<String> getComment(Session session, QualifiedTablePrefix prefix, SchemaTableName name, Map<SchemaTableName, ViewInfo> views, Map<SchemaTableName, ViewInfo> materializedViews) {
    ViewInfo materializedViewDefinition = materializedViews.get(name);
    if (materializedViewDefinition != null) {
        return materializedViewDefinition.getComment();
    }
    ViewInfo viewInfo = views.get(name);
    if (viewInfo != null) {
        return viewInfo.getComment();
    }
    QualifiedObjectName tableName = new QualifiedObjectName(prefix.getCatalogName(), name.getSchemaName(), name.getTableName());
    return metadata.getRedirectionAwareTableHandle(session, tableName).getTableHandle().map(handle -> metadata.getTableMetadata(session, handle)).map(metadata -> metadata.getMetadata().getComment()).orElseGet(() -> {
        // A previously listed table might have been dropped concurrently
        LOG.warn("Failed to get metadata for table: %s", name);
        return Optional.empty();
    });
}
Also used : FilterUtil.tryGetSingleVarcharValue(io.trino.connector.system.jdbc.FilterUtil.tryGetSingleVarcharValue) TableMetadataBuilder.tableMetadataBuilder(io.trino.metadata.MetadataUtil.TableMetadataBuilder.tableMetadataBuilder) MetadataListing.getViews(io.trino.metadata.MetadataListing.getViews) Logger(io.airlift.log.Logger) FullConnectorSession(io.trino.FullConnectorSession) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) ConnectorTableMetadata(io.trino.spi.connector.ConnectorTableMetadata) Inject(javax.inject.Inject) Map(java.util.Map) Sets.union(com.google.common.collect.Sets.union) Objects.requireNonNull(java.util.Objects.requireNonNull) ViewInfo(io.trino.metadata.ViewInfo) QualifiedTablePrefix(io.trino.metadata.QualifiedTablePrefix) MetadataListing.listTables(io.trino.metadata.MetadataListing.listTables) MetadataListing.listCatalogs(io.trino.metadata.MetadataListing.listCatalogs) SINGLE_COORDINATOR(io.trino.spi.connector.SystemTable.Distribution.SINGLE_COORDINATOR) ImmutableSet(com.google.common.collect.ImmutableSet) RecordCursor(io.trino.spi.connector.RecordCursor) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) TrinoException(io.trino.spi.TrinoException) ConnectorSession(io.trino.spi.connector.ConnectorSession) TupleDomain(io.trino.spi.predicate.TupleDomain) InMemoryRecordSet(io.trino.spi.connector.InMemoryRecordSet) Builder(io.trino.spi.connector.InMemoryRecordSet.Builder) SchemaTableName(io.trino.spi.connector.SchemaTableName) QualifiedObjectName(io.trino.metadata.QualifiedObjectName) AccessControl(io.trino.security.AccessControl) MetadataListing.getMaterializedViews(io.trino.metadata.MetadataListing.getMaterializedViews) Metadata(io.trino.metadata.Metadata) Optional(java.util.Optional) FilterUtil.tablePrefix(io.trino.connector.system.jdbc.FilterUtil.tablePrefix) SystemTable(io.trino.spi.connector.SystemTable) Session(io.trino.Session) ConnectorTransactionHandle(io.trino.spi.connector.ConnectorTransactionHandle) QualifiedObjectName(io.trino.metadata.QualifiedObjectName) ViewInfo(io.trino.metadata.ViewInfo)

Aggregations

SchemaTableName (io.trino.spi.connector.SchemaTableName)446 Test (org.testng.annotations.Test)212 ImmutableList (com.google.common.collect.ImmutableList)131 ImmutableMap (com.google.common.collect.ImmutableMap)106 List (java.util.List)102 TrinoException (io.trino.spi.TrinoException)100 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)98 ConnectorSession (io.trino.spi.connector.ConnectorSession)98 CatalogSchemaTableName (io.trino.spi.connector.CatalogSchemaTableName)92 TableNotFoundException (io.trino.spi.connector.TableNotFoundException)89 ColumnMetadata (io.trino.spi.connector.ColumnMetadata)86 ConnectorTableMetadata (io.trino.spi.connector.ConnectorTableMetadata)86 Optional (java.util.Optional)78 Map (java.util.Map)67 ColumnHandle (io.trino.spi.connector.ColumnHandle)66 TupleDomain (io.trino.spi.predicate.TupleDomain)59 ConnectorMetadata (io.trino.spi.connector.ConnectorMetadata)58 Path (org.apache.hadoop.fs.Path)55 ConnectorTableHandle (io.trino.spi.connector.ConnectorTableHandle)53 ImmutableSet (com.google.common.collect.ImmutableSet)52