Search in sources :

Example 1 with ViewDto

use of com.netflix.metacat.common.dto.ViewDto in project metacat by Netflix.

the class HiveConvertersImpl method hiveToMetacatTable.

/**
 * {@inheritDoc}
 */
@Override
public TableDto hiveToMetacatTable(final QualifiedName name, final Table table) {
    final TableDto dto = new TableDto();
    dto.setSerde(toStorageDto(table.getSd(), table.getOwner()));
    dto.setAudit(new AuditDto());
    dto.setName(name);
    if (table.isSetCreateTime()) {
        dto.getAudit().setCreatedDate(epochSecondsToDate(table.getCreateTime()));
    }
    dto.setMetadata(table.getParameters());
    final List<FieldSchema> nonPartitionColumns = table.getSd().getCols();
    final List<FieldSchema> partitionColumns = table.getPartitionKeys();
    final List<FieldDto> allFields = Lists.newArrayListWithCapacity(nonPartitionColumns.size() + partitionColumns.size());
    nonPartitionColumns.stream().map(field -> this.hiveToMetacatField(field, false)).forEachOrdered(allFields::add);
    partitionColumns.stream().map(field -> this.hiveToMetacatField(field, true)).forEachOrdered(allFields::add);
    dto.setFields(allFields);
    dto.setView(new ViewDto(table.getViewOriginalText(), table.getViewExpandedText()));
    return dto;
}
Also used : TableDto(com.netflix.metacat.common.dto.TableDto) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) Date(java.util.Date) HashMap(java.util.HashMap) DatabaseDto(com.netflix.metacat.common.dto.DatabaseDto) StorageDto(com.netflix.metacat.common.dto.StorageDto) SerDeInfo(org.apache.hadoop.hive.metastore.api.SerDeInfo) Partition(org.apache.hadoop.hive.metastore.api.Partition) Warehouse(org.apache.hadoop.hive.metastore.Warehouse) LinkedHashMap(java.util.LinkedHashMap) Strings(com.google.common.base.Strings) Lists(com.google.common.collect.Lists) ImmutableList(com.google.common.collect.ImmutableList) ViewDto(com.netflix.metacat.common.dto.ViewDto) Map(java.util.Map) AuditDto(com.netflix.metacat.common.dto.AuditDto) Splitter(com.google.common.base.Splitter) StorageDescriptor(org.apache.hadoop.hive.metastore.api.StorageDescriptor) Nullable(javax.annotation.Nullable) QualifiedName(com.netflix.metacat.common.QualifiedName) FieldDto(com.netflix.metacat.common.dto.FieldDto) Instant(java.time.Instant) Maps(com.google.common.collect.Maps) Collectors(java.util.stream.Collectors) Table(org.apache.hadoop.hive.metastore.api.Table) FieldSchema(org.apache.hadoop.hive.metastore.api.FieldSchema) List(java.util.List) PartitionDto(com.netflix.metacat.common.dto.PartitionDto) TableType(org.apache.hadoop.hive.metastore.TableType) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Database(org.apache.hadoop.hive.metastore.api.Database) Collections(java.util.Collections) ViewDto(com.netflix.metacat.common.dto.ViewDto) AuditDto(com.netflix.metacat.common.dto.AuditDto) FieldSchema(org.apache.hadoop.hive.metastore.api.FieldSchema) TableDto(com.netflix.metacat.common.dto.TableDto) FieldDto(com.netflix.metacat.common.dto.FieldDto)

Example 2 with ViewDto

use of com.netflix.metacat.common.dto.ViewDto in project metacat by Netflix.

the class HiveConvertersImpl method updateTableTypeAndViewInfo.

private void updateTableTypeAndViewInfo(final TableDto dto, final Table table) {
    final ViewDto viewDto = dto.getView();
    if (null == dto.getView() || Strings.isNullOrEmpty(viewDto.getViewOriginalText())) {
        table.setTableType(TableType.EXTERNAL_TABLE.name());
        return;
    }
    table.setTableType(TableType.VIRTUAL_VIEW.name());
    table.setViewOriginalText(viewDto.getViewOriginalText());
    table.setViewExpandedText(viewDto.getViewExpandedText());
}
Also used : ViewDto(com.netflix.metacat.common.dto.ViewDto)

Aggregations

ViewDto (com.netflix.metacat.common.dto.ViewDto)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 Splitter (com.google.common.base.Splitter)1 Strings (com.google.common.base.Strings)1 ImmutableList (com.google.common.collect.ImmutableList)1 Lists (com.google.common.collect.Lists)1 Maps (com.google.common.collect.Maps)1 QualifiedName (com.netflix.metacat.common.QualifiedName)1 AuditDto (com.netflix.metacat.common.dto.AuditDto)1 DatabaseDto (com.netflix.metacat.common.dto.DatabaseDto)1 FieldDto (com.netflix.metacat.common.dto.FieldDto)1 PartitionDto (com.netflix.metacat.common.dto.PartitionDto)1 StorageDto (com.netflix.metacat.common.dto.StorageDto)1 TableDto (com.netflix.metacat.common.dto.TableDto)1 Instant (java.time.Instant)1 Collections (java.util.Collections)1 Date (java.util.Date)1 HashMap (java.util.HashMap)1 LinkedHashMap (java.util.LinkedHashMap)1 List (java.util.List)1