Search in sources :

Example 46 with TableHandle

use of com.facebook.presto.spi.TableHandle in project presto by prestodb.

the class MockRemoteTaskFactory method createTableScanTask.

public MockRemoteTask createTableScanTask(TaskId taskId, InternalNode newNode, List<Split> splits, NodeTaskMap.NodeStatsTracker nodeStatsTracker) {
    VariableReferenceExpression variable = new VariableReferenceExpression(Optional.empty(), "column", VARCHAR);
    PlanNodeId sourceId = new PlanNodeId("sourceId");
    PlanFragment testFragment = new PlanFragment(new PlanFragmentId(0), new TableScanNode(Optional.empty(), sourceId, new TableHandle(new ConnectorId("test"), new TestingTableHandle(), TestingTransactionHandle.create(), Optional.of(TestingHandle.INSTANCE)), ImmutableList.of(variable), ImmutableMap.of(variable, new TestingColumnHandle("column")), TupleDomain.all(), TupleDomain.all()), ImmutableSet.of(variable), SOURCE_DISTRIBUTION, ImmutableList.of(sourceId), new PartitioningScheme(Partitioning.create(SINGLE_DISTRIBUTION, ImmutableList.of()), ImmutableList.of(variable)), StageExecutionDescriptor.ungroupedExecution(), false, StatsAndCosts.empty(), Optional.empty());
    ImmutableMultimap.Builder<PlanNodeId, Split> initialSplits = ImmutableMultimap.builder();
    for (Split sourceSplit : splits) {
        initialSplits.put(sourceId, sourceSplit);
    }
    return createRemoteTask(TEST_SESSION, taskId, newNode, testFragment, initialSplits.build(), createInitialEmptyOutputBuffers(BROADCAST), nodeStatsTracker, true, new TableWriteInfo(Optional.empty(), Optional.empty(), Optional.empty()));
}
Also used : TableWriteInfo(com.facebook.presto.execution.scheduler.TableWriteInfo) TestingTableHandle(com.facebook.presto.testing.TestingMetadata.TestingTableHandle) PartitioningScheme(com.facebook.presto.sql.planner.PartitioningScheme) PlanFragment(com.facebook.presto.sql.planner.PlanFragment) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) TestingColumnHandle(com.facebook.presto.testing.TestingMetadata.TestingColumnHandle) TableScanNode(com.facebook.presto.spi.plan.TableScanNode) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) TableHandle(com.facebook.presto.spi.TableHandle) TestingTableHandle(com.facebook.presto.testing.TestingMetadata.TestingTableHandle) PlanFragmentId(com.facebook.presto.sql.planner.plan.PlanFragmentId) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) Split(com.facebook.presto.metadata.Split) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 47 with TableHandle

use of com.facebook.presto.spi.TableHandle in project presto by prestodb.

the class TestHivePageSink method createPageSource.

private static ConnectorPageSource createPageSource(HiveTransactionHandle transaction, HiveClientConfig config, MetastoreClientConfig metastoreClientConfig, File outputFile) {
    HiveSplit split = new HiveSplit(SCHEMA_NAME, TABLE_NAME, "", "file:///" + outputFile.getAbsolutePath(), 0, outputFile.length(), outputFile.length(), outputFile.lastModified(), new Storage(StorageFormat.create(config.getHiveStorageFormat().getSerDe(), config.getHiveStorageFormat().getInputFormat(), config.getHiveStorageFormat().getOutputFormat()), "location", Optional.empty(), false, ImmutableMap.of(), ImmutableMap.of()), ImmutableList.of(), ImmutableList.of(), OptionalInt.empty(), OptionalInt.empty(), NO_PREFERENCE, getColumnHandles().size(), TableToPartitionMapping.empty(), Optional.empty(), false, Optional.empty(), NO_CACHE_REQUIREMENT, Optional.empty(), ImmutableMap.of(), ImmutableSet.of(), SplitWeight.standard());
    TableHandle tableHandle = new TableHandle(new ConnectorId(HIVE_CATALOG), new HiveTableHandle(SCHEMA_NAME, TABLE_NAME), transaction, Optional.of(new HiveTableLayoutHandle(new SchemaTableName(SCHEMA_NAME, TABLE_NAME), "path", ImmutableList.of(), getColumnHandles().stream().map(column -> new Column(column.getName(), column.getHiveType(), Optional.empty(), Optional.empty())).collect(toImmutableList()), ImmutableMap.of(), TupleDomain.all(), TRUE_CONSTANT, ImmutableMap.of(), TupleDomain.all(), Optional.empty(), Optional.empty(), false, "layout", Optional.empty(), false)));
    HivePageSourceProvider provider = new HivePageSourceProvider(config, createTestHdfsEnvironment(config, metastoreClientConfig), getDefaultHiveRecordCursorProvider(config, metastoreClientConfig), getDefaultHiveBatchPageSourceFactories(config, metastoreClientConfig), getDefaultHiveSelectivePageSourceFactories(config, metastoreClientConfig), FUNCTION_AND_TYPE_MANAGER, ROW_EXPRESSION_SERVICE);
    return provider.createPageSource(transaction, getSession(config), split, tableHandle.getLayout().get(), ImmutableList.copyOf(getColumnHandles()), NON_CACHEABLE);
}
Also used : Storage(com.facebook.presto.hive.metastore.Storage) Column(com.facebook.presto.hive.metastore.Column) LineItemColumn(io.airlift.tpch.LineItemColumn) TableHandle(com.facebook.presto.spi.TableHandle) SchemaTableName(com.facebook.presto.spi.SchemaTableName) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 48 with TableHandle

use of com.facebook.presto.spi.TableHandle in project presto by prestodb.

the class MetadataManager method getTableHandle.

@Override
public Optional<TableHandle> getTableHandle(Session session, QualifiedObjectName table) {
    requireNonNull(table, "table is null");
    Optional<CatalogMetadata> catalog = getOptionalCatalogMetadata(session, table.getCatalogName());
    if (catalog.isPresent()) {
        CatalogMetadata catalogMetadata = catalog.get();
        ConnectorId connectorId = catalogMetadata.getConnectorId(session, table);
        ConnectorMetadata metadata = catalogMetadata.getMetadataFor(connectorId);
        ConnectorTableHandle tableHandle = metadata.getTableHandle(session.toConnectorSession(connectorId), toSchemaTableName(table));
        if (tableHandle != null) {
            return Optional.of(new TableHandle(connectorId, tableHandle, catalogMetadata.getTransactionHandleFor(connectorId), Optional.empty()));
        }
    }
    return Optional.empty();
}
Also used : ConnectorOutputTableHandle(com.facebook.presto.spi.ConnectorOutputTableHandle) TableHandle(com.facebook.presto.spi.TableHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle)

Example 49 with TableHandle

use of com.facebook.presto.spi.TableHandle in project presto by prestodb.

the class MetadataManager method createTemporaryTable.

@Override
public TableHandle createTemporaryTable(Session session, String catalogName, List<ColumnMetadata> columns, Optional<PartitioningMetadata> partitioningMetadata) {
    CatalogMetadata catalogMetadata = getCatalogMetadataForWrite(session, catalogName);
    ConnectorId connectorId = catalogMetadata.getConnectorId();
    ConnectorMetadata metadata = catalogMetadata.getMetadata();
    ConnectorTableHandle connectorTableHandle = metadata.createTemporaryTable(session.toConnectorSession(connectorId), columns, partitioningMetadata.map(partitioning -> createConnectorPartitioningMetadata(connectorId, partitioning)));
    return new TableHandle(connectorId, connectorTableHandle, catalogMetadata.getTransactionHandleFor(connectorId), Optional.empty());
}
Also used : NOT_FOUND(com.facebook.presto.spi.StandardErrorCode.NOT_FOUND) PrestoPrincipal(com.facebook.presto.spi.security.PrestoPrincipal) ComputedStatistics(com.facebook.presto.spi.statistics.ComputedStatistics) JsonCodecFactory(com.facebook.airlift.json.JsonCodecFactory) TypeSignature(com.facebook.presto.common.type.TypeSignature) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) JsonObjectMapperProvider(com.facebook.airlift.json.JsonObjectMapperProvider) Map(java.util.Map) QualifiedObjectName(com.facebook.presto.common.QualifiedObjectName) ConnectorMaterializedViewDefinition(com.facebook.presto.spi.ConnectorMaterializedViewDefinition) SystemTable(com.facebook.presto.spi.SystemTable) ENGLISH(java.util.Locale.ENGLISH) HASH_CODE(com.facebook.presto.common.function.OperatorType.HASH_CODE) InMemoryTransactionManager.createTestTransactionManager(com.facebook.presto.transaction.InMemoryTransactionManager.createTestTransactionManager) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) Set(java.util.Set) ConnectorResolvedIndex(com.facebook.presto.spi.ConnectorResolvedIndex) ConnectorSession(com.facebook.presto.spi.ConnectorSession) FeaturesConfig(com.facebook.presto.sql.analyzer.FeaturesConfig) TableStatisticsMetadata(com.facebook.presto.spi.statistics.TableStatisticsMetadata) Joiner(com.google.common.base.Joiner) ConnectorPartitioningHandle(com.facebook.presto.spi.connector.ConnectorPartitioningHandle) Slice(io.airlift.slice.Slice) ConnectorOutputTableHandle(com.facebook.presto.spi.ConnectorOutputTableHandle) TypeSignatureProvider.fromTypes(com.facebook.presto.sql.analyzer.TypeSignatureProvider.fromTypes) ArrayList(java.util.ArrayList) LinkedHashMap(java.util.LinkedHashMap) OptionalLong(java.util.OptionalLong) BlockEncodingManager(com.facebook.presto.common.block.BlockEncodingManager) ImmutableSet.toImmutableSet(com.google.common.collect.ImmutableSet.toImmutableSet) TableHandle(com.facebook.presto.spi.TableHandle) SqlFunction(com.facebook.presto.spi.function.SqlFunction) TransactionManager(com.facebook.presto.transaction.TransactionManager) LinkedHashSet(java.util.LinkedHashSet) ConnectorTableMetadata(com.facebook.presto.spi.ConnectorTableMetadata) ConnectorMetadataUpdateHandle(com.facebook.presto.spi.ConnectorMetadataUpdateHandle) SystemSessionProperties.isIgnoreStatsCalculatorFailures(com.facebook.presto.SystemSessionProperties.isIgnoreStatsCalculatorFailures) MetadataUtil.toSchemaTableName(com.facebook.presto.metadata.MetadataUtil.toSchemaTableName) Session(com.facebook.presto.Session) Constraint(com.facebook.presto.spi.Constraint) OperatorType(com.facebook.presto.common.function.OperatorType) GREATER_THAN(com.facebook.presto.common.function.OperatorType.GREATER_THAN) ConnectorTableLayoutResult(com.facebook.presto.spi.ConnectorTableLayoutResult) INVALID_VIEW(com.facebook.presto.spi.StandardErrorCode.INVALID_VIEW) SchemaTablePrefix(com.facebook.presto.spi.SchemaTablePrefix) BETWEEN(com.facebook.presto.common.function.OperatorType.BETWEEN) ColumnHandle(com.facebook.presto.spi.ColumnHandle) ViewColumn(com.facebook.presto.metadata.ViewDefinition.ViewColumn) QueryId(com.facebook.presto.spi.QueryId) PartitioningHandle(com.facebook.presto.sql.planner.PartitioningHandle) TableLayoutFilterCoverage(com.facebook.presto.spi.TableLayoutFilterCoverage) TableLayout.fromConnectorLayout(com.facebook.presto.metadata.TableLayout.fromConnectorLayout) ConnectorViewDefinition(com.facebook.presto.spi.ConnectorViewDefinition) ConnectorCapabilities(com.facebook.presto.spi.connector.ConnectorCapabilities) JsonCodec(com.facebook.airlift.json.JsonCodec) TableStatistics(com.facebook.presto.spi.statistics.TableStatistics) Privilege(com.facebook.presto.spi.security.Privilege) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) SchemaTableName(com.facebook.presto.spi.SchemaTableName) HashMultimap(com.google.common.collect.HashMultimap) Locale(java.util.Locale) EQUAL(com.facebook.presto.common.function.OperatorType.EQUAL) LESS_THAN(com.facebook.presto.common.function.OperatorType.LESS_THAN) Function(com.google.common.base.Function) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Collection(java.util.Collection) GET_LAYOUT_TIME_NANOS(com.facebook.presto.common.RuntimeMetricName.GET_LAYOUT_TIME_NANOS) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) String.format(java.lang.String.format) List(java.util.List) ColumnMetadata(com.facebook.presto.spi.ColumnMetadata) RoleGrant(com.facebook.presto.spi.security.RoleGrant) NOT_SUPPORTED(com.facebook.presto.spi.StandardErrorCode.NOT_SUPPORTED) Entry(java.util.Map.Entry) TypeSignatureProvider(com.facebook.presto.sql.analyzer.TypeSignatureProvider) Optional(java.util.Optional) ConnectorId(com.facebook.presto.spi.ConnectorId) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) NOT_EQUAL(com.facebook.presto.common.function.OperatorType.NOT_EQUAL) Logger(com.facebook.airlift.log.Logger) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ConnectorTableLayoutHandle(com.facebook.presto.spi.ConnectorTableLayoutHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) HashMap(java.util.HashMap) QueryManager(com.facebook.presto.execution.QueryManager) PrestoException(com.facebook.presto.spi.PrestoException) Multimap(com.google.common.collect.Multimap) BlockEncodingSerde(com.facebook.presto.common.block.BlockEncodingSerde) ConcurrentMap(java.util.concurrent.ConcurrentMap) Inject(javax.inject.Inject) LESS_THAN_OR_EQUAL(com.facebook.presto.common.function.OperatorType.LESS_THAN_OR_EQUAL) ImmutableList(com.google.common.collect.ImmutableList) Objects.requireNonNull(java.util.Objects.requireNonNull) Type(com.facebook.presto.common.type.Type) ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) GrantInfo(com.facebook.presto.spi.security.GrantInfo) NOT_APPLICABLE(com.facebook.presto.spi.TableLayoutFilterCoverage.NOT_APPLICABLE) ConnectorOutputMetadata(com.facebook.presto.spi.connector.ConnectorOutputMetadata) Constraint.alwaysTrue(com.facebook.presto.spi.Constraint.alwaysTrue) TupleDomain(com.facebook.presto.common.predicate.TupleDomain) GREATER_THAN_OR_EQUAL(com.facebook.presto.common.function.OperatorType.GREATER_THAN_OR_EQUAL) MoreFutures.toListenableFuture(com.facebook.airlift.concurrent.MoreFutures.toListenableFuture) TypeDeserializer(com.facebook.presto.type.TypeDeserializer) SYNTAX_ERROR(com.facebook.presto.spi.StandardErrorCode.SYNTAX_ERROR) ConnectorPartitioningMetadata(com.facebook.presto.spi.connector.ConnectorPartitioningMetadata) CatalogSchemaName(com.facebook.presto.common.CatalogSchemaName) MaterializedViewStatus(com.facebook.presto.spi.MaterializedViewStatus) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectorOutputTableHandle(com.facebook.presto.spi.ConnectorOutputTableHandle) TableHandle(com.facebook.presto.spi.TableHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle)

Example 50 with TableHandle

use of com.facebook.presto.spi.TableHandle in project presto by prestodb.

the class MetadataManager method getMaterializedViewStatus.

@Override
public MaterializedViewStatus getMaterializedViewStatus(Session session, QualifiedObjectName materializedViewName) {
    Optional<TableHandle> materializedViewHandle = getTableHandle(session, materializedViewName);
    ConnectorId connectorId = materializedViewHandle.get().getConnectorId();
    ConnectorMetadata metadata = getMetadata(session, connectorId);
    return metadata.getMaterializedViewStatus(session.toConnectorSession(connectorId), toSchemaTableName(materializedViewName));
}
Also used : ConnectorOutputTableHandle(com.facebook.presto.spi.ConnectorOutputTableHandle) TableHandle(com.facebook.presto.spi.TableHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) ConnectorInsertTableHandle(com.facebook.presto.spi.ConnectorInsertTableHandle) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId)

Aggregations

TableHandle (com.facebook.presto.spi.TableHandle)70 ConnectorId (com.facebook.presto.spi.ConnectorId)37 ColumnHandle (com.facebook.presto.spi.ColumnHandle)29 VariableReferenceExpression (com.facebook.presto.spi.relation.VariableReferenceExpression)22 QualifiedObjectName (com.facebook.presto.common.QualifiedObjectName)21 ConnectorTableHandle (com.facebook.presto.spi.ConnectorTableHandle)21 ImmutableList (com.google.common.collect.ImmutableList)20 ImmutableMap (com.google.common.collect.ImmutableMap)19 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)18 ColumnMetadata (com.facebook.presto.spi.ColumnMetadata)16 Session (com.facebook.presto.Session)15 Type (com.facebook.presto.common.type.Type)15 ConnectorOutputTableHandle (com.facebook.presto.spi.ConnectorOutputTableHandle)15 Optional (java.util.Optional)15 ConnectorMetadata (com.facebook.presto.spi.connector.ConnectorMetadata)14 TableScanNode (com.facebook.presto.spi.plan.TableScanNode)14 List (java.util.List)14 Metadata (com.facebook.presto.metadata.Metadata)13 ConnectorInsertTableHandle (com.facebook.presto.spi.ConnectorInsertTableHandle)13 ConnectorSession (com.facebook.presto.spi.ConnectorSession)13