Search in sources :

Example 96 with ConnectorId

use of com.facebook.presto.spi.ConnectorId in project presto by prestodb.

the class MockRemoteTaskFactory method createTableScanTask.

public MockRemoteTask createTableScanTask(TaskId taskId, InternalNode newNode, List<Split> splits, NodeTaskMap.NodeStatsTracker nodeStatsTracker) {
    VariableReferenceExpression variable = new VariableReferenceExpression(Optional.empty(), "column", VARCHAR);
    PlanNodeId sourceId = new PlanNodeId("sourceId");
    PlanFragment testFragment = new PlanFragment(new PlanFragmentId(0), new TableScanNode(Optional.empty(), sourceId, new TableHandle(new ConnectorId("test"), new TestingTableHandle(), TestingTransactionHandle.create(), Optional.of(TestingHandle.INSTANCE)), ImmutableList.of(variable), ImmutableMap.of(variable, new TestingColumnHandle("column")), TupleDomain.all(), TupleDomain.all()), ImmutableSet.of(variable), SOURCE_DISTRIBUTION, ImmutableList.of(sourceId), new PartitioningScheme(Partitioning.create(SINGLE_DISTRIBUTION, ImmutableList.of()), ImmutableList.of(variable)), StageExecutionDescriptor.ungroupedExecution(), false, StatsAndCosts.empty(), Optional.empty());
    ImmutableMultimap.Builder<PlanNodeId, Split> initialSplits = ImmutableMultimap.builder();
    for (Split sourceSplit : splits) {
        initialSplits.put(sourceId, sourceSplit);
    }
    return createRemoteTask(TEST_SESSION, taskId, newNode, testFragment, initialSplits.build(), createInitialEmptyOutputBuffers(BROADCAST), nodeStatsTracker, true, new TableWriteInfo(Optional.empty(), Optional.empty(), Optional.empty()));
}
Also used : TableWriteInfo(com.facebook.presto.execution.scheduler.TableWriteInfo) TestingTableHandle(com.facebook.presto.testing.TestingMetadata.TestingTableHandle) PartitioningScheme(com.facebook.presto.sql.planner.PartitioningScheme) PlanFragment(com.facebook.presto.sql.planner.PlanFragment) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) TestingColumnHandle(com.facebook.presto.testing.TestingMetadata.TestingColumnHandle) TableScanNode(com.facebook.presto.spi.plan.TableScanNode) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) TableHandle(com.facebook.presto.spi.TableHandle) TestingTableHandle(com.facebook.presto.testing.TestingMetadata.TestingTableHandle) PlanFragmentId(com.facebook.presto.sql.planner.plan.PlanFragmentId) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) Split(com.facebook.presto.metadata.Split) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 97 with ConnectorId

use of com.facebook.presto.spi.ConnectorId in project presto by prestodb.

the class TestInput method testRoundTrip.

@Test
public void testRoundTrip() {
    Input expected = new Input(new ConnectorId("connectorId"), "schema", "table", Optional.empty(), ImmutableList.of(new Column("column1", "string"), new Column("column2", "string"), new Column("column3", "string")), Optional.empty());
    String json = codec.toJson(expected);
    Input actual = codec.fromJson(json);
    assertEquals(actual, expected);
}
Also used : ConnectorId(com.facebook.presto.spi.ConnectorId) Test(org.testng.annotations.Test)

Example 98 with ConnectorId

use of com.facebook.presto.spi.ConnectorId in project presto by prestodb.

the class AbstractAnalyzerTest method createTestingCatalog.

private Catalog createTestingCatalog(String catalogName, ConnectorId connectorId) {
    ConnectorId systemId = createSystemTablesConnectorId(connectorId);
    Connector connector = AbstractAnalyzerTest.createTestingConnector();
    InternalNodeManager nodeManager = new InMemoryNodeManager();
    return new Catalog(catalogName, connectorId, connector, createInformationSchemaConnectorId(connectorId), new InformationSchemaConnector(catalogName, nodeManager, metadata, accessControl, ImmutableList.of()), systemId, new SystemConnector(systemId, nodeManager, connector.getSystemTables(), transactionId -> transactionManager.getConnectorTransaction(transactionId, connectorId)));
}
Also used : SystemConnector(com.facebook.presto.connector.system.SystemConnector) WarningCollector(com.facebook.presto.spi.WarningCollector) Parameter(com.facebook.presto.spi.function.Parameter) JsonCodec(com.facebook.airlift.json.JsonCodec) SqlInvokedFunctionNamespaceManagerConfig(com.facebook.presto.functionNamespace.SqlInvokedFunctionNamespaceManagerConfig) SQL(com.facebook.presto.spi.function.RoutineCharacteristics.Language.SQL) TestingWarningCollectorConfig(com.facebook.presto.testing.TestingWarningCollectorConfig) APPLY_FUNCTION(com.facebook.presto.operator.scalar.ApplyFunction.APPLY_FUNCTION) SqlInvokedFunction(com.facebook.presto.spi.function.SqlInvokedFunction) PropertyMetadata(com.facebook.presto.spi.session.PropertyMetadata) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) InternalNodeManager(com.facebook.presto.metadata.InternalNodeManager) SchemaTableName(com.facebook.presto.spi.SchemaTableName) WarningCollectorConfig(com.facebook.presto.execution.warnings.WarningCollectorConfig) TransactionBuilder.transaction(com.facebook.presto.transaction.TransactionBuilder.transaction) QualifiedObjectName(com.facebook.presto.common.QualifiedObjectName) PropertyMetadata.stringProperty(com.facebook.presto.spi.session.PropertyMetadata.stringProperty) ConnectorSplitManager(com.facebook.presto.spi.connector.ConnectorSplitManager) ConnectorId.createInformationSchemaConnectorId(com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId) ImmutableMap(com.google.common.collect.ImmutableMap) DOUBLE(com.facebook.presto.common.type.DoubleType.DOUBLE) FunctionImplementationType(com.facebook.presto.spi.function.FunctionImplementationType) InMemoryTransactionManager.createTestTransactionManager(com.facebook.presto.transaction.InMemoryTransactionManager.createTestTransactionManager) Collections.emptyList(java.util.Collections.emptyList) BeforeClass(org.testng.annotations.BeforeClass) DETERMINISTIC(com.facebook.presto.spi.function.RoutineCharacteristics.Determinism.DETERMINISTIC) InMemoryFunctionNamespaceManager(com.facebook.presto.functionNamespace.testing.InMemoryFunctionNamespaceManager) RealType(com.facebook.presto.common.type.RealType) TestingMetadata(com.facebook.presto.testing.TestingMetadata) TestingWarningCollector(com.facebook.presto.testing.TestingWarningCollector) String.format(java.lang.String.format) SqlParser(com.facebook.presto.sql.parser.SqlParser) List(java.util.List) ColumnMetadata(com.facebook.presto.spi.ColumnMetadata) RETURNS_NULL_ON_NULL_INPUT(com.facebook.presto.spi.function.RoutineCharacteristics.NullCallClause.RETURNS_NULL_ON_NULL_INPUT) Catalog(com.facebook.presto.metadata.Catalog) Optional(java.util.Optional) SystemConnector(com.facebook.presto.connector.system.SystemConnector) ConnectorId(com.facebook.presto.spi.ConnectorId) AllowAllAccessControl(com.facebook.presto.security.AllowAllAccessControl) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) StandardTypes(com.facebook.presto.common.type.StandardTypes) ViewDefinition(com.facebook.presto.metadata.ViewDefinition) VARCHAR(com.facebook.presto.common.type.VarcharType.VARCHAR) AccessControlManager(com.facebook.presto.security.AccessControlManager) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) ImmutableList(com.google.common.collect.ImmutableList) ConnectorId.createSystemTablesConnectorId(com.facebook.presto.spi.ConnectorId.createSystemTablesConnectorId) ArrayType(com.facebook.presto.common.type.ArrayType) TransactionManager(com.facebook.presto.transaction.TransactionManager) InMemoryNodeManager(com.facebook.presto.metadata.InMemoryNodeManager) RoutineCharacteristics(com.facebook.presto.spi.function.RoutineCharacteristics) ConnectorTableMetadata(com.facebook.presto.spi.ConnectorTableMetadata) Connector(com.facebook.presto.spi.connector.Connector) BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) Language(org.intellij.lang.annotations.Language) Session(com.facebook.presto.Session) NodeLocation(com.facebook.presto.sql.tree.NodeLocation) Assert.fail(org.testng.Assert.fail) TestingSession.testSessionBuilder(com.facebook.presto.testing.TestingSession.testSessionBuilder) CatalogManager(com.facebook.presto.metadata.CatalogManager) Consumer(java.util.function.Consumer) SqlFunctionExecutors(com.facebook.presto.functionNamespace.execution.SqlFunctionExecutors) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) IsolationLevel(com.facebook.presto.spi.transaction.IsolationLevel) MetadataManager.createTestMetadataManager(com.facebook.presto.metadata.MetadataManager.createTestMetadataManager) FunctionVersion.notVersioned(com.facebook.presto.spi.function.FunctionVersion.notVersioned) PropertyMetadata.integerProperty(com.facebook.presto.spi.session.PropertyMetadata.integerProperty) NoopSqlFunctionExecutor(com.facebook.presto.functionNamespace.execution.NoopSqlFunctionExecutor) RowType(com.facebook.presto.common.type.RowType) Metadata(com.facebook.presto.metadata.Metadata) AccessControl(com.facebook.presto.security.AccessControl) Statement(com.facebook.presto.sql.tree.Statement) SystemConnector(com.facebook.presto.connector.system.SystemConnector) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) Connector(com.facebook.presto.spi.connector.Connector) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) InternalNodeManager(com.facebook.presto.metadata.InternalNodeManager) Catalog(com.facebook.presto.metadata.Catalog) ConnectorId.createInformationSchemaConnectorId(com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId) ConnectorId(com.facebook.presto.spi.ConnectorId) ConnectorId.createSystemTablesConnectorId(com.facebook.presto.spi.ConnectorId.createSystemTablesConnectorId) InMemoryNodeManager(com.facebook.presto.metadata.InMemoryNodeManager)

Example 99 with ConnectorId

use of com.facebook.presto.spi.ConnectorId in project presto by prestodb.

the class TestHivePageSink method createPageSource.

private static ConnectorPageSource createPageSource(HiveTransactionHandle transaction, HiveClientConfig config, MetastoreClientConfig metastoreClientConfig, File outputFile) {
    HiveSplit split = new HiveSplit(SCHEMA_NAME, TABLE_NAME, "", "file:///" + outputFile.getAbsolutePath(), 0, outputFile.length(), outputFile.length(), outputFile.lastModified(), new Storage(StorageFormat.create(config.getHiveStorageFormat().getSerDe(), config.getHiveStorageFormat().getInputFormat(), config.getHiveStorageFormat().getOutputFormat()), "location", Optional.empty(), false, ImmutableMap.of(), ImmutableMap.of()), ImmutableList.of(), ImmutableList.of(), OptionalInt.empty(), OptionalInt.empty(), NO_PREFERENCE, getColumnHandles().size(), TableToPartitionMapping.empty(), Optional.empty(), false, Optional.empty(), NO_CACHE_REQUIREMENT, Optional.empty(), ImmutableMap.of(), ImmutableSet.of(), SplitWeight.standard());
    TableHandle tableHandle = new TableHandle(new ConnectorId(HIVE_CATALOG), new HiveTableHandle(SCHEMA_NAME, TABLE_NAME), transaction, Optional.of(new HiveTableLayoutHandle(new SchemaTableName(SCHEMA_NAME, TABLE_NAME), "path", ImmutableList.of(), getColumnHandles().stream().map(column -> new Column(column.getName(), column.getHiveType(), Optional.empty(), Optional.empty())).collect(toImmutableList()), ImmutableMap.of(), TupleDomain.all(), TRUE_CONSTANT, ImmutableMap.of(), TupleDomain.all(), Optional.empty(), Optional.empty(), false, "layout", Optional.empty(), false)));
    HivePageSourceProvider provider = new HivePageSourceProvider(config, createTestHdfsEnvironment(config, metastoreClientConfig), getDefaultHiveRecordCursorProvider(config, metastoreClientConfig), getDefaultHiveBatchPageSourceFactories(config, metastoreClientConfig), getDefaultHiveSelectivePageSourceFactories(config, metastoreClientConfig), FUNCTION_AND_TYPE_MANAGER, ROW_EXPRESSION_SERVICE);
    return provider.createPageSource(transaction, getSession(config), split, tableHandle.getLayout().get(), ImmutableList.copyOf(getColumnHandles()), NON_CACHEABLE);
}
Also used : Storage(com.facebook.presto.hive.metastore.Storage) Column(com.facebook.presto.hive.metastore.Column) LineItemColumn(io.airlift.tpch.LineItemColumn) TableHandle(com.facebook.presto.spi.TableHandle) SchemaTableName(com.facebook.presto.spi.SchemaTableName) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 100 with ConnectorId

use of com.facebook.presto.spi.ConnectorId in project presto by prestodb.

the class MetadataManager method listApplicableRoles.

@Override
public Set<RoleGrant> listApplicableRoles(Session session, PrestoPrincipal principal, String catalog) {
    Optional<CatalogMetadata> catalogMetadata = getOptionalCatalogMetadata(session, catalog);
    if (!catalogMetadata.isPresent()) {
        return ImmutableSet.of();
    }
    ConnectorId connectorId = catalogMetadata.get().getConnectorId();
    ConnectorSession connectorSession = session.toConnectorSession(connectorId);
    ConnectorMetadata metadata = catalogMetadata.get().getMetadataFor(connectorId);
    return ImmutableSet.copyOf(metadata.listApplicableRoles(connectorSession, principal));
}
Also used : ConnectorSession(com.facebook.presto.spi.ConnectorSession) ConnectorMetadata(com.facebook.presto.spi.connector.ConnectorMetadata) ConnectorId(com.facebook.presto.spi.ConnectorId)

Aggregations

ConnectorId (com.facebook.presto.spi.ConnectorId)162 ConnectorMetadata (com.facebook.presto.spi.connector.ConnectorMetadata)75 TableHandle (com.facebook.presto.spi.TableHandle)33 Test (org.testng.annotations.Test)29 ConnectorSession (com.facebook.presto.spi.ConnectorSession)26 ConnectorTableHandle (com.facebook.presto.spi.ConnectorTableHandle)18 PrestoException (com.facebook.presto.spi.PrestoException)18 ConnectorTransactionHandle (com.facebook.presto.spi.connector.ConnectorTransactionHandle)18 Session (com.facebook.presto.Session)16 ImmutableList (com.google.common.collect.ImmutableList)16 QualifiedObjectName (com.facebook.presto.common.QualifiedObjectName)15 List (java.util.List)15 PlanNodeId (com.facebook.presto.spi.plan.PlanNodeId)14 Map (java.util.Map)12 Optional (java.util.Optional)12 InMemoryNodeManager (com.facebook.presto.metadata.InMemoryNodeManager)11 ConnectorOutputTableHandle (com.facebook.presto.spi.ConnectorOutputTableHandle)11 SchemaTableName (com.facebook.presto.spi.SchemaTableName)11 ImmutableMap (com.google.common.collect.ImmutableMap)11 ConnectorInsertTableHandle (com.facebook.presto.spi.ConnectorInsertTableHandle)10