Search in sources :

Example 6 with Connector

use of com.facebook.presto.spi.connector.Connector in project presto by prestodb.

the class TpcdsConnectorFactory method create.

@Override
public Connector create(String catalogName, Map<String, String> config, ConnectorContext context) {
    int splitsPerNode = getSplitsPerNode(config);
    NodeManager nodeManager = context.getNodeManager();
    return new Connector() {

        @Override
        public ConnectorTransactionHandle beginTransaction(IsolationLevel isolationLevel, boolean readOnly) {
            return TpcdsTransactionHandle.INSTANCE;
        }

        @Override
        public ConnectorMetadata getMetadata(ConnectorTransactionHandle transactionHandle) {
            return new TpcdsMetadata();
        }

        @Override
        public ConnectorSplitManager getSplitManager() {
            return new TpcdsSplitManager(nodeManager, splitsPerNode, isWithNoSexism(config));
        }

        @Override
        public ConnectorRecordSetProvider getRecordSetProvider() {
            return new TpcdsRecordSetProvider();
        }

        @Override
        public ConnectorNodePartitioningProvider getNodePartitioningProvider() {
            return new TpcdsNodePartitioningProvider(nodeManager, splitsPerNode);
        }
    };
}
Also used : NodeManager(com.facebook.presto.spi.NodeManager) Connector(com.facebook.presto.spi.connector.Connector) IsolationLevel(com.facebook.presto.spi.transaction.IsolationLevel) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle)

Example 7 with Connector

use of com.facebook.presto.spi.connector.Connector in project presto by prestodb.

the class TestThriftPlugin method testPlugin.

@Test
public void testPlugin() {
    ThriftPlugin plugin = loadPlugin(ThriftPlugin.class);
    ConnectorFactory factory = getOnlyElement(plugin.getConnectorFactories());
    assertInstanceOf(factory, ThriftConnectorFactory.class);
    Map<String, String> config = ImmutableMap.of("presto.thrift.client.addresses", "localhost:7779");
    Connector connector = factory.create("test", config, new TestingConnectorContext());
    assertNotNull(connector);
    assertInstanceOf(connector, ThriftConnector.class);
}
Also used : Connector(com.facebook.presto.spi.connector.Connector) ConnectorFactory(com.facebook.presto.spi.connector.ConnectorFactory) TestingConnectorContext(com.facebook.presto.testing.TestingConnectorContext) Test(org.testng.annotations.Test)

Example 8 with Connector

use of com.facebook.presto.spi.connector.Connector in project presto by prestodb.

the class TestAccessControlManager method registerBogusConnector.

private static ConnectorId registerBogusConnector(CatalogManager catalogManager, TransactionManager transactionManager, AccessControl accessControl, String catalogName) {
    ConnectorId connectorId = new ConnectorId(catalogName);
    Connector connector = new TpchConnectorFactory().create(catalogName, ImmutableMap.of(), new TestingConnectorContext());
    InMemoryNodeManager nodeManager = new InMemoryNodeManager();
    MetadataManager metadata = MetadataManager.createTestMetadataManager(catalogManager);
    ConnectorId systemId = createSystemTablesConnectorId(connectorId);
    catalogManager.registerCatalog(new Catalog(catalogName, connectorId, connector, createInformationSchemaConnectorId(connectorId), new InformationSchemaConnector(catalogName, nodeManager, metadata, accessControl, ImmutableList.of()), systemId, new SystemConnector(systemId, nodeManager, connector.getSystemTables(), transactionId -> transactionManager.getConnectorTransaction(transactionId, connectorId))));
    return connectorId;
}
Also used : SystemConnector(com.facebook.presto.connector.system.SystemConnector) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) Connector(com.facebook.presto.spi.connector.Connector) SystemConnector(com.facebook.presto.connector.system.SystemConnector) TpchConnectorFactory(com.facebook.presto.tpch.TpchConnectorFactory) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) MetadataManager(com.facebook.presto.metadata.MetadataManager) TestingConnectorContext(com.facebook.presto.testing.TestingConnectorContext) Catalog(com.facebook.presto.metadata.Catalog) ConnectorId.createSystemTablesConnectorId(com.facebook.presto.spi.ConnectorId.createSystemTablesConnectorId) ConnectorId.createInformationSchemaConnectorId(com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId) ConnectorId(com.facebook.presto.spi.ConnectorId) InMemoryNodeManager(com.facebook.presto.metadata.InMemoryNodeManager)

Example 9 with Connector

use of com.facebook.presto.spi.connector.Connector in project presto by prestodb.

the class AtopConnectorFactory method create.

@Override
public Connector create(String catalogName, Map<String, String> requiredConfig, ConnectorContext context) {
    requireNonNull(requiredConfig, "requiredConfig is null");
    try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(classLoader)) {
        Bootstrap app = new Bootstrap(new AtopModule(atopFactoryClass, context.getTypeManager(), context.getNodeManager(), context.getNodeManager().getEnvironment(), catalogName), installModuleIf(AtopConnectorConfig.class, config -> config.getSecurity().equalsIgnoreCase(SECURITY_NONE), new AllowAllAccessControlModule()), installModuleIf(AtopConnectorConfig.class, config -> config.getSecurity().equalsIgnoreCase(SECURITY_FILE), binder -> {
            binder.install(new FileBasedAccessControlModule());
            binder.install(new JsonModule());
        }));
        Injector injector = app.doNotInitializeLogging().setRequiredConfigurationProperties(requiredConfig).initialize();
        return injector.getInstance(AtopConnector.class);
    } catch (Exception e) {
        throwIfUnchecked(e);
        throw new RuntimeException(e);
    }
}
Also used : Connector(com.facebook.presto.spi.connector.Connector) SECURITY_FILE(com.facebook.presto.atop.AtopConnectorConfig.SECURITY_FILE) Bootstrap(com.facebook.airlift.bootstrap.Bootstrap) ConnectorHandleResolver(com.facebook.presto.spi.ConnectorHandleResolver) ConditionalModule.installModuleIf(com.facebook.airlift.configuration.ConditionalModule.installModuleIf) SECURITY_NONE(com.facebook.presto.atop.AtopConnectorConfig.SECURITY_NONE) Throwables.throwIfUnchecked(com.google.common.base.Throwables.throwIfUnchecked) AllowAllAccessControlModule(com.facebook.presto.plugin.base.security.AllowAllAccessControlModule) ConnectorFactory(com.facebook.presto.spi.connector.ConnectorFactory) Injector(com.google.inject.Injector) FileBasedAccessControlModule(com.facebook.presto.plugin.base.security.FileBasedAccessControlModule) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) ThreadContextClassLoader(com.facebook.presto.spi.classloader.ThreadContextClassLoader) ConnectorContext(com.facebook.presto.spi.connector.ConnectorContext) JsonModule(com.facebook.airlift.json.JsonModule) Injector(com.google.inject.Injector) AllowAllAccessControlModule(com.facebook.presto.plugin.base.security.AllowAllAccessControlModule) Bootstrap(com.facebook.airlift.bootstrap.Bootstrap) FileBasedAccessControlModule(com.facebook.presto.plugin.base.security.FileBasedAccessControlModule) ThreadContextClassLoader(com.facebook.presto.spi.classloader.ThreadContextClassLoader) JsonModule(com.facebook.airlift.json.JsonModule)

Example 10 with Connector

use of com.facebook.presto.spi.connector.Connector in project presto by prestodb.

the class ConnectorManager method addCatalogConnector.

private synchronized void addCatalogConnector(String catalogName, ConnectorId connectorId, ConnectorFactory factory, Map<String, String> properties) {
    // create all connectors before adding, so a broken connector does not leave the system half updated
    MaterializedConnector connector = new MaterializedConnector(connectorId, createConnector(connectorId, factory, properties));
    MaterializedConnector informationSchemaConnector = new MaterializedConnector(createInformationSchemaConnectorId(connectorId), new InformationSchemaConnector(catalogName, nodeManager, metadataManager, accessControlManager, connector.getSessionProperties()));
    ConnectorId systemId = createSystemTablesConnectorId(connectorId);
    SystemTablesProvider systemTablesProvider;
    if (nodeManager.getCurrentNode().isCoordinator()) {
        systemTablesProvider = new DelegatingSystemTablesProvider(new StaticSystemTablesProvider(connector.getSystemTables()), new MetadataBasedSystemTablesProvider(metadataManager, catalogName));
    } else {
        systemTablesProvider = new StaticSystemTablesProvider(connector.getSystemTables());
    }
    MaterializedConnector systemConnector = new MaterializedConnector(systemId, new SystemConnector(systemId, nodeManager, systemTablesProvider, transactionId -> transactionManager.getConnectorTransaction(transactionId, connectorId), connector.getSessionProperties()));
    Catalog catalog = new Catalog(catalogName, connector.getConnectorId(), connector.getConnector(), informationSchemaConnector.getConnectorId(), informationSchemaConnector.getConnector(), systemConnector.getConnectorId(), systemConnector.getConnector());
    try {
        addConnectorInternal(connector);
        addConnectorInternal(informationSchemaConnector);
        addConnectorInternal(systemConnector);
        catalogManager.registerCatalog(catalog);
    } catch (Throwable e) {
        catalogManager.removeCatalog(catalog.getCatalogName());
        removeConnectorInternal(systemConnector.getConnectorId());
        removeConnectorInternal(informationSchemaConnector.getConnectorId());
        removeConnectorInternal(connector.getConnectorId());
        throw e;
    }
}
Also used : SystemConnector(com.facebook.presto.connector.system.SystemConnector) MetadataBasedSystemTablesProvider(com.facebook.presto.connector.system.MetadataBasedSystemTablesProvider) MetadataManager(com.facebook.presto.metadata.MetadataManager) ConnectorFilterStatsCalculatorService(com.facebook.presto.cost.ConnectorFilterStatsCalculatorService) RecordPageSourceProvider(com.facebook.presto.split.RecordPageSourceProvider) RowExpressionOptimizer(com.facebook.presto.sql.relational.RowExpressionOptimizer) ConnectorPlanOptimizerManager(com.facebook.presto.sql.planner.ConnectorPlanOptimizerManager) PropertyMetadata(com.facebook.presto.spi.session.PropertyMetadata) ConnectorNodePartitioningProvider(com.facebook.presto.spi.connector.ConnectorNodePartitioningProvider) PreDestroy(javax.annotation.PreDestroy) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) InternalNodeManager(com.facebook.presto.metadata.InternalNodeManager) ConnectorRecordSetProvider(com.facebook.presto.spi.connector.ConnectorRecordSetProvider) Map(java.util.Map) PageIndexerFactory(com.facebook.presto.spi.PageIndexerFactory) SystemTable(com.facebook.presto.spi.SystemTable) ConnectorMetadataUpdaterManager(com.facebook.presto.metadata.ConnectorMetadataUpdaterManager) ConnectorPageSourceProvider(com.facebook.presto.spi.connector.ConnectorPageSourceProvider) FilterStatsCalculator(com.facebook.presto.cost.FilterStatsCalculator) ConnectorSplitManager(com.facebook.presto.spi.connector.ConnectorSplitManager) ImmutableSet(com.google.common.collect.ImmutableSet) PageSinkManager(com.facebook.presto.split.PageSinkManager) ConnectorId.createInformationSchemaConnectorId(com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) DomainTranslator(com.facebook.presto.spi.relation.DomainTranslator) Set(java.util.Set) ThreadSafe(javax.annotation.concurrent.ThreadSafe) SplitManager(com.facebook.presto.split.SplitManager) GuardedBy(javax.annotation.concurrent.GuardedBy) String.format(java.lang.String.format) SystemTablesProvider(com.facebook.presto.connector.system.SystemTablesProvider) Preconditions.checkState(com.google.common.base.Preconditions.checkState) Procedure(com.facebook.presto.spi.procedure.Procedure) List(java.util.List) Catalog(com.facebook.presto.metadata.Catalog) Optional(java.util.Optional) SystemConnector(com.facebook.presto.connector.system.SystemConnector) PredicateCompiler(com.facebook.presto.spi.relation.PredicateCompiler) ConnectorId(com.facebook.presto.spi.ConnectorId) StaticSystemTablesProvider(com.facebook.presto.connector.system.StaticSystemTablesProvider) ConnectorRowExpressionService(com.facebook.presto.sql.relational.ConnectorRowExpressionService) DeterminismEvaluator(com.facebook.presto.spi.relation.DeterminismEvaluator) Logger(com.facebook.airlift.log.Logger) NodeInfo(com.facebook.airlift.node.NodeInfo) DelegatingSystemTablesProvider(com.facebook.presto.connector.system.DelegatingSystemTablesProvider) AccessControlManager(com.facebook.presto.security.AccessControlManager) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) BlockEncodingSerde(com.facebook.presto.common.block.BlockEncodingSerde) ConcurrentMap(java.util.concurrent.ConcurrentMap) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) ConnectorPlanOptimizerProvider(com.facebook.presto.spi.connector.ConnectorPlanOptimizerProvider) Inject(javax.inject.Inject) RowExpressionFormatter(com.facebook.presto.sql.planner.planPrinter.RowExpressionFormatter) ImmutableList(com.google.common.collect.ImmutableList) PageSourceManager(com.facebook.presto.split.PageSourceManager) TypeManager(com.facebook.presto.common.type.TypeManager) Objects.requireNonNull(java.util.Objects.requireNonNull) ConnectorId.createSystemTablesConnectorId(com.facebook.presto.spi.ConnectorId.createSystemTablesConnectorId) IndexManager(com.facebook.presto.index.IndexManager) ThreadContextClassLoader(com.facebook.presto.spi.classloader.ThreadContextClassLoader) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) PageSorter(com.facebook.presto.spi.PageSorter) ConnectorContext(com.facebook.presto.spi.connector.ConnectorContext) ConnectorPageSinkProvider(com.facebook.presto.spi.connector.ConnectorPageSinkProvider) TransactionManager(com.facebook.presto.transaction.TransactionManager) Connector(com.facebook.presto.spi.connector.Connector) ConnectorIndexProvider(com.facebook.presto.spi.connector.ConnectorIndexProvider) ConnectorFactory(com.facebook.presto.spi.connector.ConnectorFactory) CatalogManager(com.facebook.presto.metadata.CatalogManager) ConnectorMetadataUpdaterProvider(com.facebook.presto.spi.connector.ConnectorMetadataUpdaterProvider) HandleResolver(com.facebook.presto.metadata.HandleResolver) ConnectorAccessControl(com.facebook.presto.spi.connector.ConnectorAccessControl) PartitioningProviderManager(com.facebook.presto.sql.planner.PartitioningProviderManager) StaticSystemTablesProvider(com.facebook.presto.connector.system.StaticSystemTablesProvider) InformationSchemaConnector(com.facebook.presto.connector.informationSchema.InformationSchemaConnector) DelegatingSystemTablesProvider(com.facebook.presto.connector.system.DelegatingSystemTablesProvider) MetadataBasedSystemTablesProvider(com.facebook.presto.connector.system.MetadataBasedSystemTablesProvider) MetadataBasedSystemTablesProvider(com.facebook.presto.connector.system.MetadataBasedSystemTablesProvider) SystemTablesProvider(com.facebook.presto.connector.system.SystemTablesProvider) StaticSystemTablesProvider(com.facebook.presto.connector.system.StaticSystemTablesProvider) DelegatingSystemTablesProvider(com.facebook.presto.connector.system.DelegatingSystemTablesProvider) Catalog(com.facebook.presto.metadata.Catalog) ConnectorId.createInformationSchemaConnectorId(com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId) ConnectorId(com.facebook.presto.spi.ConnectorId) ConnectorId.createSystemTablesConnectorId(com.facebook.presto.spi.ConnectorId.createSystemTablesConnectorId)

Aggregations

Connector (com.facebook.presto.spi.connector.Connector)19 TestingConnectorContext (com.facebook.presto.testing.TestingConnectorContext)11 Test (org.testng.annotations.Test)9 InformationSchemaConnector (com.facebook.presto.connector.informationSchema.InformationSchemaConnector)8 SystemConnector (com.facebook.presto.connector.system.SystemConnector)8 ConnectorFactory (com.facebook.presto.spi.connector.ConnectorFactory)7 CatalogManager (com.facebook.presto.metadata.CatalogManager)6 ConnectorTransactionHandle (com.facebook.presto.spi.connector.ConnectorTransactionHandle)6 ConnectorMetadata (com.facebook.presto.spi.connector.ConnectorMetadata)5 Catalog (com.facebook.presto.metadata.Catalog)4 IsolationLevel (com.facebook.presto.spi.transaction.IsolationLevel)4 TpchConnectorFactory (com.facebook.presto.tpch.TpchConnectorFactory)4 InMemoryNodeManager (com.facebook.presto.metadata.InMemoryNodeManager)3 InternalNodeManager (com.facebook.presto.metadata.InternalNodeManager)3 MetadataManager (com.facebook.presto.metadata.MetadataManager)3 AccessControlManager (com.facebook.presto.security.AccessControlManager)3 NodeManager (com.facebook.presto.spi.NodeManager)3 Session (com.facebook.presto.Session)2 ConnectorId (com.facebook.presto.spi.ConnectorId)2 ConnectorId.createInformationSchemaConnectorId (com.facebook.presto.spi.ConnectorId.createInformationSchemaConnectorId)2