Search in sources :

Example 1 with MetacatCatalogConfig

use of com.netflix.metacat.main.spi.MetacatCatalogConfig in project metacat by Netflix.

the class ConnectorManager method createConnection.

/**
     * Creates a connection for the given catalog.
     *
     * @param catalogName   catalog name
     * @param connectorType connector type
     * @param properties    properties
     */
synchronized void createConnection(final String catalogName, final String connectorType, final Map<String, String> properties, final Registry registry) {
    Preconditions.checkState(!stopped.get(), "ConnectorManager is stopped");
    Preconditions.checkNotNull(catalogName, "catalogName is null");
    Preconditions.checkNotNull(connectorType, "connectorName is null");
    Preconditions.checkNotNull(properties, "properties is null");
    final ConnectorPlugin connectorPlugin = plugins.get(connectorType);
    if (connectorPlugin != null) {
        Preconditions.checkState(!connectorFactories.containsKey(catalogName), "A connector %s already exists", catalogName);
        final ConnectorFactory connectorFactory = connectorPlugin.create(this.config, catalogName, properties, registry);
        connectorFactories.put(catalogName, connectorFactory);
        final MetacatCatalogConfig catalogConfig = MetacatCatalogConfig.createFromMapAndRemoveProperties(connectorType, properties);
        catalogs.put(catalogName, catalogConfig);
    } else {
        log.warn("No plugin for connector with type %s", connectorType);
    }
}
Also used : ConnectorPlugin(com.netflix.metacat.common.server.connectors.ConnectorPlugin) ConnectorFactory(com.netflix.metacat.common.server.connectors.ConnectorFactory) MetacatCatalogConfig(com.netflix.metacat.main.spi.MetacatCatalogConfig)

Example 2 with MetacatCatalogConfig

use of com.netflix.metacat.main.spi.MetacatCatalogConfig in project metacat by Netflix.

the class CatalogServiceImpl method get.

/**
     * {@inheritDoc}
     */
@Nonnull
@Override
public CatalogDto get(@Nonnull final QualifiedName name) {
    final MetacatCatalogConfig config = connectorManager.getCatalogConfig(name);
    final CatalogDto result = new CatalogDto();
    result.setName(name);
    result.setType(config.getType());
    final ConnectorContext context = converterUtil.toConnectorContext(MetacatContextManager.getContext());
    result.setDatabases(connectorManager.getDatabaseService(name.getCatalogName()).listNames(context, name, null, null, null).stream().map(QualifiedName::getDatabaseName).filter(s -> config.getSchemaBlacklist().isEmpty() || !config.getSchemaBlacklist().contains(s)).filter(s -> config.getSchemaWhitelist().isEmpty() || config.getSchemaWhitelist().contains(s)).sorted(String.CASE_INSENSITIVE_ORDER).collect(Collectors.toList()));
    userMetadataService.populateMetadata(result);
    return result;
}
Also used : UserMetadataService(com.netflix.metacat.common.server.usermetadata.UserMetadataService) CatalogService(com.netflix.metacat.main.services.CatalogService) CreateCatalogDto(com.netflix.metacat.common.dto.CreateCatalogDto) MetacatEventBus(com.netflix.metacat.common.server.events.MetacatEventBus) ConverterUtil(com.netflix.metacat.common.server.converter.ConverterUtil) MetacatContextManager(com.netflix.metacat.common.server.util.MetacatContextManager) MetacatUpdateDatabasePreEvent(com.netflix.metacat.common.server.events.MetacatUpdateDatabasePreEvent) CatalogMappingDto(com.netflix.metacat.common.dto.CatalogMappingDto) QualifiedName(com.netflix.metacat.common.QualifiedName) MetacatUpdateDatabasePostEvent(com.netflix.metacat.common.server.events.MetacatUpdateDatabasePostEvent) Collectors(java.util.stream.Collectors) MetacatNotFoundException(com.netflix.metacat.common.exception.MetacatNotFoundException) CatalogDto(com.netflix.metacat.common.dto.CatalogDto) MetacatCatalogConfig(com.netflix.metacat.main.spi.MetacatCatalogConfig) List(java.util.List) Map(java.util.Map) MetacatRequestContext(com.netflix.metacat.common.MetacatRequestContext) ConnectorContext(com.netflix.metacat.common.server.connectors.ConnectorContext) ConnectorManager(com.netflix.metacat.main.manager.ConnectorManager) Nonnull(javax.annotation.Nonnull) CreateCatalogDto(com.netflix.metacat.common.dto.CreateCatalogDto) CatalogDto(com.netflix.metacat.common.dto.CatalogDto) MetacatCatalogConfig(com.netflix.metacat.main.spi.MetacatCatalogConfig) ConnectorContext(com.netflix.metacat.common.server.connectors.ConnectorContext) Nonnull(javax.annotation.Nonnull)

Example 3 with MetacatCatalogConfig

use of com.netflix.metacat.main.spi.MetacatCatalogConfig in project metacat by Netflix.

the class DatabaseServiceImpl method get.

@Override
public DatabaseDto get(@Nonnull final QualifiedName name, final boolean includeUserMetadata) {
    validate(name);
    final MetacatRequestContext metacatRequestContext = MetacatContextManager.getContext();
    final MetacatCatalogConfig config = connectorManager.getCatalogConfig(name.getCatalogName());
    final ConnectorDatabaseService service = connectorManager.getDatabaseService(name.getCatalogName());
    final ConnectorTableService tableService = connectorManager.getTableService(name.getCatalogName());
    final ConnectorContext connectorContext = converterUtil.toConnectorContext(metacatRequestContext);
    final List<QualifiedName> tableNames = tableService.listNames(connectorContext, name, null, null, null);
    List<QualifiedName> viewNames = Collections.emptyList();
    if (config.isIncludeViewsWithTables()) {
        // TODO JdbcMetadata returns ImmutableList.of() for views.  We should change it to fetch views.
        try {
            viewNames = service.listViewNames(connectorContext, name);
        } catch (UnsupportedOperationException ignored) {
        }
    }
    // Check to see if schema exists
    if (tableNames.isEmpty() && viewNames.isEmpty() && !exists(name)) {
        throw new DatabaseNotFoundException(name);
    }
    final DatabaseDto dto = converterUtil.toDatabaseDto(service.get(connectorContext, name));
    dto.setType(connectorManager.getCatalogConfig(name).getType());
    dto.setTables(Stream.concat(tableNames.stream(), viewNames.stream()).map(QualifiedName::getTableName).sorted(String.CASE_INSENSITIVE_ORDER).collect(Collectors.toList()));
    if (includeUserMetadata) {
        log.info("Populate user metadata for schema {}", name);
        userMetadataService.populateMetadata(dto);
    }
    return dto;
}
Also used : MetacatRequestContext(com.netflix.metacat.common.MetacatRequestContext) ConnectorDatabaseService(com.netflix.metacat.common.server.connectors.ConnectorDatabaseService) ConnectorTableService(com.netflix.metacat.common.server.connectors.ConnectorTableService) MetacatCatalogConfig(com.netflix.metacat.main.spi.MetacatCatalogConfig) QualifiedName(com.netflix.metacat.common.QualifiedName) DatabaseNotFoundException(com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException) DatabaseDto(com.netflix.metacat.common.dto.DatabaseDto) ConnectorContext(com.netflix.metacat.common.server.connectors.ConnectorContext)

Aggregations

MetacatCatalogConfig (com.netflix.metacat.main.spi.MetacatCatalogConfig)3 MetacatRequestContext (com.netflix.metacat.common.MetacatRequestContext)2 QualifiedName (com.netflix.metacat.common.QualifiedName)2 ConnectorContext (com.netflix.metacat.common.server.connectors.ConnectorContext)2 CatalogDto (com.netflix.metacat.common.dto.CatalogDto)1 CatalogMappingDto (com.netflix.metacat.common.dto.CatalogMappingDto)1 CreateCatalogDto (com.netflix.metacat.common.dto.CreateCatalogDto)1 DatabaseDto (com.netflix.metacat.common.dto.DatabaseDto)1 MetacatNotFoundException (com.netflix.metacat.common.exception.MetacatNotFoundException)1 ConnectorDatabaseService (com.netflix.metacat.common.server.connectors.ConnectorDatabaseService)1 ConnectorFactory (com.netflix.metacat.common.server.connectors.ConnectorFactory)1 ConnectorPlugin (com.netflix.metacat.common.server.connectors.ConnectorPlugin)1 ConnectorTableService (com.netflix.metacat.common.server.connectors.ConnectorTableService)1 DatabaseNotFoundException (com.netflix.metacat.common.server.connectors.exception.DatabaseNotFoundException)1 ConverterUtil (com.netflix.metacat.common.server.converter.ConverterUtil)1 MetacatEventBus (com.netflix.metacat.common.server.events.MetacatEventBus)1 MetacatUpdateDatabasePostEvent (com.netflix.metacat.common.server.events.MetacatUpdateDatabasePostEvent)1 MetacatUpdateDatabasePreEvent (com.netflix.metacat.common.server.events.MetacatUpdateDatabasePreEvent)1 UserMetadataService (com.netflix.metacat.common.server.usermetadata.UserMetadataService)1 MetacatContextManager (com.netflix.metacat.common.server.util.MetacatContextManager)1