Search in sources :

Example 1 with UsernamePasswordCredentials

use of org.apache.drill.exec.store.security.UsernamePasswordCredentials in project drill by apache.

the class JdbcStoragePlugin method initDataSource.

/**
 * Initializes {@link HikariDataSource} instance and configures it based on given
 * storage plugin configuration.
 * Basic parameters such as driver, url, user name and password are set using setters.
 * Other source parameters are set dynamically through the properties. See the list
 * of available Hikari properties: <a href="https://github.com/brettwooldridge/HikariCP">.
 *
 * @param config storage plugin config
 * @return Hikari data source instance
 * @throws UserException if unable to configure Hikari data source
 */
@VisibleForTesting
static HikariDataSource initDataSource(JdbcStorageConfig config) {
    try {
        Properties properties = new Properties();
        /*
        Set default HikariCP values which prefer to connect lazily to avoid overwhelming source
      systems with connections which mostly remain idle.  A data source that is present in N
      storage configs replicated over P drillbits with a HikariCP minimumIdle value of Q will
      have N×P×Q connections made to it eagerly.
        The trade off of lazier connections is increased latency should there be a spike in user
      queries involving a JDBC data source.  When comparing the defaults that follow with e.g. the
      HikariCP defaults, bear in mind that the context here is OLAP, not OLTP.  It is normal
      for queries to run for a long time and to be separated by long intermissions. Users who
      prefer eager to lazy connections remain free to overwrite the following defaults in their
      storage config.
      */
        // maximum amount of time that a connection is allowed to sit idle in the pool, 0 = forever
        // 1 hour
        properties.setProperty("dataSource.idleTimeout", String.format("%d000", 1 * 60 * 60));
        // how frequently HikariCP will attempt to keep a connection alive, 0 = disabled
        properties.setProperty("dataSource.keepaliveTime", String.format("%d000", 0));
        // maximum lifetime of a connection in the pool, 0 = forever
        // 6 hours
        properties.setProperty("dataSource.maxLifetime", String.format("%d000", 6 * 60 * 60));
        // minimum number of idle connections that HikariCP tries to maintain in the pool, 0 = none
        properties.setProperty("dataSource.minimumIdle", "0");
        // maximum size that the pool is allowed to reach, including both idle and in-use connections
        properties.setProperty("dataSource.maximumPoolSize", "10");
        // apply any HikariCP parameters the user may have set, overwriting defaults
        properties.putAll(config.getSourceParameters());
        HikariConfig hikariConfig = new HikariConfig(properties);
        hikariConfig.setDriverClassName(config.getDriver());
        hikariConfig.setJdbcUrl(config.getUrl());
        UsernamePasswordCredentials credentials = config.getUsernamePasswordCredentials();
        hikariConfig.setUsername(credentials.getUsername());
        hikariConfig.setPassword(credentials.getPassword());
        // this serves as a hint to the driver, which *might* enable database optimizations
        hikariConfig.setReadOnly(!config.isWritable());
        return new HikariDataSource(hikariConfig);
    } catch (RuntimeException e) {
        throw UserException.connectionError(e).message("Unable to configure data source: %s", e.getMessage()).build(logger);
    }
}
Also used : HikariDataSource(com.zaxxer.hikari.HikariDataSource) Properties(java.util.Properties) HikariConfig(com.zaxxer.hikari.HikariConfig) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials) VisibleForTesting(org.apache.drill.shaded.guava.com.google.common.annotations.VisibleForTesting)

Example 2 with UsernamePasswordCredentials

use of org.apache.drill.exec.store.security.UsernamePasswordCredentials in project drill by apache.

the class SimpleHttp method setupHttpClient.

/**
 * Configures the OkHTTP3 server object with configuration info from the user.
 *
 * @return OkHttpClient configured server
 */
private OkHttpClient setupHttpClient() {
    Builder builder = new OkHttpClient.Builder();
    // Set up the HTTP Cache.   Future possibilities include making the cache size and retention configurable but
    // right now it is on or off.  The writer will write to the Drill temp directory if it is accessible and
    // output a warning if not.
    HttpStoragePluginConfig config = scanDefn.tableSpec().config();
    if (config.cacheResults()) {
        setupCache(builder);
    }
    HttpApiConfig apiConfig = scanDefn.tableSpec().connectionConfig();
    // If OAuth information is provided, we will assume that the user does not want to use
    // basic authentication
    HttpOAuthConfig oAuthConfig = scanDefn.tableSpec().config().oAuthConfig();
    if (oAuthConfig != null) {
        // Add interceptors for OAuth2
        logger.debug("Adding OAuth2 Interceptor");
        AccessTokenRepository repository = new AccessTokenRepository(proxyConfig, config, tokenTable);
        builder.authenticator(new AccessTokenAuthenticator(repository));
        builder.addInterceptor(new AccessTokenInterceptor(repository));
    } else if (apiConfig.authType().equalsIgnoreCase("basic")) {
        // If the API uses basic authentication add the authentication code.
        logger.debug("Adding Interceptor");
        UsernamePasswordCredentials credentials = apiConfig.getUsernamePasswordCredentials();
        builder.addInterceptor(new BasicAuthInterceptor(credentials.getUsername(), credentials.getPassword()));
    }
    // Set timeouts
    int timeout = Math.max(1, config.timeout());
    builder.connectTimeout(timeout, TimeUnit.SECONDS);
    builder.writeTimeout(timeout, TimeUnit.SECONDS);
    builder.readTimeout(timeout, TimeUnit.SECONDS);
    // Sourced from https://stackoverflow.com/questions/60110848/how-to-disable-ssl-verification
    if (!apiConfig.verifySSLCert()) {
        try {
            TrustManager[] trustAllCerts = getAllTrustingTrustManager();
            SSLContext sslContext = SSLContext.getInstance("SSL");
            sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
            SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
            builder.sslSocketFactory(sslSocketFactory, (X509TrustManager) trustAllCerts[0]);
            HostnameVerifier verifier = (hostname, session) -> true;
            builder.hostnameVerifier(verifier);
        } catch (KeyManagementException | NoSuchAlgorithmException e) {
            logger.error("Error when configuring Drill not to verify SSL certs. {}", e.getMessage());
        }
    }
    // Set the proxy configuration
    addProxyInfo(builder, proxyConfig);
    return builder.build();
}
Also used : HttpSubScan(org.apache.drill.exec.store.http.HttpSubScan) X509Certificate(java.security.cert.X509Certificate) SSLContext(javax.net.ssl.SSLContext) Cache(okhttp3.Cache) URLDecoder(java.net.URLDecoder) UserException(org.apache.drill.common.exceptions.UserException) LoggerFactory(org.slf4j.LoggerFactory) TrustManager(javax.net.ssl.TrustManager) HttpOAuthConfig(org.apache.drill.exec.store.http.HttpOAuthConfig) StringUtils(org.apache.commons.lang3.StringUtils) FormBody(okhttp3.FormBody) PersistentTokenTable(org.apache.drill.exec.oauth.PersistentTokenTable) Matcher(java.util.regex.Matcher) Proxy(java.net.Proxy) Map(java.util.Map) HostnameVerifier(javax.net.ssl.HostnameVerifier) Interceptor(okhttp3.Interceptor) Request(okhttp3.Request) HttpMethod(org.apache.drill.exec.store.http.HttpApiConfig.HttpMethod) HttpApiConfig(org.apache.drill.exec.store.http.HttpApiConfig) KeyManagementException(java.security.KeyManagementException) Credentials(okhttp3.Credentials) InetSocketAddress(java.net.InetSocketAddress) SSLSocketFactory(javax.net.ssl.SSLSocketFactory) AccessTokenAuthenticator(org.apache.drill.exec.store.http.oauth.AccessTokenAuthenticator) Objects(java.util.Objects) List(java.util.List) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) AccessTokenInterceptor(org.apache.drill.exec.store.http.oauth.AccessTokenInterceptor) Pattern(java.util.regex.Pattern) HttpUrl(okhttp3.HttpUrl) NotNull(org.jetbrains.annotations.NotNull) UnsupportedEncodingException(java.io.UnsupportedEncodingException) Builder(okhttp3.OkHttpClient.Builder) StoragePluginRegistry(org.apache.drill.exec.store.StoragePluginRegistry) CustomErrorContext(org.apache.drill.common.exceptions.CustomErrorContext) ArrayList(java.util.ArrayList) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials) AccessTokenRepository(org.apache.drill.exec.store.http.oauth.AccessTokenRepository) Response(okhttp3.Response) Logger(org.slf4j.Logger) IOException(java.io.IOException) CaseInsensitiveMap(org.apache.drill.common.map.CaseInsensitiveMap) Paginator(org.apache.drill.exec.store.http.paginator.Paginator) File(java.io.File) TimeUnit(java.util.concurrent.TimeUnit) OkHttpClient(okhttp3.OkHttpClient) X509TrustManager(javax.net.ssl.X509TrustManager) HttpStoragePluginConfig(org.apache.drill.exec.store.http.HttpStoragePluginConfig) InputStream(java.io.InputStream) AccessTokenInterceptor(org.apache.drill.exec.store.http.oauth.AccessTokenInterceptor) HttpApiConfig(org.apache.drill.exec.store.http.HttpApiConfig) Builder(okhttp3.OkHttpClient.Builder) AccessTokenRepository(org.apache.drill.exec.store.http.oauth.AccessTokenRepository) SSLContext(javax.net.ssl.SSLContext) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) KeyManagementException(java.security.KeyManagementException) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials) TrustManager(javax.net.ssl.TrustManager) X509TrustManager(javax.net.ssl.X509TrustManager) HostnameVerifier(javax.net.ssl.HostnameVerifier) HttpOAuthConfig(org.apache.drill.exec.store.http.HttpOAuthConfig) AccessTokenAuthenticator(org.apache.drill.exec.store.http.oauth.AccessTokenAuthenticator) HttpStoragePluginConfig(org.apache.drill.exec.store.http.HttpStoragePluginConfig) SSLSocketFactory(javax.net.ssl.SSLSocketFactory)

Example 3 with UsernamePasswordCredentials

use of org.apache.drill.exec.store.security.UsernamePasswordCredentials in project drill by apache.

the class CassandraStorageConfig method toConfigMap.

@JsonIgnore
public Map<String, Object> toConfigMap() {
    UsernamePasswordCredentials credentials = getUsernamePasswordCredentials();
    Map<String, Object> result = new HashMap<>();
    result.put("host", host);
    result.put("port", port);
    result.put("username", credentials.getUsername());
    result.put("password", credentials.getPassword());
    return result;
}
Also used : HashMap(java.util.HashMap) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials) JsonIgnore(com.fasterxml.jackson.annotation.JsonIgnore)

Example 4 with UsernamePasswordCredentials

use of org.apache.drill.exec.store.security.UsernamePasswordCredentials in project drill by apache.

the class HttpBatchReader method proxySettings.

protected HttpProxyConfig proxySettings(Config drillConfig, HttpUrl url) {
    final HttpStoragePluginConfig config = subScan.tableSpec().config();
    final ProxyBuilder builder = HttpProxyConfig.builder().fromConfigForURL(drillConfig, url.toString());
    final String proxyType = config.proxyType();
    if (proxyType != null && !"direct".equals(proxyType)) {
        UsernamePasswordCredentials credentials = config.getUsernamePasswordCredentials();
        builder.type(config.proxyType()).host(config.proxyHost()).port(config.proxyPort()).username(credentials.getUsername()).password(credentials.getPassword());
    }
    return builder.build();
}
Also used : ProxyBuilder(org.apache.drill.exec.store.http.util.HttpProxyConfig.ProxyBuilder) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials)

Example 5 with UsernamePasswordCredentials

use of org.apache.drill.exec.store.security.UsernamePasswordCredentials in project drill by apache.

the class MongoStoragePlugin method addCredentialsFromCredentialsProvider.

private String addCredentialsFromCredentialsProvider(String connection, String name) {
    ConnectionString parsed = new ConnectionString(connection);
    if (parsed.getCredential() == null) {
        UsernamePasswordCredentials credentials = getUsernamePasswordCredentials(name);
        try {
            // each will need their own credentials.
            if (credentials.getUsername() != null && credentials.getPassword() != null) {
                String username = URLEncoder.encode(credentials.getUsername(), "UTF-8");
                String password = URLEncoder.encode(credentials.getPassword(), "UTF-8");
                return connection.replaceFirst("://", String.format("://%s:%s@", username, password));
            }
        } catch (IOException e) {
            logger.error("Error fetching mongodb username and password from configuration", e);
        }
    }
    return connection;
}
Also used : ConnectionString(com.mongodb.ConnectionString) IOException(java.io.IOException) ConnectionString(com.mongodb.ConnectionString) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials)

Aggregations

UsernamePasswordCredentials (org.apache.drill.exec.store.security.UsernamePasswordCredentials)5 IOException (java.io.IOException)2 JsonIgnore (com.fasterxml.jackson.annotation.JsonIgnore)1 ConnectionString (com.mongodb.ConnectionString)1 HikariConfig (com.zaxxer.hikari.HikariConfig)1 HikariDataSource (com.zaxxer.hikari.HikariDataSource)1 File (java.io.File)1 InputStream (java.io.InputStream)1 UnsupportedEncodingException (java.io.UnsupportedEncodingException)1 InetSocketAddress (java.net.InetSocketAddress)1 Proxy (java.net.Proxy)1 URLDecoder (java.net.URLDecoder)1 KeyManagementException (java.security.KeyManagementException)1 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)1 X509Certificate (java.security.cert.X509Certificate)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1 Objects (java.util.Objects)1