Search in sources :

Example 11 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project hive by apache.

the class DruidStorageHandler method buildConnector.

private SQLMetadataConnector buildConnector() {
    if (connector != null) {
        return connector;
    }
    final String dbType = HiveConf.getVar(getConf(), HiveConf.ConfVars.DRUID_METADATA_DB_TYPE);
    final String username = HiveConf.getVar(getConf(), HiveConf.ConfVars.DRUID_METADATA_DB_USERNAME);
    final String password = getPassword(getConf(), HiveConf.ConfVars.DRUID_METADATA_DB_PASSWORD);
    final String uri = HiveConf.getVar(getConf(), HiveConf.ConfVars.DRUID_METADATA_DB_URI);
    LOG.debug("Supplying SQL Connector with DB type {}, URI {}, User {}", dbType, uri, username);
    @SuppressWarnings("Guava") final Supplier<MetadataStorageConnectorConfig> storageConnectorConfigSupplier = Suppliers.ofInstance(new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return uri;
        }

        @Override
        public String getUser() {
            return Strings.emptyToNull(username);
        }

        @Override
        public String getPassword() {
            return Strings.emptyToNull(password);
        }
    });
    switch(dbType) {
        case "mysql":
            connector = new MySQLConnector(storageConnectorConfigSupplier, Suppliers.ofInstance(getDruidMetadataStorageTablesConfig()), new MySQLConnectorConfig());
            break;
        case "postgresql":
            connector = new PostgreSQLConnector(storageConnectorConfigSupplier, Suppliers.ofInstance(getDruidMetadataStorageTablesConfig()), new PostgreSQLConnectorConfig(), new PostgreSQLTablesConfig());
            break;
        case "derby":
            connector = new DerbyConnector(new DerbyMetadataStorage(storageConnectorConfigSupplier.get()), storageConnectorConfigSupplier, Suppliers.ofInstance(getDruidMetadataStorageTablesConfig()));
            break;
        default:
            throw new IllegalStateException(String.format("Unknown metadata storage type [%s]", dbType));
    }
    return connector;
}
Also used : DerbyConnector(org.apache.druid.metadata.storage.derby.DerbyConnector) MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) PostgreSQLConnectorConfig(org.apache.druid.metadata.storage.postgresql.PostgreSQLConnectorConfig) PostgreSQLTablesConfig(org.apache.druid.metadata.storage.postgresql.PostgreSQLTablesConfig) PostgreSQLConnector(org.apache.druid.metadata.storage.postgresql.PostgreSQLConnector) MySQLConnectorConfig(org.apache.druid.metadata.storage.mysql.MySQLConnectorConfig) MySQLConnector(org.apache.druid.metadata.storage.mysql.MySQLConnector) DerbyMetadataStorage(org.apache.druid.metadata.storage.derby.DerbyMetadataStorage)

Example 12 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class PostgresqlFirehoseDatabaseConnectorTest method testFailWhenNoAllowlistAndHaveProperty.

@Test
public void testFailWhenNoAllowlistAndHaveProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:postgresql://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of(""));
    expectedException.expectMessage("is not in the allowed list");
    expectedException.expect(IllegalArgumentException.class);
    new PostgresqlFirehoseDatabaseConnector(connectorConfig, securityConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Example 13 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class PostgresqlFirehoseDatabaseConnectorTest method testSerde.

@Test
public void testSerde() throws JsonProcessingException {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:postgresql://localhost:3306/test";
        }
    };
    PostgresqlFirehoseDatabaseConnector connector = new PostgresqlFirehoseDatabaseConnector(connectorConfig, INJECTED_CONF);
    PostgresqlFirehoseDatabaseConnector andBack = MAPPER.readValue(MAPPER.writeValueAsString(connector), PostgresqlFirehoseDatabaseConnector.class);
    Assert.assertEquals(connector, andBack);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) Test(org.junit.Test)

Example 14 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class PostgresqlFirehoseDatabaseConnectorTest method testSuccessOnlyValidProperty.

@Test
public void testSuccessOnlyValidProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:postgresql://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("user", "password", "keyonly", "etc"));
    new PostgresqlFirehoseDatabaseConnector(connectorConfig, securityConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Example 15 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class PostgresqlFirehoseDatabaseConnectorTest method testFailOnlyInvalidProperty.

@Test
public void testFailOnlyInvalidProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:postgresql://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("none", "nonenone"));
    expectedException.expectMessage("is not in the allowed list");
    expectedException.expect(IllegalArgumentException.class);
    new PostgresqlFirehoseDatabaseConnector(connectorConfig, securityConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Aggregations

MetadataStorageConnectorConfig (org.apache.druid.metadata.MetadataStorageConnectorConfig)26 Test (org.junit.Test)21 JdbcAccessSecurityConfig (org.apache.druid.server.initialization.JdbcAccessSecurityConfig)17 MetadataStorageTablesConfig (org.apache.druid.metadata.MetadataStorageTablesConfig)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 DruidProcessingModule (org.apache.druid.guice.DruidProcessingModule)2 QueryRunnerFactoryModule (org.apache.druid.guice.QueryRunnerFactoryModule)2 QueryableModule (org.apache.druid.guice.QueryableModule)2 Self (org.apache.druid.guice.annotations.Self)2 MetadataStorageUpdaterJobSpec (org.apache.druid.indexer.updater.MetadataStorageUpdaterJobSpec)2 IndexerSQLMetadataStorageCoordinator (org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator)2 DruidNode (org.apache.druid.server.DruidNode)2 InjectableValues (com.fasterxml.jackson.databind.InjectableValues)1 TypeLiteral (com.google.inject.TypeLiteral)1 TestUtils (org.apache.druid.indexing.common.TestUtils)1 TaskStorageConfig (org.apache.druid.indexing.common.config.TaskStorageConfig)1 HeapMemoryTaskStorage (org.apache.druid.indexing.overlord.HeapMemoryTaskStorage)1 TaskLockbox (org.apache.druid.indexing.overlord.TaskLockbox)1 DefaultObjectMapper (org.apache.druid.jackson.DefaultObjectMapper)1 SegmentsMetadataManagerConfig (org.apache.druid.metadata.SegmentsMetadataManagerConfig)1