Search in sources :

Example 6 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class TaskActionTestKit method before.

@Override
public void before() {
    taskStorage = new HeapMemoryTaskStorage(new TaskStorageConfig(new Period("PT24H")));
    testDerbyConnector = new TestDerbyConnector(Suppliers.ofInstance(new MetadataStorageConnectorConfig()), Suppliers.ofInstance(metadataStorageTablesConfig));
    final ObjectMapper objectMapper = new TestUtils().getTestObjectMapper();
    metadataStorageCoordinator = new IndexerSQLMetadataStorageCoordinator(objectMapper, metadataStorageTablesConfig, testDerbyConnector) {

        @Override
        public int getSqlMetadataMaxRetry() {
            return 2;
        }
    };
    taskLockbox = new TaskLockbox(taskStorage, metadataStorageCoordinator);
    segmentsMetadataManager = new SqlSegmentsMetadataManager(objectMapper, Suppliers.ofInstance(new SegmentsMetadataManagerConfig()), Suppliers.ofInstance(metadataStorageTablesConfig), testDerbyConnector);
    taskActionToolbox = new TaskActionToolbox(taskLockbox, taskStorage, metadataStorageCoordinator, new NoopServiceEmitter(), EasyMock.createMock(SupervisorManager.class));
    testDerbyConnector.createDataSourceTable();
    testDerbyConnector.createPendingSegmentsTable();
    testDerbyConnector.createSegmentTable();
    testDerbyConnector.createRulesTable();
    testDerbyConnector.createConfigTable();
    testDerbyConnector.createTaskTables();
    testDerbyConnector.createAuditTable();
}
Also used : IndexerSQLMetadataStorageCoordinator(org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator) MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) TaskStorageConfig(org.apache.druid.indexing.common.config.TaskStorageConfig) HeapMemoryTaskStorage(org.apache.druid.indexing.overlord.HeapMemoryTaskStorage) Period(org.joda.time.Period) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) TestDerbyConnector(org.apache.druid.metadata.TestDerbyConnector) TestUtils(org.apache.druid.indexing.common.TestUtils) SegmentsMetadataManagerConfig(org.apache.druid.metadata.SegmentsMetadataManagerConfig) TaskLockbox(org.apache.druid.indexing.overlord.TaskLockbox) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SqlSegmentsMetadataManager(org.apache.druid.metadata.SqlSegmentsMetadataManager)

Example 7 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class MySQLFirehoseDatabaseConnectorTest method testSuccessOnlyValidPropertyMariaDb.

@Test
public void testSuccessOnlyValidPropertyMariaDb() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:mariadb://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("user", "password", "keyonly", "etc"));
    new MySQLFirehoseDatabaseConnector(connectorConfig, null, securityConfig, mySQLConnectorDriverConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Example 8 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class MySQLFirehoseDatabaseConnectorTest method testSuccessWhenAllowlistAndNoProperty.

@Test
public void testSuccessWhenAllowlistAndNoProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:mysql://localhost:3306/test";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("user"));
    new MySQLFirehoseDatabaseConnector(connectorConfig, null, securityConfig, mySQLConnectorDriverConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Example 9 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class MySQLFirehoseDatabaseConnectorTest method testFailValidAndInvalidProperty.

@Test
public void testFailValidAndInvalidProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:mysql://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("user", "nonenone"));
    expectedException.expectMessage("The property [password] is not in the allowed list");
    expectedException.expect(IllegalArgumentException.class);
    new MySQLFirehoseDatabaseConnector(connectorConfig, null, securityConfig, mySQLConnectorDriverConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Example 10 with MetadataStorageConnectorConfig

use of org.apache.druid.metadata.MetadataStorageConnectorConfig in project druid by druid-io.

the class MySQLFirehoseDatabaseConnectorTest method testFailOnlyInvalidProperty.

@Test
public void testFailOnlyInvalidProperty() {
    MetadataStorageConnectorConfig connectorConfig = new MetadataStorageConnectorConfig() {

        @Override
        public String getConnectURI() {
            return "jdbc:mysql://localhost:3306/test?user=maytas&password=secret&keyonly";
        }
    };
    JdbcAccessSecurityConfig securityConfig = newSecurityConfigEnforcingAllowList(ImmutableSet.of("none", "nonenone"));
    expectedException.expectMessage("The property [password] is not in the allowed list");
    expectedException.expect(IllegalArgumentException.class);
    new MySQLFirehoseDatabaseConnector(connectorConfig, null, securityConfig, mySQLConnectorDriverConfig);
}
Also used : MetadataStorageConnectorConfig(org.apache.druid.metadata.MetadataStorageConnectorConfig) JdbcAccessSecurityConfig(org.apache.druid.server.initialization.JdbcAccessSecurityConfig) Test(org.junit.Test)

Aggregations

MetadataStorageConnectorConfig (org.apache.druid.metadata.MetadataStorageConnectorConfig)26 Test (org.junit.Test)21 JdbcAccessSecurityConfig (org.apache.druid.server.initialization.JdbcAccessSecurityConfig)17 MetadataStorageTablesConfig (org.apache.druid.metadata.MetadataStorageTablesConfig)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 DruidProcessingModule (org.apache.druid.guice.DruidProcessingModule)2 QueryRunnerFactoryModule (org.apache.druid.guice.QueryRunnerFactoryModule)2 QueryableModule (org.apache.druid.guice.QueryableModule)2 Self (org.apache.druid.guice.annotations.Self)2 MetadataStorageUpdaterJobSpec (org.apache.druid.indexer.updater.MetadataStorageUpdaterJobSpec)2 IndexerSQLMetadataStorageCoordinator (org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator)2 DruidNode (org.apache.druid.server.DruidNode)2 InjectableValues (com.fasterxml.jackson.databind.InjectableValues)1 TypeLiteral (com.google.inject.TypeLiteral)1 TestUtils (org.apache.druid.indexing.common.TestUtils)1 TaskStorageConfig (org.apache.druid.indexing.common.config.TaskStorageConfig)1 HeapMemoryTaskStorage (org.apache.druid.indexing.overlord.HeapMemoryTaskStorage)1 TaskLockbox (org.apache.druid.indexing.overlord.TaskLockbox)1 DefaultObjectMapper (org.apache.druid.jackson.DefaultObjectMapper)1 SegmentsMetadataManagerConfig (org.apache.druid.metadata.SegmentsMetadataManagerConfig)1