Search in sources :

Example 1 with ExtensionsConfig

use of org.apache.druid.guice.ExtensionsConfig in project druid by druid-io.

the class Initialization method makeInjectorWithModules.

public static Injector makeInjectorWithModules(final Set<NodeRole> nodeRoles, final Injector baseInjector, final Iterable<? extends Module> modules) {
    final ModuleList defaultModules = new ModuleList(baseInjector, nodeRoles);
    defaultModules.addModules(// New modules should be added after Log4jShutterDownerModule
    new Log4jShutterDownerModule(), new DruidAuthModule(), new LifecycleModule(), TLSCertificateCheckerModule.class, EmitterModule.class, HttpClientModule.global(), HttpClientModule.escalatedGlobal(), new HttpClientModule("druid.broker.http", Client.class), new HttpClientModule("druid.broker.http", EscalatedClient.class), new CuratorModule(), new AnnouncerModule(), new MetricsModule(), new SegmentWriteOutMediumModule(), new ServerModule(), new DruidProcessingConfigModule(), new StorageNodeModule(), new JettyServerModule(), new ExpressionModule(), new DiscoveryModule(), new ServerViewModule(), new MetadataConfigModule(), new DerbyMetadataStorageDruidModule(), new JacksonConfigManagerModule(), new IndexingServiceDiscoveryModule(), new CoordinatorDiscoveryModule(), new LocalDataStorageDruidModule(), new FirehoseModule(), new JavaScriptModule(), new AuthenticatorModule(), new AuthenticatorMapperModule(), new EscalatorModule(), new AuthorizerModule(), new AuthorizerMapperModule(), new StartupLoggingModule(), new ExternalStorageAccessSecurityModule());
    ModuleList actualModules = new ModuleList(baseInjector, nodeRoles);
    actualModules.addModule(DruidSecondaryModule.class);
    for (Object module : modules) {
        actualModules.addModule(module);
    }
    Module intermediateModules = Modules.override(defaultModules.getModules()).with(actualModules.getModules());
    ModuleList extensionModules = new ModuleList(baseInjector, nodeRoles);
    final ExtensionsConfig config = baseInjector.getInstance(ExtensionsConfig.class);
    for (DruidModule module : Initialization.getFromExtensions(config, DruidModule.class)) {
        extensionModules.addModule(module);
    }
    return Guice.createInjector(Modules.override(intermediateModules).with(extensionModules.getModules()));
}
Also used : CuratorModule(org.apache.druid.curator.CuratorModule) IndexingServiceDiscoveryModule(org.apache.druid.guice.IndexingServiceDiscoveryModule) DiscoveryModule(org.apache.druid.curator.discovery.DiscoveryModule) CoordinatorDiscoveryModule(org.apache.druid.guice.CoordinatorDiscoveryModule) ServerViewModule(org.apache.druid.guice.ServerViewModule) EscalatorModule(org.apache.druid.guice.security.EscalatorModule) JettyServerModule(org.apache.druid.server.initialization.jetty.JettyServerModule) JettyServerModule(org.apache.druid.server.initialization.jetty.JettyServerModule) ServerModule(org.apache.druid.guice.ServerModule) AuthenticatorMapperModule(org.apache.druid.server.initialization.AuthenticatorMapperModule) FirehoseModule(org.apache.druid.guice.FirehoseModule) LocalDataStorageDruidModule(org.apache.druid.guice.LocalDataStorageDruidModule) DerbyMetadataStorageDruidModule(org.apache.druid.metadata.storage.derby.DerbyMetadataStorageDruidModule) Client(org.apache.druid.guice.annotations.Client) EscalatedClient(org.apache.druid.guice.annotations.EscalatedClient) ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) JacksonConfigManagerModule(org.apache.druid.guice.JacksonConfigManagerModule) LifecycleModule(org.apache.druid.guice.LifecycleModule) ExternalStorageAccessSecurityModule(org.apache.druid.server.initialization.ExternalStorageAccessSecurityModule) AuthorizerModule(org.apache.druid.guice.security.AuthorizerModule) IndexingServiceDiscoveryModule(org.apache.druid.guice.IndexingServiceDiscoveryModule) ExpressionModule(org.apache.druid.guice.ExpressionModule) LocalDataStorageDruidModule(org.apache.druid.guice.LocalDataStorageDruidModule) StartupLoggingModule(org.apache.druid.guice.StartupLoggingModule) SegmentWriteOutMediumModule(org.apache.druid.segment.writeout.SegmentWriteOutMediumModule) DerbyMetadataStorageDruidModule(org.apache.druid.metadata.storage.derby.DerbyMetadataStorageDruidModule) DruidProcessingConfigModule(org.apache.druid.guice.DruidProcessingConfigModule) EscalatedClient(org.apache.druid.guice.annotations.EscalatedClient) MetadataConfigModule(org.apache.druid.guice.MetadataConfigModule) CoordinatorDiscoveryModule(org.apache.druid.guice.CoordinatorDiscoveryModule) AuthorizerMapperModule(org.apache.druid.server.initialization.AuthorizerMapperModule) DruidAuthModule(org.apache.druid.guice.security.DruidAuthModule) MetricsModule(org.apache.druid.server.metrics.MetricsModule) AnnouncerModule(org.apache.druid.guice.AnnouncerModule) StorageNodeModule(org.apache.druid.guice.StorageNodeModule) AuthenticatorModule(org.apache.druid.guice.security.AuthenticatorModule) Module(com.google.inject.Module) ExpressionModule(org.apache.druid.guice.ExpressionModule) DruidAuthModule(org.apache.druid.guice.security.DruidAuthModule) LifecycleModule(org.apache.druid.guice.LifecycleModule) StartupLoggingModule(org.apache.druid.guice.StartupLoggingModule) JacksonConfigManagerModule(org.apache.druid.guice.JacksonConfigManagerModule) AuthorizerMapperModule(org.apache.druid.server.initialization.AuthorizerMapperModule) AuthenticatorMapperModule(org.apache.druid.server.initialization.AuthenticatorMapperModule) TLSCertificateCheckerModule(org.apache.druid.server.security.TLSCertificateCheckerModule) IndexingServiceDiscoveryModule(org.apache.druid.guice.IndexingServiceDiscoveryModule) AuthenticatorModule(org.apache.druid.guice.security.AuthenticatorModule) HttpClientModule(org.apache.druid.guice.http.HttpClientModule) LocalDataStorageDruidModule(org.apache.druid.guice.LocalDataStorageDruidModule) DiscoveryModule(org.apache.druid.curator.discovery.DiscoveryModule) JettyServerModule(org.apache.druid.server.initialization.jetty.JettyServerModule) MetricsModule(org.apache.druid.server.metrics.MetricsModule) SegmentWriteOutMediumModule(org.apache.druid.segment.writeout.SegmentWriteOutMediumModule) FirehoseModule(org.apache.druid.guice.FirehoseModule) DerbyMetadataStorageDruidModule(org.apache.druid.metadata.storage.derby.DerbyMetadataStorageDruidModule) StorageNodeModule(org.apache.druid.guice.StorageNodeModule) MetadataConfigModule(org.apache.druid.guice.MetadataConfigModule) JavaScriptModule(org.apache.druid.guice.JavaScriptModule) ServerViewModule(org.apache.druid.guice.ServerViewModule) CoordinatorDiscoveryModule(org.apache.druid.guice.CoordinatorDiscoveryModule) CuratorModule(org.apache.druid.curator.CuratorModule) AnnouncerModule(org.apache.druid.guice.AnnouncerModule) DruidSecondaryModule(org.apache.druid.guice.DruidSecondaryModule) EscalatorModule(org.apache.druid.guice.security.EscalatorModule) ExternalStorageAccessSecurityModule(org.apache.druid.server.initialization.ExternalStorageAccessSecurityModule) DruidProcessingConfigModule(org.apache.druid.guice.DruidProcessingConfigModule) ServerModule(org.apache.druid.guice.ServerModule) EmitterModule(org.apache.druid.server.emitter.EmitterModule) AuthorizerModule(org.apache.druid.guice.security.AuthorizerModule) HttpClientModule(org.apache.druid.guice.http.HttpClientModule) JavaScriptModule(org.apache.druid.guice.JavaScriptModule)

Example 2 with ExtensionsConfig

use of org.apache.druid.guice.ExtensionsConfig in project druid by druid-io.

the class Main method main.

@SuppressWarnings("unchecked")
@SuppressForbidden(reason = "System#out")
public static void main(String[] args) {
    final CliBuilder<Runnable> builder = Cli.builder("druid");
    builder.withDescription("Druid command-line runner.").withDefaultCommand(Help.class).withCommands(Help.class, Version.class);
    List<Class<? extends Runnable>> serverCommands = Arrays.asList(CliCoordinator.class, CliHistorical.class, CliBroker.class, CliOverlord.class, CliIndexer.class, CliMiddleManager.class, CliRouter.class);
    builder.withGroup("server").withDescription("Run one of the Druid server types.").withDefaultCommand(Help.class).withCommands(serverCommands);
    List<Class<? extends Runnable>> toolCommands = Arrays.asList(DruidJsonValidator.class, PullDependencies.class, CreateTables.class, DumpSegment.class, ResetCluster.class, ValidateSegments.class, ExportMetadata.class);
    builder.withGroup("tools").withDescription("Various tools for working with Druid").withDefaultCommand(Help.class).withCommands(toolCommands);
    builder.withGroup("index").withDescription("Run indexing for druid").withDefaultCommand(Help.class).withCommands(CliHadoopIndexer.class);
    builder.withGroup("internal").withDescription("Processes that Druid runs \"internally\", you should rarely use these directly").withDefaultCommand(Help.class).withCommands(CliPeon.class, CliInternalHadoopIndexer.class);
    final Injector injector = GuiceInjectors.makeStartupInjector();
    final ExtensionsConfig config = injector.getInstance(ExtensionsConfig.class);
    final Collection<CliCommandCreator> extensionCommands = Initialization.getFromExtensions(config, CliCommandCreator.class);
    for (CliCommandCreator creator : extensionCommands) {
        creator.addCommands(builder);
    }
    final Cli<Runnable> cli = builder.build();
    try {
        final Runnable command = cli.parse(args);
        if (!(command instanceof Help)) {
            // Hack to work around Help not liking being injected
            injector.injectMembers(command);
        }
        command.run();
    } catch (ParseException e) {
        System.out.println("ERROR!!!!");
        System.out.println(e.getMessage());
        System.out.println("===");
        cli.parse(new String[] { "help" }).run();
        System.exit(1);
    }
}
Also used : Help(com.github.rvesse.airline.help.Help) Injector(com.google.inject.Injector) ParseException(com.github.rvesse.airline.parser.errors.ParseException) ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) SuppressForbidden(io.netty.util.SuppressForbidden)

Example 3 with ExtensionsConfig

use of org.apache.druid.guice.ExtensionsConfig in project druid by druid-io.

the class InitializationTest method test04DuplicateClassLoaderExtensions.

@Test
public void test04DuplicateClassLoaderExtensions() throws Exception {
    final File extensionDir = temporaryFolder.newFolder();
    Initialization.getLoadersMap().put(extensionDir, new URLClassLoader(new URL[] {}, Initialization.class.getClassLoader()));
    Collection<DruidModule> modules = Initialization.getFromExtensions(new ExtensionsConfig(), DruidModule.class);
    Set<String> loadedModuleNames = new HashSet<>();
    for (DruidModule module : modules) {
        Assert.assertFalse("Duplicate extensions are loaded", loadedModuleNames.contains(module.getClass().getName()));
        loadedModuleNames.add(module.getClass().getName());
    }
    Initialization.getLoadersMap().clear();
}
Also used : URLClassLoader(java.net.URLClassLoader) ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) File(java.io.File) URL(java.net.URL) HashSet(java.util.HashSet) LinkedHashSet(java.util.LinkedHashSet) Test(org.junit.Test)

Example 4 with ExtensionsConfig

use of org.apache.druid.guice.ExtensionsConfig in project druid by druid-io.

the class InitializationTest method testGetExtensionFilesToLoad_non_exist_extensions_dir.

@Test
public void testGetExtensionFilesToLoad_non_exist_extensions_dir() throws IOException {
    final File tmpDir = temporaryFolder.newFolder();
    Assert.assertTrue("could not create missing folder", !tmpDir.exists() || tmpDir.delete());
    Assert.assertArrayEquals("Non-exist root extensionsDir should return an empty array of File", new File[] {}, Initialization.getExtensionFilesToLoad(new ExtensionsConfig() {

        @Override
        public String getDirectory() {
            return tmpDir.getAbsolutePath();
        }
    }));
}
Also used : ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) File(java.io.File) Test(org.junit.Test)

Example 5 with ExtensionsConfig

use of org.apache.druid.guice.ExtensionsConfig in project druid by druid-io.

the class InitializationTest method testGetExtensionFilesToLoad_wrong_type_extensions_dir.

@Test(expected = ISE.class)
public void testGetExtensionFilesToLoad_wrong_type_extensions_dir() throws IOException {
    final File extensionsDir = temporaryFolder.newFile();
    final ExtensionsConfig config = new ExtensionsConfig() {

        @Override
        public String getDirectory() {
            return extensionsDir.getAbsolutePath();
        }
    };
    Initialization.getExtensionFilesToLoad(config);
}
Also used : ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) File(java.io.File) Test(org.junit.Test)

Aggregations

ExtensionsConfig (org.apache.druid.guice.ExtensionsConfig)12 File (java.io.File)10 Test (org.junit.Test)10 Help (com.github.rvesse.airline.help.Help)1 ParseException (com.github.rvesse.airline.parser.errors.ParseException)1 Injector (com.google.inject.Injector)1 Module (com.google.inject.Module)1 SuppressForbidden (io.netty.util.SuppressForbidden)1 URL (java.net.URL)1 URLClassLoader (java.net.URLClassLoader)1 HashSet (java.util.HashSet)1 LinkedHashSet (java.util.LinkedHashSet)1 CuratorModule (org.apache.druid.curator.CuratorModule)1 DiscoveryModule (org.apache.druid.curator.discovery.DiscoveryModule)1 AnnouncerModule (org.apache.druid.guice.AnnouncerModule)1 CoordinatorDiscoveryModule (org.apache.druid.guice.CoordinatorDiscoveryModule)1 DruidProcessingConfigModule (org.apache.druid.guice.DruidProcessingConfigModule)1 DruidSecondaryModule (org.apache.druid.guice.DruidSecondaryModule)1 ExpressionModule (org.apache.druid.guice.ExpressionModule)1 FirehoseModule (org.apache.druid.guice.FirehoseModule)1