Search in sources :

Example 1 with SuppressForbidden

use of io.netty.util.SuppressForbidden in project druid by druid-io.

the class CuratorModule method makeCurator.

@Provides
@LazySingleton
@SuppressForbidden(reason = "System#err")
public CuratorFramework makeCurator(ZkEnablementConfig zkEnablementConfig, CuratorConfig config, EnsembleProvider ensembleProvider, Lifecycle lifecycle) {
    if (!zkEnablementConfig.isEnabled()) {
        throw new RuntimeException("Zookeeper is disabled, Can't create CuratorFramework.");
    }
    final CuratorFrameworkFactory.Builder builder = CuratorFrameworkFactory.builder();
    if (!Strings.isNullOrEmpty(config.getZkUser()) && !Strings.isNullOrEmpty(config.getZkPwd())) {
        builder.authorization(config.getAuthScheme(), StringUtils.format("%s:%s", config.getZkUser(), config.getZkPwd()).getBytes(StandardCharsets.UTF_8));
    }
    RetryPolicy retryPolicy = new BoundedExponentialBackoffRetry(BASE_SLEEP_TIME_MS, MAX_SLEEP_TIME_MS, MAX_RETRIES);
    final CuratorFramework framework = builder.ensembleProvider(ensembleProvider).sessionTimeoutMs(config.getZkSessionTimeoutMs()).connectionTimeoutMs(config.getZkConnectionTimeoutMs()).retryPolicy(retryPolicy).compressionProvider(new PotentiallyGzippedCompressionProvider(config.getEnableCompression())).aclProvider(config.getEnableAcl() ? new SecuredACLProvider() : new DefaultACLProvider()).build();
    framework.getUnhandledErrorListenable().addListener((message, e) -> {
        log.error(e, "Unhandled error in Curator, stopping server.");
        shutdown(lifecycle);
    });
    lifecycle.addHandler(new Lifecycle.Handler() {

        @Override
        public void start() {
            log.debug("Starting Curator");
            framework.start();
        }

        @Override
        public void stop() {
            log.debug("Stopping Curator");
            framework.close();
        }
    });
    return framework;
}
Also used : CuratorFramework(org.apache.curator.framework.CuratorFramework) CuratorFrameworkFactory(org.apache.curator.framework.CuratorFrameworkFactory) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) DefaultACLProvider(org.apache.curator.framework.imps.DefaultACLProvider) RetryPolicy(org.apache.curator.RetryPolicy) BoundedExponentialBackoffRetry(org.apache.curator.retry.BoundedExponentialBackoffRetry) LazySingleton(org.apache.druid.guice.LazySingleton) SuppressForbidden(io.netty.util.SuppressForbidden) Provides(com.google.inject.Provides)

Example 2 with SuppressForbidden

use of io.netty.util.SuppressForbidden in project druid by druid-io.

the class Main method main.

@SuppressWarnings("unchecked")
@SuppressForbidden(reason = "System#out")
public static void main(String[] args) {
    final CliBuilder<Runnable> builder = Cli.builder("druid");
    builder.withDescription("Druid command-line runner.").withDefaultCommand(Help.class).withCommands(Help.class, Version.class);
    List<Class<? extends Runnable>> serverCommands = Arrays.asList(CliCoordinator.class, CliHistorical.class, CliBroker.class, CliOverlord.class, CliIndexer.class, CliMiddleManager.class, CliRouter.class);
    builder.withGroup("server").withDescription("Run one of the Druid server types.").withDefaultCommand(Help.class).withCommands(serverCommands);
    List<Class<? extends Runnable>> toolCommands = Arrays.asList(DruidJsonValidator.class, PullDependencies.class, CreateTables.class, DumpSegment.class, ResetCluster.class, ValidateSegments.class, ExportMetadata.class);
    builder.withGroup("tools").withDescription("Various tools for working with Druid").withDefaultCommand(Help.class).withCommands(toolCommands);
    builder.withGroup("index").withDescription("Run indexing for druid").withDefaultCommand(Help.class).withCommands(CliHadoopIndexer.class);
    builder.withGroup("internal").withDescription("Processes that Druid runs \"internally\", you should rarely use these directly").withDefaultCommand(Help.class).withCommands(CliPeon.class, CliInternalHadoopIndexer.class);
    final Injector injector = GuiceInjectors.makeStartupInjector();
    final ExtensionsConfig config = injector.getInstance(ExtensionsConfig.class);
    final Collection<CliCommandCreator> extensionCommands = Initialization.getFromExtensions(config, CliCommandCreator.class);
    for (CliCommandCreator creator : extensionCommands) {
        creator.addCommands(builder);
    }
    final Cli<Runnable> cli = builder.build();
    try {
        final Runnable command = cli.parse(args);
        if (!(command instanceof Help)) {
            // Hack to work around Help not liking being injected
            injector.injectMembers(command);
        }
        command.run();
    } catch (ParseException e) {
        System.out.println("ERROR!!!!");
        System.out.println(e.getMessage());
        System.out.println("===");
        cli.parse(new String[] { "help" }).run();
        System.exit(1);
    }
}
Also used : Help(com.github.rvesse.airline.help.Help) Injector(com.google.inject.Injector) ParseException(com.github.rvesse.airline.parser.errors.ParseException) ExtensionsConfig(org.apache.druid.guice.ExtensionsConfig) SuppressForbidden(io.netty.util.SuppressForbidden)

Example 3 with SuppressForbidden

use of io.netty.util.SuppressForbidden in project druid by druid-io.

the class PullDependencies method getAetherClient.

@SuppressForbidden(reason = "System#out")
private DefaultTeslaAether getAetherClient() {
    /*
    DefaultTeslaAether logs a bunch of stuff to System.out, which is annoying.  We choose to disable that
    unless debug logging is turned on.  "Disabling" it, however, is kinda bass-ackwards.  We copy out a reference
    to the current System.out, and set System.out to a noop output stream.  Then after DefaultTeslaAether has pulled
    The reference we swap things back.

    This has implications for other things that are running in parallel to this.  Namely, if anything else also grabs
    a reference to System.out or tries to log to it while we have things adjusted like this, then they will also log
    to nothingness.  Fortunately, the code that calls this is single-threaded and shouldn't hopefully be running
    alongside anything else that's grabbing System.out.  But who knows.
    */
    final List<String> remoteUriList = new ArrayList<>();
    if (!noDefaultRemoteRepositories) {
        remoteUriList.addAll(DEFAULT_REMOTE_REPOSITORIES);
    }
    remoteUriList.addAll(remoteRepositories);
    List<Repository> remoteRepositories = new ArrayList<>();
    for (String uri : remoteUriList) {
        try {
            URI u = new URI(uri);
            Repository r = new Repository(uri);
            if (u.getUserInfo() != null) {
                String[] auth = u.getUserInfo().split(":", 2);
                if (auth.length == 2) {
                    r.setUsername(auth[0]);
                    r.setPassword(auth[1]);
                } else {
                    log.warn("Invalid credentials in repository URI, expecting [<user>:<password>], got [%s] for [%s]", u.getUserInfo(), uri);
                }
            }
            remoteRepositories.add(r);
        } catch (URISyntaxException e) {
            throw new RuntimeException(e);
        }
    }
    if (log.isTraceEnabled() || log.isDebugEnabled()) {
        return createTeslaAether(remoteRepositories);
    }
    PrintStream oldOut = System.out;
    try {
        System.setOut(new PrintStream(new OutputStream() {

            @Override
            public void write(int b) {
            }

            @Override
            public void write(byte[] b) {
            }

            @Override
            public void write(byte[] b, int off, int len) {
            }
        }, false, StringUtils.UTF8_STRING));
        return createTeslaAether(remoteRepositories);
    } catch (UnsupportedEncodingException e) {
        // should never happen
        throw new IllegalStateException(e);
    } finally {
        System.setOut(oldOut);
    }
}
Also used : PrintStream(java.io.PrintStream) OutputStream(java.io.OutputStream) ArrayList(java.util.ArrayList) UnsupportedEncodingException(java.io.UnsupportedEncodingException) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) Repository(io.tesla.aether.Repository) RemoteRepository(org.eclipse.aether.repository.RemoteRepository) SuppressForbidden(io.netty.util.SuppressForbidden)

Example 4 with SuppressForbidden

use of io.netty.util.SuppressForbidden in project netty by netty.

the class SctpLimitStreamsTest method testSctpInitMaxstreams.

@SuppressForbidden(reason = "test-only")
@Test
@Timeout(value = 5000, unit = TimeUnit.MILLISECONDS)
public void testSctpInitMaxstreams() throws Exception {
    EventLoopGroup loop = newEventLoopGroup();
    try {
        ServerBootstrap serverBootstrap = new ServerBootstrap();
        serverBootstrap.group(loop).channel(serverClass()).option(ChannelOption.SO_REUSEADDR, true).option(SctpChannelOption.SCTP_INIT_MAXSTREAMS, SctpStandardSocketOptions.InitMaxStreams.create(1, 1)).localAddress(new InetSocketAddress(0)).childHandler(new ChannelInboundHandlerAdapter());
        Bootstrap clientBootstrap = new Bootstrap().group(loop).channel(clientClass()).option(SctpChannelOption.SCTP_INIT_MAXSTREAMS, SctpStandardSocketOptions.InitMaxStreams.create(112, 112)).handler(new ChannelInboundHandlerAdapter());
        Channel serverChannel = serverBootstrap.bind().syncUninterruptibly().channel();
        SctpChannel clientChannel = (SctpChannel) clientBootstrap.connect(serverChannel.localAddress()).syncUninterruptibly().channel();
        assertEquals(1, clientChannel.association().maxOutboundStreams());
        assertEquals(1, clientChannel.association().maxInboundStreams());
        serverChannel.close().syncUninterruptibly();
        clientChannel.close().syncUninterruptibly();
    } finally {
        loop.shutdownGracefully();
    }
}
Also used : EventLoopGroup(io.netty.channel.EventLoopGroup) InetSocketAddress(java.net.InetSocketAddress) Channel(io.netty.channel.Channel) Bootstrap(io.netty.bootstrap.Bootstrap) ServerBootstrap(io.netty.bootstrap.ServerBootstrap) ServerBootstrap(io.netty.bootstrap.ServerBootstrap) ChannelInboundHandlerAdapter(io.netty.channel.ChannelInboundHandlerAdapter) Test(org.junit.jupiter.api.Test) Timeout(org.junit.jupiter.api.Timeout) SuppressForbidden(io.netty.util.SuppressForbidden)

Example 5 with SuppressForbidden

use of io.netty.util.SuppressForbidden in project druid by druid-io.

the class CliPeon method run.

@SuppressForbidden(reason = "System#out, System#err")
@Override
public void run() {
    try {
        Injector injector = makeInjector();
        try {
            final Lifecycle lifecycle = initLifecycle(injector);
            final Thread hook = new Thread(() -> {
                log.info("Running shutdown hook");
                lifecycle.stop();
            });
            Runtime.getRuntime().addShutdownHook(hook);
            injector.getInstance(ExecutorLifecycle.class).join();
            // Sanity check to help debug unexpected non-daemon threads
            final Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
            for (Thread thread : threadSet) {
                if (!thread.isDaemon() && thread != Thread.currentThread()) {
                    log.info("Thread [%s] is non daemon.", thread);
                }
            }
            // Explicitly call lifecycle stop, dont rely on shutdown hook.
            lifecycle.stop();
            try {
                Runtime.getRuntime().removeShutdownHook(hook);
            } catch (IllegalStateException e) {
                System.err.println("Cannot remove shutdown hook, already shutting down!");
            }
        } catch (Throwable t) {
            System.err.println("Error!");
            System.err.println(Throwables.getStackTraceAsString(t));
            System.exit(1);
        }
        System.out.println("Finished peon task");
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : Injector(com.google.inject.Injector) ExecutorLifecycle(org.apache.druid.indexing.worker.executor.ExecutorLifecycle) ManageLifecycle(org.apache.druid.guice.ManageLifecycle) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) ExecutorLifecycle(org.apache.druid.indexing.worker.executor.ExecutorLifecycle) IOException(java.io.IOException) SuppressForbidden(io.netty.util.SuppressForbidden)

Aggregations

SuppressForbidden (io.netty.util.SuppressForbidden)5 Injector (com.google.inject.Injector)2 Lifecycle (org.apache.druid.java.util.common.lifecycle.Lifecycle)2 Help (com.github.rvesse.airline.help.Help)1 ParseException (com.github.rvesse.airline.parser.errors.ParseException)1 Provides (com.google.inject.Provides)1 Bootstrap (io.netty.bootstrap.Bootstrap)1 ServerBootstrap (io.netty.bootstrap.ServerBootstrap)1 Channel (io.netty.channel.Channel)1 ChannelInboundHandlerAdapter (io.netty.channel.ChannelInboundHandlerAdapter)1 EventLoopGroup (io.netty.channel.EventLoopGroup)1 Repository (io.tesla.aether.Repository)1 IOException (java.io.IOException)1 OutputStream (java.io.OutputStream)1 PrintStream (java.io.PrintStream)1 UnsupportedEncodingException (java.io.UnsupportedEncodingException)1 InetSocketAddress (java.net.InetSocketAddress)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 ArrayList (java.util.ArrayList)1