Search in sources :

Example 31 with Lifecycle

use of org.apache.druid.java.util.common.lifecycle.Lifecycle in project druid by druid-io.

the class HttpClientInit method createClient.

public static HttpClient createClient(HttpClientConfig config, Lifecycle lifecycle) {
    try {
        // We need to use the full constructor in order to set a ThreadNameDeterminer. The other parameters are taken
        // from the defaults in HashedWheelTimer's other constructors.
        final HashedWheelTimer timer = new HashedWheelTimer(new ThreadFactoryBuilder().setDaemon(true).setNameFormat("HttpClient-Timer-%s").build(), ThreadNameDeterminer.CURRENT, 100, TimeUnit.MILLISECONDS, 512);
        lifecycle.addMaybeStartHandler(new Lifecycle.Handler() {

            @Override
            public void start() {
                timer.start();
            }

            @Override
            public void stop() {
                timer.stop();
            }
        });
        return lifecycle.addMaybeStartManagedInstance(new NettyHttpClient(new ResourcePool<>(new ChannelResourceFactory(createBootstrap(lifecycle, timer, config.getBossPoolSize(), config.getWorkerPoolSize()), config.getSslContext(), config.getProxyConfig(), timer, config.getSslHandshakeTimeout() == null ? -1 : config.getSslHandshakeTimeout().getMillis()), new ResourcePoolConfig(config.getNumConnections(), config.getUnusedConnectionTimeoutDuration().getMillis())), config.getReadTimeout(), config.getCompressionCodec(), timer));
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : ChannelResourceFactory(org.apache.druid.java.util.http.client.pool.ChannelResourceFactory) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) ThreadFactoryBuilder(com.google.common.util.concurrent.ThreadFactoryBuilder) HashedWheelTimer(org.jboss.netty.util.HashedWheelTimer) ResourcePool(org.apache.druid.java.util.http.client.pool.ResourcePool) ResourcePoolConfig(org.apache.druid.java.util.http.client.pool.ResourcePoolConfig)

Example 32 with Lifecycle

use of org.apache.druid.java.util.common.lifecycle.Lifecycle in project druid by druid-io.

the class LifecycleScopeTest method testAnnotatedAndExplicit.

/**
 * This is a test for documentation purposes.  It's there to show what weird things Guice will do when
 * it sees both the annotation and an explicit binding.
 *
 * @throws Exception
 */
@Test
public void testAnnotatedAndExplicit() throws Exception {
    final Injector injector = Guice.createInjector(new DruidGuiceExtensions(), new LifecycleModule(), new Module() {

        @Override
        public void configure(Binder binder) {
            binder.bind(TestInterface.class).to(AnnotatedClass.class).in(ManageLifecycle.class);
        }
    });
    final Lifecycle lifecycle = injector.getInstance(Lifecycle.class);
    final TestInterface instance = injector.getInstance(TestInterface.class);
    Assert.assertEquals(0, instance.getStarted());
    Assert.assertEquals(0, instance.getStopped());
    Assert.assertEquals(0, instance.getRan());
    instance.run();
    Assert.assertEquals(0, instance.getStarted());
    Assert.assertEquals(0, instance.getStopped());
    Assert.assertEquals(1, instance.getRan());
    lifecycle.start();
    Assert.assertEquals(2, instance.getStarted());
    Assert.assertEquals(0, instance.getStopped());
    Assert.assertEquals(1, instance.getRan());
    // It's a singleton
    injector.getInstance(TestInterface.class).run();
    Assert.assertEquals(2, instance.getStarted());
    Assert.assertEquals(0, instance.getStopped());
    Assert.assertEquals(2, instance.getRan());
    lifecycle.stop();
    Assert.assertEquals(2, instance.getStarted());
    Assert.assertEquals(2, instance.getStopped());
    Assert.assertEquals(2, instance.getRan());
}
Also used : Binder(com.google.inject.Binder) Injector(com.google.inject.Injector) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) Module(com.google.inject.Module) Test(org.junit.Test)

Example 33 with Lifecycle

use of org.apache.druid.java.util.common.lifecycle.Lifecycle in project druid by druid-io.

the class UriCacheGeneratorTest method getParameters.

@Parameterized.Parameters(name = "{0}")
public static Iterable<Object[]> getParameters() {
    final List<Object[]> compressionParams = ImmutableList.of(new Object[] { ".dat", new Function<File, OutputStream>() {

        @Nullable
        @Override
        public OutputStream apply(@Nullable File outFile) {
            try {
                return new FileOutputStream(outFile);
            } catch (IOException ex) {
                throw new RuntimeException(ex);
            }
        }
    } }, new Object[] { ".gz", new Function<File, OutputStream>() {

        @Nullable
        @Override
        public OutputStream apply(@Nullable File outFile) {
            try {
                final FileOutputStream fos = new FileOutputStream(outFile);
                return new GZIPOutputStream(fos) {

                    @Override
                    public void close() throws IOException {
                        try {
                            super.close();
                        } finally {
                            fos.close();
                        }
                    }
                };
            } catch (IOException ex) {
                throw new RuntimeException(ex);
            }
        }
    } });
    final List<Function<Lifecycle, NamespaceExtractionCacheManager>> cacheManagerCreators = ImmutableList.of(new Function<Lifecycle, NamespaceExtractionCacheManager>() {

        @Override
        public NamespaceExtractionCacheManager apply(Lifecycle lifecycle) {
            return new OnHeapNamespaceExtractionCacheManager(lifecycle, new NoopServiceEmitter(), new NamespaceExtractionConfig());
        }
    }, new Function<Lifecycle, NamespaceExtractionCacheManager>() {

        @Override
        public NamespaceExtractionCacheManager apply(Lifecycle lifecycle) {
            return new OffHeapNamespaceExtractionCacheManager(lifecycle, new NoopServiceEmitter(), new NamespaceExtractionConfig());
        }
    });
    return new Iterable<Object[]>() {

        @Override
        public Iterator<Object[]> iterator() {
            return new Iterator<Object[]>() {

                Iterator<Object[]> compressionIt = compressionParams.iterator();

                Iterator<Function<Lifecycle, NamespaceExtractionCacheManager>> cacheManagerCreatorsIt = cacheManagerCreators.iterator();

                Object[] compressions = compressionIt.next();

                @Override
                public boolean hasNext() {
                    return compressionIt.hasNext() || cacheManagerCreatorsIt.hasNext();
                }

                @Override
                public Object[] next() {
                    if (cacheManagerCreatorsIt.hasNext()) {
                        Function<Lifecycle, NamespaceExtractionCacheManager> cacheManagerCreator = cacheManagerCreatorsIt.next();
                        return new Object[] { compressions[0], compressions[1], cacheManagerCreator };
                    } else {
                        cacheManagerCreatorsIt = cacheManagerCreators.iterator();
                        compressions = compressionIt.next();
                        return next();
                    }
                }

                @Override
                public void remove() {
                    throw new UOE("Cannot remove");
                }
            };
        }
    };
}
Also used : OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) GZIPOutputStream(java.util.zip.GZIPOutputStream) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) UOE(org.apache.druid.java.util.common.UOE) IOException(java.io.IOException) OffHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OffHeapNamespaceExtractionCacheManager) Function(com.google.common.base.Function) OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) OffHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OffHeapNamespaceExtractionCacheManager) NamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.NamespaceExtractionCacheManager) GZIPOutputStream(java.util.zip.GZIPOutputStream) FileOutputStream(java.io.FileOutputStream) Iterator(java.util.Iterator) File(java.io.File) Nullable(javax.annotation.Nullable)

Example 34 with Lifecycle

use of org.apache.druid.java.util.common.lifecycle.Lifecycle in project druid by druid-io.

the class StaticMapCacheGeneratorTest method setup.

@Before
public void setup() throws Exception {
    lifecycle = new Lifecycle();
    lifecycle.start();
    NoopServiceEmitter noopServiceEmitter = new NoopServiceEmitter();
    scheduler = new CacheScheduler(noopServiceEmitter, Collections.emptyMap(), new OnHeapNamespaceExtractionCacheManager(lifecycle, noopServiceEmitter, new NamespaceExtractionConfig()));
}
Also used : OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) CacheScheduler(org.apache.druid.server.lookup.namespace.cache.CacheScheduler) Before(org.junit.Before)

Example 35 with Lifecycle

use of org.apache.druid.java.util.common.lifecycle.Lifecycle in project druid by druid-io.

the class CacheSchedulerTest method setUp.

@Before
public void setUp() throws Exception {
    lifecycle = new Lifecycle();
    lifecycle.start();
    cacheManager = createCacheManager.apply(lifecycle);
    final Path tmpDir = temporaryFolder.newFolder().toPath();
    final CacheGenerator<UriExtractionNamespace> cacheGenerator = new CacheGenerator<UriExtractionNamespace>() {

        @Override
        public CacheScheduler.VersionedCache generateCache(final UriExtractionNamespace extractionNamespace, final CacheScheduler.EntryImpl<UriExtractionNamespace> id, final String lastVersion, final CacheScheduler scheduler) throws InterruptedException {
            // To make absolutely sure there is a unique currentTimeMillis
            Thread.sleep(2);
            String version = Long.toString(System.currentTimeMillis());
            CacheScheduler.VersionedCache versionedCache = scheduler.createVersionedCache(id, version);
            // Don't actually read off disk because TravisCI doesn't like that
            versionedCache.getCache().put(KEY, VALUE);
            return versionedCache;
        }
    };
    scheduler = new CacheScheduler(new NoopServiceEmitter(), ImmutableMap.of(UriExtractionNamespace.class, cacheGenerator), cacheManager);
    tmpFile = Files.createTempFile(tmpDir, "druidTestURIExtractionNS", ".dat").toFile();
    try (OutputStream ostream = new FileOutputStream(tmpFile)) {
        try (OutputStreamWriter out = new OutputStreamWriter(ostream, StandardCharsets.UTF_8)) {
            // Since Travis sucks with disk related stuff, we override the disk reading part above.
            // This is safe and should shake out any problem areas that accidentally read the file.
            out.write("SHOULDN'T TRY TO PARSE");
            out.flush();
        }
    }
}
Also used : Path(java.nio.file.Path) CacheGenerator(org.apache.druid.query.lookup.namespace.CacheGenerator) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) FileOutputStream(java.io.FileOutputStream) UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) OutputStreamWriter(java.io.OutputStreamWriter) Before(org.junit.Before)

Aggregations

Lifecycle (org.apache.druid.java.util.common.lifecycle.Lifecycle)46 Test (org.junit.Test)21 Injector (com.google.inject.Injector)12 URL (java.net.URL)12 ExecutionException (java.util.concurrent.ExecutionException)12 StatusResponseHolder (org.apache.druid.java.util.http.client.response.StatusResponseHolder)10 Before (org.junit.Before)10 ExecutorService (java.util.concurrent.ExecutorService)6 Binder (com.google.inject.Binder)5 Module (com.google.inject.Module)5 OutputStream (java.io.OutputStream)5 Properties (java.util.Properties)5 ManageLifecycle (org.apache.druid.guice.ManageLifecycle)5 NoopServiceEmitter (org.apache.druid.server.metrics.NoopServiceEmitter)5 ChannelException (org.jboss.netty.channel.ChannelException)5 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)4 IOException (java.io.IOException)4 SSLHandshakeException (javax.net.ssl.SSLHandshakeException)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 Provides (com.google.inject.Provides)3