Search in sources :

Example 1 with OnHeapNamespaceExtractionCacheManager

use of org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager in project druid by druid-io.

the class NamespacedExtractorModuleTest method setUp.

@Before
public void setUp() throws Exception {
    final Map<Class<? extends ExtractionNamespace>, CacheGenerator<?>> factoryMap = ImmutableMap.of(UriExtractionNamespace.class, new UriCacheGenerator(ImmutableMap.of("file", new LocalFileTimestampVersionFinder())), JdbcExtractionNamespace.class, new JdbcCacheGenerator());
    lifecycle = new Lifecycle();
    lifecycle.start();
    NoopServiceEmitter noopServiceEmitter = new NoopServiceEmitter();
    scheduler = new CacheScheduler(noopServiceEmitter, factoryMap, new OnHeapNamespaceExtractionCacheManager(lifecycle, noopServiceEmitter, new NamespaceExtractionConfig()));
}
Also used : CacheGenerator(org.apache.druid.query.lookup.namespace.CacheGenerator) OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) ExtractionNamespace(org.apache.druid.query.lookup.namespace.ExtractionNamespace) UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) JdbcExtractionNamespace(org.apache.druid.query.lookup.namespace.JdbcExtractionNamespace) LocalFileTimestampVersionFinder(org.apache.druid.segment.loading.LocalFileTimestampVersionFinder) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) CacheScheduler(org.apache.druid.server.lookup.namespace.cache.CacheScheduler) Before(org.junit.Before)

Example 2 with OnHeapNamespaceExtractionCacheManager

use of org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager in project druid by druid-io.

the class UriCacheGeneratorTest method getParameters.

@Parameterized.Parameters(name = "{0}")
public static Iterable<Object[]> getParameters() {
    final List<Object[]> compressionParams = ImmutableList.of(new Object[] { ".dat", new Function<File, OutputStream>() {

        @Nullable
        @Override
        public OutputStream apply(@Nullable File outFile) {
            try {
                return new FileOutputStream(outFile);
            } catch (IOException ex) {
                throw new RuntimeException(ex);
            }
        }
    } }, new Object[] { ".gz", new Function<File, OutputStream>() {

        @Nullable
        @Override
        public OutputStream apply(@Nullable File outFile) {
            try {
                final FileOutputStream fos = new FileOutputStream(outFile);
                return new GZIPOutputStream(fos) {

                    @Override
                    public void close() throws IOException {
                        try {
                            super.close();
                        } finally {
                            fos.close();
                        }
                    }
                };
            } catch (IOException ex) {
                throw new RuntimeException(ex);
            }
        }
    } });
    final List<Function<Lifecycle, NamespaceExtractionCacheManager>> cacheManagerCreators = ImmutableList.of(new Function<Lifecycle, NamespaceExtractionCacheManager>() {

        @Override
        public NamespaceExtractionCacheManager apply(Lifecycle lifecycle) {
            return new OnHeapNamespaceExtractionCacheManager(lifecycle, new NoopServiceEmitter(), new NamespaceExtractionConfig());
        }
    }, new Function<Lifecycle, NamespaceExtractionCacheManager>() {

        @Override
        public NamespaceExtractionCacheManager apply(Lifecycle lifecycle) {
            return new OffHeapNamespaceExtractionCacheManager(lifecycle, new NoopServiceEmitter(), new NamespaceExtractionConfig());
        }
    });
    return new Iterable<Object[]>() {

        @Override
        public Iterator<Object[]> iterator() {
            return new Iterator<Object[]>() {

                Iterator<Object[]> compressionIt = compressionParams.iterator();

                Iterator<Function<Lifecycle, NamespaceExtractionCacheManager>> cacheManagerCreatorsIt = cacheManagerCreators.iterator();

                Object[] compressions = compressionIt.next();

                @Override
                public boolean hasNext() {
                    return compressionIt.hasNext() || cacheManagerCreatorsIt.hasNext();
                }

                @Override
                public Object[] next() {
                    if (cacheManagerCreatorsIt.hasNext()) {
                        Function<Lifecycle, NamespaceExtractionCacheManager> cacheManagerCreator = cacheManagerCreatorsIt.next();
                        return new Object[] { compressions[0], compressions[1], cacheManagerCreator };
                    } else {
                        cacheManagerCreatorsIt = cacheManagerCreators.iterator();
                        compressions = compressionIt.next();
                        return next();
                    }
                }

                @Override
                public void remove() {
                    throw new UOE("Cannot remove");
                }
            };
        }
    };
}
Also used : OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) GZIPOutputStream(java.util.zip.GZIPOutputStream) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) UOE(org.apache.druid.java.util.common.UOE) IOException(java.io.IOException) OffHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OffHeapNamespaceExtractionCacheManager) Function(com.google.common.base.Function) OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) OffHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OffHeapNamespaceExtractionCacheManager) NamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.NamespaceExtractionCacheManager) GZIPOutputStream(java.util.zip.GZIPOutputStream) FileOutputStream(java.io.FileOutputStream) Iterator(java.util.Iterator) File(java.io.File) Nullable(javax.annotation.Nullable)

Example 3 with OnHeapNamespaceExtractionCacheManager

use of org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager in project druid by druid-io.

the class StaticMapCacheGeneratorTest method setup.

@Before
public void setup() throws Exception {
    lifecycle = new Lifecycle();
    lifecycle.start();
    NoopServiceEmitter noopServiceEmitter = new NoopServiceEmitter();
    scheduler = new CacheScheduler(noopServiceEmitter, Collections.emptyMap(), new OnHeapNamespaceExtractionCacheManager(lifecycle, noopServiceEmitter, new NamespaceExtractionConfig()));
}
Also used : OnHeapNamespaceExtractionCacheManager(org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager) Lifecycle(org.apache.druid.java.util.common.lifecycle.Lifecycle) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) CacheScheduler(org.apache.druid.server.lookup.namespace.cache.CacheScheduler) Before(org.junit.Before)

Aggregations

Lifecycle (org.apache.druid.java.util.common.lifecycle.Lifecycle)3 OnHeapNamespaceExtractionCacheManager (org.apache.druid.server.lookup.namespace.cache.OnHeapNamespaceExtractionCacheManager)3 NoopServiceEmitter (org.apache.druid.server.metrics.NoopServiceEmitter)3 CacheScheduler (org.apache.druid.server.lookup.namespace.cache.CacheScheduler)2 Before (org.junit.Before)2 Function (com.google.common.base.Function)1 File (java.io.File)1 FileOutputStream (java.io.FileOutputStream)1 IOException (java.io.IOException)1 OutputStream (java.io.OutputStream)1 Iterator (java.util.Iterator)1 GZIPOutputStream (java.util.zip.GZIPOutputStream)1 Nullable (javax.annotation.Nullable)1 UOE (org.apache.druid.java.util.common.UOE)1 CacheGenerator (org.apache.druid.query.lookup.namespace.CacheGenerator)1 ExtractionNamespace (org.apache.druid.query.lookup.namespace.ExtractionNamespace)1 JdbcExtractionNamespace (org.apache.druid.query.lookup.namespace.JdbcExtractionNamespace)1 UriExtractionNamespace (org.apache.druid.query.lookup.namespace.UriExtractionNamespace)1 LocalFileTimestampVersionFinder (org.apache.druid.segment.loading.LocalFileTimestampVersionFinder)1 NamespaceExtractionCacheManager (org.apache.druid.server.lookup.namespace.cache.NamespaceExtractionCacheManager)1