Search in sources :

Example 1 with UriExtractionNamespace

use of org.apache.druid.query.lookup.namespace.UriExtractionNamespace in project druid by druid-io.

the class NamespaceLookupExtractorFactoryTest method testSimpleSerde.

@Test
public void testSimpleSerde() throws Exception {
    final UriExtractionNamespace uriExtractionNamespace = new UriExtractionNamespace(temporaryFolder.newFolder().toURI(), null, null, new UriExtractionNamespace.ObjectMapperFlatDataParser(mapper), Period.millis(0), null, null);
    final NamespaceLookupExtractorFactory namespaceLookupExtractorFactory = new NamespaceLookupExtractorFactory(uriExtractionNamespace, scheduler);
    Assert.assertEquals(uriExtractionNamespace, mapper.readValue(mapper.writeValueAsString(namespaceLookupExtractorFactory), NamespaceLookupExtractorFactory.class).getExtractionNamespace());
}
Also used : UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) Test(org.junit.Test)

Example 2 with UriExtractionNamespace

use of org.apache.druid.query.lookup.namespace.UriExtractionNamespace in project druid by druid-io.

the class CacheSchedulerTest method testPeriodicUpdatesScheduled.

@Test(timeout = 60_000L)
public void testPeriodicUpdatesScheduled() throws InterruptedException {
    final int repeatCount = 5;
    final long delay = 5;
    try {
        final UriExtractionNamespace namespace = getUriExtractionNamespace(delay);
        final long start = System.currentTimeMillis();
        try (CacheScheduler.Entry entry = scheduler.schedule(namespace)) {
            Assert.assertFalse(entry.getUpdaterFuture().isDone());
            Assert.assertFalse(entry.getUpdaterFuture().isCancelled());
            entry.awaitTotalUpdates(repeatCount);
            long minEnd = start + ((repeatCount - 1) * delay);
            long end = System.currentTimeMillis();
            Assert.assertTrue(StringUtils.format("Didn't wait long enough between runs. Expected more than %d was %d", minEnd - start, end - start), minEnd <= end);
        }
    } finally {
        lifecycle.stop();
        cacheManager.waitForServiceToEnd(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
    }
    checkNoMoreRunning();
}
Also used : UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) UriExtractionNamespaceTest(org.apache.druid.query.lookup.namespace.UriExtractionNamespaceTest) Test(org.junit.Test)

Example 3 with UriExtractionNamespace

use of org.apache.druid.query.lookup.namespace.UriExtractionNamespace in project druid by druid-io.

the class CacheSchedulerTest method testSimpleSubmission.

@Test(timeout = 60_000L)
public void testSimpleSubmission() throws InterruptedException {
    UriExtractionNamespace namespace = new UriExtractionNamespace(tmpFile.toURI(), null, null, new UriExtractionNamespace.ObjectMapperFlatDataParser(UriExtractionNamespaceTest.registerTypes(new ObjectMapper())), new Period(0), null, null);
    CacheScheduler.Entry entry = scheduler.schedule(namespace);
    waitFor(entry);
    Map<String, String> cache = entry.getCache();
    Assert.assertNull(cache.put("key", "val"));
    Assert.assertEquals("val", cache.get("key"));
}
Also used : Period(org.joda.time.Period) UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) UriExtractionNamespaceTest(org.apache.druid.query.lookup.namespace.UriExtractionNamespaceTest) Test(org.junit.Test)

Example 4 with UriExtractionNamespace

use of org.apache.druid.query.lookup.namespace.UriExtractionNamespace in project druid by druid-io.

the class NamespacedExtractorModuleTest method testListNamespaces.

@Test
public void testListNamespaces() throws Exception {
    final File tmpFile = temporaryFolder.newFile();
    try (Writer out = Files.newWriter(tmpFile, StandardCharsets.UTF_8)) {
        out.write(MAPPER.writeValueAsString(ImmutableMap.of("foo", "bar")));
    }
    final UriExtractionNamespace namespace = new UriExtractionNamespace(tmpFile.toURI(), null, null, new UriExtractionNamespace.ObjectMapperFlatDataParser(UriExtractionNamespaceTest.registerTypes(new DefaultObjectMapper())), new Period(0), null, null);
    try (CacheScheduler.Entry entry = scheduler.scheduleAndWait(namespace, 1_000)) {
        Assert.assertNotNull(entry);
        entry.awaitTotalUpdates(1);
        Assert.assertEquals(1, scheduler.getActiveEntries());
    }
}
Also used : Period(org.joda.time.Period) UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) File(java.io.File) CacheScheduler(org.apache.druid.server.lookup.namespace.cache.CacheScheduler) Writer(java.io.Writer) UriExtractionNamespaceTest(org.apache.druid.query.lookup.namespace.UriExtractionNamespaceTest) Test(org.junit.Test)

Example 5 with UriExtractionNamespace

use of org.apache.druid.query.lookup.namespace.UriExtractionNamespace in project druid by druid-io.

the class NamespacedExtractorModuleTest method testDeleteNamespaces.

// (timeout = 60_000L)
@Test
public void testDeleteNamespaces() throws Exception {
    final File tmpFile = temporaryFolder.newFile();
    try (Writer out = Files.newWriter(tmpFile, StandardCharsets.UTF_8)) {
        out.write(MAPPER.writeValueAsString(ImmutableMap.of("foo", "bar")));
    }
    final UriExtractionNamespace namespace = new UriExtractionNamespace(tmpFile.toURI(), null, null, new UriExtractionNamespace.ObjectMapperFlatDataParser(UriExtractionNamespaceTest.registerTypes(new DefaultObjectMapper())), new Period(0), null, null);
    try (CacheScheduler.Entry entry = scheduler.scheduleAndWait(namespace, 1_000)) {
        Assert.assertNotNull(entry);
    }
}
Also used : Period(org.joda.time.Period) UriExtractionNamespace(org.apache.druid.query.lookup.namespace.UriExtractionNamespace) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) File(java.io.File) CacheScheduler(org.apache.druid.server.lookup.namespace.cache.CacheScheduler) Writer(java.io.Writer) UriExtractionNamespaceTest(org.apache.druid.query.lookup.namespace.UriExtractionNamespaceTest) Test(org.junit.Test)

Aggregations

UriExtractionNamespace (org.apache.druid.query.lookup.namespace.UriExtractionNamespace)18 Test (org.junit.Test)15 UriExtractionNamespaceTest (org.apache.druid.query.lookup.namespace.UriExtractionNamespaceTest)14 File (java.io.File)7 DefaultObjectMapper (org.apache.druid.jackson.DefaultObjectMapper)7 Period (org.joda.time.Period)7 CacheScheduler (org.apache.druid.server.lookup.namespace.cache.CacheScheduler)6 CacheSchedulerTest (org.apache.druid.server.lookup.namespace.cache.CacheSchedulerTest)6 Writer (java.io.Writer)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)3 FileOutputStream (java.io.FileOutputStream)2 OutputStream (java.io.OutputStream)2 OutputStreamWriter (java.io.OutputStreamWriter)2 URI (java.net.URI)2 Before (org.junit.Before)2 Path (java.nio.file.Path)1 ArrayList (java.util.ArrayList)1 CancellationException (java.util.concurrent.CancellationException)1 ExecutionException (java.util.concurrent.ExecutionException)1 GZIPOutputStream (java.util.zip.GZIPOutputStream)1