Search in sources :

Example 1 with RegexDimFilter

use of org.apache.druid.query.filter.RegexDimFilter in project druid by druid-io.

the class LongFilteringTest method testLongFilterWithExtractionFn.

@Test
public void testLongFilterWithExtractionFn() {
    final Map<String, String> stringMap = new HashMap<>();
    stringMap.put("1", "Monday");
    stringMap.put("2", "Tuesday");
    stringMap.put("3", "Wednesday");
    stringMap.put("4", "Thursday");
    stringMap.put("5", "Friday");
    stringMap.put("6", "Saturday");
    LookupExtractor mapExtractor = new MapLookupExtractor(stringMap, false);
    LookupExtractionFn exfn = new LookupExtractionFn(mapExtractor, false, "UNKNOWN", false, true);
    assertFilterMatches(new SelectorDimFilter(LONG_COLUMN, "Monday", exfn), ImmutableList.of("1"));
    assertFilterMatches(new SelectorDimFilter(LONG_COLUMN, "Notaday", exfn), ImmutableList.of());
    assertFilterMatches(new BoundDimFilter(LONG_COLUMN, "Fridax", "Fridaz", false, false, null, exfn, StringComparators.ALPHANUMERIC), ImmutableList.of("5"));
    assertFilterMatches(new BoundDimFilter(LONG_COLUMN, "Friday", "Friday", true, true, null, exfn, StringComparators.ALPHANUMERIC), ImmutableList.of());
    assertFilterMatches(new InDimFilter(LONG_COLUMN, Arrays.asList("Caturday", "Saturday", "Tuesday"), exfn), ImmutableList.of("2", "6"));
    // test InFilter HashSet implementation
    List<String> bigList = Arrays.asList("Saturday", "Tuesday", "Caturday", "Xanaday", "Vojuday", "Gribaday", "Kipoday", "Dheferday", "Fakeday", "Qeearaday", "Hello", "World", "1", "2", "3", "4", "5", "6", "7");
    assertFilterMatches(new InDimFilter(LONG_COLUMN, bigList, exfn), ImmutableList.of("2", "6"));
    String jsFn = "function(x) { return(x === 'Wednesday' || x === 'Thursday') }";
    assertFilterMatchesSkipVectorize(new JavaScriptDimFilter(LONG_COLUMN, jsFn, exfn, JavaScriptConfig.getEnabledInstance()), ImmutableList.of("3", "4"));
    assertFilterMatches(new RegexDimFilter(LONG_COLUMN, ".*day", exfn), ImmutableList.of("1", "2", "3", "4", "5", "6"));
    assertFilterMatches(new SearchQueryDimFilter(LONG_COLUMN, new ContainsSearchQuerySpec("s", true), exfn), ImmutableList.of("2", "3", "4"));
}
Also used : RegexDimFilter(org.apache.druid.query.filter.RegexDimFilter) BoundDimFilter(org.apache.druid.query.filter.BoundDimFilter) HashMap(java.util.HashMap) ContainsSearchQuerySpec(org.apache.druid.query.search.ContainsSearchQuerySpec) MapLookupExtractor(org.apache.druid.query.extraction.MapLookupExtractor) LookupExtractor(org.apache.druid.query.lookup.LookupExtractor) LookupExtractionFn(org.apache.druid.query.lookup.LookupExtractionFn) SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) InDimFilter(org.apache.druid.query.filter.InDimFilter) SearchQueryDimFilter(org.apache.druid.query.filter.SearchQueryDimFilter) MapLookupExtractor(org.apache.druid.query.extraction.MapLookupExtractor) JavaScriptDimFilter(org.apache.druid.query.filter.JavaScriptDimFilter) Test(org.junit.Test)

Example 2 with RegexDimFilter

use of org.apache.druid.query.filter.RegexDimFilter in project druid by druid-io.

the class FloatAndDoubleFilteringTest method doTestFloatFilterWithExtractionFn.

private void doTestFloatFilterWithExtractionFn(final String columnName) {
    final Map<String, String> stringMap = new HashMap<>();
    stringMap.put("1.0", "Monday");
    stringMap.put("2.0", "Tuesday");
    stringMap.put("3.0", "Wednesday");
    stringMap.put("4.0", "Thursday");
    stringMap.put("5.0", "Friday");
    stringMap.put("6.0", "Saturday");
    LookupExtractor mapExtractor = new MapLookupExtractor(stringMap, false);
    LookupExtractionFn exfn = new LookupExtractionFn(mapExtractor, false, "UNKNOWN", false, true);
    assertFilterMatches(new SelectorDimFilter(columnName, "Monday", exfn), ImmutableList.of("1"));
    assertFilterMatches(new SelectorDimFilter(columnName, "Notaday", exfn), ImmutableList.of());
    assertFilterMatches(new BoundDimFilter(columnName, "Fridax", "Fridaz", false, false, null, exfn, StringComparators.ALPHANUMERIC), ImmutableList.of("5"));
    assertFilterMatches(new BoundDimFilter(columnName, "Friday", "Friday", true, true, null, exfn, StringComparators.ALPHANUMERIC), ImmutableList.of());
    assertFilterMatches(new InDimFilter(columnName, Arrays.asList("Caturday", "Saturday", "Tuesday"), exfn), ImmutableList.of("2", "6"));
    // test InFilter HashSet implementation
    List<String> bigList = Arrays.asList("Saturday", "Tuesday", "Caturday", "Xanaday", "Vojuday", "Gribaday", "Kipoday", "Dheferday", "Fakeday", "Qeearaday", "Hello", "World", "1", "2", "3", "4", "5", "6", "7");
    assertFilterMatches(new InDimFilter(columnName, bigList, exfn), ImmutableList.of("2", "6"));
    String jsFn = "function(x) { return(x === 'Wednesday' || x === 'Thursday') }";
    assertFilterMatchesSkipVectorize(new JavaScriptDimFilter(columnName, jsFn, exfn, JavaScriptConfig.getEnabledInstance()), ImmutableList.of("3", "4"));
    assertFilterMatches(new RegexDimFilter(columnName, ".*day", exfn), ImmutableList.of("1", "2", "3", "4", "5", "6"));
    assertFilterMatches(new SearchQueryDimFilter(columnName, new ContainsSearchQuerySpec("s", true), exfn), ImmutableList.of("2", "3", "4"));
}
Also used : RegexDimFilter(org.apache.druid.query.filter.RegexDimFilter) BoundDimFilter(org.apache.druid.query.filter.BoundDimFilter) HashMap(java.util.HashMap) ContainsSearchQuerySpec(org.apache.druid.query.search.ContainsSearchQuerySpec) MapLookupExtractor(org.apache.druid.query.extraction.MapLookupExtractor) LookupExtractor(org.apache.druid.query.lookup.LookupExtractor) LookupExtractionFn(org.apache.druid.query.lookup.LookupExtractionFn) SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) InDimFilter(org.apache.druid.query.filter.InDimFilter) SearchQueryDimFilter(org.apache.druid.query.filter.SearchQueryDimFilter) MapLookupExtractor(org.apache.druid.query.extraction.MapLookupExtractor) JavaScriptDimFilter(org.apache.druid.query.filter.JavaScriptDimFilter)

Example 3 with RegexDimFilter

use of org.apache.druid.query.filter.RegexDimFilter in project druid by druid-io.

the class FloatAndDoubleFilteringTest method doTestFloatColumnFiltering.

private void doTestFloatColumnFiltering(final String columnName) {
    assertFilterMatches(new SelectorDimFilter(columnName, "3", null), ImmutableList.of("3"));
    assertFilterMatches(new SelectorDimFilter(columnName, "3.0", null), ImmutableList.of("3"));
    assertFilterMatches(new BoundDimFilter(columnName, "2", "5", false, false, null, null, StringComparators.NUMERIC), ImmutableList.of("2", "3", "4", "5"));
    assertFilterMatches(new BoundDimFilter(columnName, "2.0", "5.0", false, false, null, null, StringComparators.NUMERIC), ImmutableList.of("2", "3", "4", "5"));
    assertFilterMatches(new BoundDimFilter(columnName, "1", "4", true, true, null, null, StringComparators.NUMERIC), ImmutableList.of("2", "3"));
    assertFilterMatches(new BoundDimFilter(columnName, "1.0", "4.0", true, true, null, null, StringComparators.NUMERIC), ImmutableList.of("2", "3"));
    assertFilterMatches(new InDimFilter(columnName, Arrays.asList("2", "4", "8"), null), ImmutableList.of("2", "4"));
    assertFilterMatches(new InDimFilter(columnName, Arrays.asList("2.0", "4.0", "8.0"), null), ImmutableList.of("2", "4"));
    // cross the hashing threshold to test hashset implementation, filter on even values
    List<String> infilterValues = new ArrayList<>(NUM_FILTER_VALUES);
    for (int i = 0; i < NUM_FILTER_VALUES; i++) {
        infilterValues.add(String.valueOf(i * 2));
    }
    assertFilterMatches(new InDimFilter(columnName, infilterValues, null), ImmutableList.of("2", "4", "6"));
    String jsFn = "function(x) { return(x === 3 || x === 5) }";
    assertFilterMatchesSkipVectorize(new JavaScriptDimFilter(columnName, jsFn, null, JavaScriptConfig.getEnabledInstance()), ImmutableList.of("3", "5"));
    String jsFn2 = "function(x) { return(x === 3.0 || x === 5.0) }";
    assertFilterMatchesSkipVectorize(new JavaScriptDimFilter(columnName, jsFn2, null, JavaScriptConfig.getEnabledInstance()), ImmutableList.of("3", "5"));
    assertFilterMatches(new RegexDimFilter(columnName, "4", null), ImmutableList.of("4"));
    assertFilterMatches(new RegexDimFilter(columnName, "4.0", null), ImmutableList.of("4"));
    assertFilterMatches(new SearchQueryDimFilter(columnName, new ContainsSearchQuerySpec("2", true), null), ImmutableList.of("2"));
    assertFilterMatches(new SearchQueryDimFilter(columnName, new ContainsSearchQuerySpec("2", true), null), ImmutableList.of("2"));
}
Also used : RegexDimFilter(org.apache.druid.query.filter.RegexDimFilter) BoundDimFilter(org.apache.druid.query.filter.BoundDimFilter) ContainsSearchQuerySpec(org.apache.druid.query.search.ContainsSearchQuerySpec) SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) InDimFilter(org.apache.druid.query.filter.InDimFilter) ArrayList(java.util.ArrayList) SearchQueryDimFilter(org.apache.druid.query.filter.SearchQueryDimFilter) JavaScriptDimFilter(org.apache.druid.query.filter.JavaScriptDimFilter)

Example 4 with RegexDimFilter

use of org.apache.druid.query.filter.RegexDimFilter in project druid by druid-io.

the class RegexFilterTest method testRegexWithExtractionFn.

@Test
public void testRegexWithExtractionFn() {
    String nullJsFn = "function(str) { if (str === null) { return 'NOT_NULL_ANYMORE'; } else { return str;} }";
    ExtractionFn changeNullFn = new JavaScriptExtractionFn(nullJsFn, false, JavaScriptConfig.getEnabledInstance());
    if (NullHandling.replaceWithDefault()) {
        assertFilterMatches(new RegexDimFilter("dim1", ".*ANYMORE", changeNullFn), ImmutableList.of("0"));
        assertFilterMatches(new RegexDimFilter("dim2", ".*ANYMORE", changeNullFn), ImmutableList.of("1", "2", "5"));
    } else {
        assertFilterMatches(new RegexDimFilter("dim1", ".*ANYMORE", changeNullFn), ImmutableList.of());
        assertFilterMatches(new RegexDimFilter("dim2", ".*ANYMORE", changeNullFn), ImmutableList.of("1", "5"));
    }
    assertFilterMatches(new RegexDimFilter("dim1", "ab.*", changeNullFn), ImmutableList.of("4", "5"));
    assertFilterMatches(new RegexDimFilter("dim2", "a.*", changeNullFn), ImmutableList.of("0", "3"));
    assertFilterMatches(new RegexDimFilter("dim3", ".*ANYMORE", changeNullFn), ImmutableList.of("0", "1", "2", "3", "4", "5"));
    assertFilterMatches(new RegexDimFilter("dim3", "a.*", changeNullFn), ImmutableList.of());
    assertFilterMatches(new RegexDimFilter("dim4", ".*ANYMORE", changeNullFn), ImmutableList.of("0", "1", "2", "3", "4", "5"));
    assertFilterMatches(new RegexDimFilter("dim4", "a.*", changeNullFn), ImmutableList.of());
}
Also used : ExtractionFn(org.apache.druid.query.extraction.ExtractionFn) JavaScriptExtractionFn(org.apache.druid.query.extraction.JavaScriptExtractionFn) RegexDimFilter(org.apache.druid.query.filter.RegexDimFilter) JavaScriptExtractionFn(org.apache.druid.query.extraction.JavaScriptExtractionFn) Test(org.junit.Test)

Example 5 with RegexDimFilter

use of org.apache.druid.query.filter.RegexDimFilter in project druid by druid-io.

the class RegexFilterTest method testSingleValueStringColumnWithoutNulls.

@Test
public void testSingleValueStringColumnWithoutNulls() {
    assertFilterMatches(new RegexDimFilter("dim0", ".*", null), ImmutableList.of("0", "1", "2", "3", "4", "5"));
    assertFilterMatches(new RegexDimFilter("dim0", "0", null), ImmutableList.of("0"));
    assertFilterMatches(new RegexDimFilter("dim0", "5", null), ImmutableList.of("5"));
}
Also used : RegexDimFilter(org.apache.druid.query.filter.RegexDimFilter) Test(org.junit.Test)

Aggregations

RegexDimFilter (org.apache.druid.query.filter.RegexDimFilter)24 Test (org.junit.Test)18 BoundDimFilter (org.apache.druid.query.filter.BoundDimFilter)12 SearchQueryDimFilter (org.apache.druid.query.filter.SearchQueryDimFilter)12 ContainsSearchQuerySpec (org.apache.druid.query.search.ContainsSearchQuerySpec)12 InDimFilter (org.apache.druid.query.filter.InDimFilter)11 JavaScriptDimFilter (org.apache.druid.query.filter.JavaScriptDimFilter)11 SelectorDimFilter (org.apache.druid.query.filter.SelectorDimFilter)9 ArrayList (java.util.ArrayList)6 LookupExtractionFn (org.apache.druid.query.lookup.LookupExtractionFn)5 HashMap (java.util.HashMap)4 MapLookupExtractor (org.apache.druid.query.extraction.MapLookupExtractor)4 OrDimFilter (org.apache.druid.query.filter.OrDimFilter)4 InitializedNullHandlingTest (org.apache.druid.testing.InitializedNullHandlingTest)4 DefaultDimensionSpec (org.apache.druid.query.dimension.DefaultDimensionSpec)3 ExtractionFn (org.apache.druid.query.extraction.ExtractionFn)3 JavaScriptExtractionFn (org.apache.druid.query.extraction.JavaScriptExtractionFn)3 DimFilter (org.apache.druid.query.filter.DimFilter)3 LookupExtractor (org.apache.druid.query.lookup.LookupExtractor)3 PeriodGranularity (org.apache.druid.java.util.common.granularity.PeriodGranularity)2