Search in sources :

Example 1 with SearchQueryQueryToolChest

use of org.apache.druid.query.search.SearchQueryQueryToolChest in project druid by druid-io.

the class CachingClusteredClientTest method testSearchCachingRenamedOutput.

@Test
public void testSearchCachingRenamedOutput() {
    final Druids.SearchQueryBuilder builder = Druids.newSearchQueryBuilder().dataSource(DATA_SOURCE).filters(DIM_FILTER).granularity(GRANULARITY).limit(1000).intervals(SEG_SPEC).dimensions(Collections.singletonList(TOP_DIM)).query("how").context(CONTEXT);
    testQueryCaching(getDefaultQueryRunner(), builder.randomQueryId().build(), Intervals.of("2011-01-01/2011-01-02"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-01"), "how", 1, "howdy", 2, "howwwwww", 3, "howwy", 4), Intervals.of("2011-01-02/2011-01-03"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-02"), "how1", 1, "howdy1", 2, "howwwwww1", 3, "howwy1", 4), Intervals.of("2011-01-05/2011-01-10"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-05"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4), Intervals.of("2011-01-05/2011-01-10"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-05T01"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06T01"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07T01"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08T01"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09T01"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4));
    QueryRunner runner = new FinalizeResultsQueryRunner(getDefaultQueryRunner(), new SearchQueryQueryToolChest(new SearchQueryConfig()));
    ResponseContext context = initializeResponseContext();
    TestHelper.assertExpectedResults(makeSearchResults(TOP_DIM, DateTimes.of("2011-01-01"), "how", 1, "howdy", 2, "howwwwww", 3, "howwy", 4, DateTimes.of("2011-01-02"), "how1", 1, "howdy1", 2, "howwwwww1", 3, "howwy1", 4, DateTimes.of("2011-01-05"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-05T01"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-06T01"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-07T01"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-08T01"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4, DateTimes.of("2011-01-09T01"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4), runner.run(QueryPlus.wrap(builder.randomQueryId().intervals("2011-01-01/2011-01-10").build()), context));
    SearchQuery query = builder.intervals("2011-01-01/2011-01-10").dimensions(new DefaultDimensionSpec(TOP_DIM, "new_dim")).randomQueryId().build();
    TestHelper.assertExpectedResults(makeSearchResults("new_dim", DateTimes.of("2011-01-01"), "how", 1, "howdy", 2, "howwwwww", 3, "howwy", 4, DateTimes.of("2011-01-02"), "how1", 1, "howdy1", 2, "howwwwww1", 3, "howwy1", 4, DateTimes.of("2011-01-05"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-05T01"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-06T01"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-07T01"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-08T01"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4, DateTimes.of("2011-01-09T01"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4), runner.run(QueryPlus.wrap(query), context));
}
Also used : SearchQuery(org.apache.druid.query.search.SearchQuery) SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) FinalizeResultsQueryRunner(org.apache.druid.query.FinalizeResultsQueryRunner) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) Druids(org.apache.druid.query.Druids) ResponseContext(org.apache.druid.query.context.ResponseContext) FinalizeResultsQueryRunner(org.apache.druid.query.FinalizeResultsQueryRunner) QueryRunner(org.apache.druid.query.QueryRunner) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) Test(org.junit.Test)

Example 2 with SearchQueryQueryToolChest

use of org.apache.druid.query.search.SearchQueryQueryToolChest in project druid by druid-io.

the class CachingClusteredClientTestUtils method createWarehouse.

/**
 * Returns a new {@link QueryToolChestWarehouse} for unit tests and a resourceCloser which should be closed at the end
 * of the test.
 */
public static Pair<QueryToolChestWarehouse, Closer> createWarehouse(ObjectMapper objectMapper) {
    final Pair<GroupByQueryRunnerFactory, Closer> factoryCloserPair = GroupByQueryRunnerTest.makeQueryRunnerFactory(new GroupByQueryConfig());
    final GroupByQueryRunnerFactory factory = factoryCloserPair.lhs;
    final Closer resourceCloser = factoryCloserPair.rhs;
    return Pair.of(new MapQueryToolChestWarehouse(ImmutableMap.<Class<? extends Query>, QueryToolChest>builder().put(TimeseriesQuery.class, new TimeseriesQueryQueryToolChest()).put(TopNQuery.class, new TopNQueryQueryToolChest(new TopNQueryConfig())).put(SearchQuery.class, new SearchQueryQueryToolChest(new SearchQueryConfig())).put(GroupByQuery.class, factory.getToolchest()).put(TimeBoundaryQuery.class, new TimeBoundaryQueryQueryToolChest()).build()), resourceCloser);
}
Also used : Closer(org.apache.druid.java.util.common.io.Closer) SearchQuery(org.apache.druid.query.search.SearchQuery) GroupByQueryRunnerFactory(org.apache.druid.query.groupby.GroupByQueryRunnerFactory) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) TopNQuery(org.apache.druid.query.topn.TopNQuery) Query(org.apache.druid.query.Query) TimeBoundaryQuery(org.apache.druid.query.timeboundary.TimeBoundaryQuery) SearchQuery(org.apache.druid.query.search.SearchQuery) GroupByQuery(org.apache.druid.query.groupby.GroupByQuery) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) GroupByQueryConfig(org.apache.druid.query.groupby.GroupByQueryConfig) TopNQueryQueryToolChest(org.apache.druid.query.topn.TopNQueryQueryToolChest) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) QueryToolChest(org.apache.druid.query.QueryToolChest) TimeBoundaryQueryQueryToolChest(org.apache.druid.query.timeboundary.TimeBoundaryQueryQueryToolChest) SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) TimeBoundaryQuery(org.apache.druid.query.timeboundary.TimeBoundaryQuery) SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) TopNQueryConfig(org.apache.druid.query.topn.TopNQueryConfig) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) TimeBoundaryQueryQueryToolChest(org.apache.druid.query.timeboundary.TimeBoundaryQueryQueryToolChest) TopNQueryQueryToolChest(org.apache.druid.query.topn.TopNQueryQueryToolChest)

Example 3 with SearchQueryQueryToolChest

use of org.apache.druid.query.search.SearchQueryQueryToolChest in project druid by druid-io.

the class TestQueryRunners method makeSearchQueryRunner.

public static <T> QueryRunner<T> makeSearchQueryRunner(Segment adapter) {
    final SearchQueryConfig config = new SearchQueryConfig();
    QueryRunnerFactory factory = new SearchQueryRunnerFactory(new SearchStrategySelector(Suppliers.ofInstance(config)), new SearchQueryQueryToolChest(config), QueryRunnerTestHelper.NOOP_QUERYWATCHER);
    return new FinalizeResultsQueryRunner<T>(factory.createRunner(adapter), factory.getToolchest());
}
Also used : SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) TimeBoundaryQueryRunnerFactory(org.apache.druid.query.timeboundary.TimeBoundaryQueryRunnerFactory) SearchQueryRunnerFactory(org.apache.druid.query.search.SearchQueryRunnerFactory) TopNQueryRunnerFactory(org.apache.druid.query.topn.TopNQueryRunnerFactory) TimeseriesQueryRunnerFactory(org.apache.druid.query.timeseries.TimeseriesQueryRunnerFactory) SearchQueryRunnerFactory(org.apache.druid.query.search.SearchQueryRunnerFactory) SearchStrategySelector(org.apache.druid.query.search.SearchStrategySelector)

Example 4 with SearchQueryQueryToolChest

use of org.apache.druid.query.search.SearchQueryQueryToolChest in project druid by druid-io.

the class SearchBenchmark method setup.

/**
 * Setup everything common for benchmarking both the incremental-index and the queriable-index.
 */
@Setup
public void setup() {
    log.info("SETUP CALLED AT " + +System.currentTimeMillis());
    ComplexMetrics.registerSerde("hyperUnique", new HyperUniquesSerde());
    setupQueries();
    String[] schemaQuery = schemaAndQuery.split("\\.");
    String schemaName = schemaQuery[0];
    String queryName = schemaQuery[1];
    schemaInfo = GeneratorBasicSchemas.SCHEMA_MAP.get(schemaName);
    queryBuilder = SCHEMA_QUERY_MAP.get(schemaName).get(queryName);
    queryBuilder.limit(limit);
    query = queryBuilder.build();
    generator = new DataGenerator(schemaInfo.getColumnSchemas(), RNG_SEED, schemaInfo.getDataInterval(), rowsPerSegment);
    final SearchQueryConfig config = new SearchQueryConfig().withOverrides(query);
    factory = new SearchQueryRunnerFactory(new SearchStrategySelector(Suppliers.ofInstance(config)), new SearchQueryQueryToolChest(config), QueryBenchmarkUtil.NOOP_QUERYWATCHER);
}
Also used : SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) DataGenerator(org.apache.druid.segment.generator.DataGenerator) HyperUniquesSerde(org.apache.druid.query.aggregation.hyperloglog.HyperUniquesSerde) SearchQueryRunnerFactory(org.apache.druid.query.search.SearchQueryRunnerFactory) SearchStrategySelector(org.apache.druid.query.search.SearchStrategySelector) Setup(org.openjdk.jmh.annotations.Setup)

Example 5 with SearchQueryQueryToolChest

use of org.apache.druid.query.search.SearchQueryQueryToolChest in project druid by druid-io.

the class CachingClusteredClientTest method testSearchCaching.

@Test
public void testSearchCaching() {
    final Druids.SearchQueryBuilder builder = Druids.newSearchQueryBuilder().dataSource(DATA_SOURCE).filters(DIM_FILTER).granularity(GRANULARITY).limit(1000).intervals(SEG_SPEC).dimensions(Collections.singletonList(TOP_DIM)).query("how").context(CONTEXT);
    testQueryCaching(getDefaultQueryRunner(), builder.randomQueryId().build(), Intervals.of("2011-01-01/2011-01-02"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-01"), "how", 1, "howdy", 2, "howwwwww", 3, "howwy", 4), Intervals.of("2011-01-02/2011-01-03"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-02"), "how1", 1, "howdy1", 2, "howwwwww1", 3, "howwy1", 4), Intervals.of("2011-01-05/2011-01-10"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-05"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4), Intervals.of("2011-01-05/2011-01-10"), makeSearchResults(TOP_DIM, DateTimes.of("2011-01-05T01"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06T01"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07T01"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08T01"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09T01"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4));
    QueryRunner runner = new FinalizeResultsQueryRunner(getDefaultQueryRunner(), new SearchQueryQueryToolChest(new SearchQueryConfig()));
    TestHelper.assertExpectedResults(makeSearchResults(TOP_DIM, DateTimes.of("2011-01-01"), "how", 1, "howdy", 2, "howwwwww", 3, "howwy", 4, DateTimes.of("2011-01-02"), "how1", 1, "howdy1", 2, "howwwwww1", 3, "howwy1", 4, DateTimes.of("2011-01-05"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-05T01"), "how2", 1, "howdy2", 2, "howwwwww2", 3, "howww2", 4, DateTimes.of("2011-01-06"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-06T01"), "how3", 1, "howdy3", 2, "howwwwww3", 3, "howww3", 4, DateTimes.of("2011-01-07"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-07T01"), "how4", 1, "howdy4", 2, "howwwwww4", 3, "howww4", 4, DateTimes.of("2011-01-08"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-08T01"), "how5", 1, "howdy5", 2, "howwwwww5", 3, "howww5", 4, DateTimes.of("2011-01-09"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4, DateTimes.of("2011-01-09T01"), "how6", 1, "howdy6", 2, "howwwwww6", 3, "howww6", 4), runner.run(QueryPlus.wrap(builder.randomQueryId().intervals("2011-01-01/2011-01-10").build())));
}
Also used : SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) FinalizeResultsQueryRunner(org.apache.druid.query.FinalizeResultsQueryRunner) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) Druids(org.apache.druid.query.Druids) FinalizeResultsQueryRunner(org.apache.druid.query.FinalizeResultsQueryRunner) QueryRunner(org.apache.druid.query.QueryRunner) Test(org.junit.Test)

Aggregations

SearchQueryConfig (org.apache.druid.query.search.SearchQueryConfig)5 SearchQueryQueryToolChest (org.apache.druid.query.search.SearchQueryQueryToolChest)5 Druids (org.apache.druid.query.Druids)2 FinalizeResultsQueryRunner (org.apache.druid.query.FinalizeResultsQueryRunner)2 QueryRunner (org.apache.druid.query.QueryRunner)2 SearchQuery (org.apache.druid.query.search.SearchQuery)2 SearchQueryRunnerFactory (org.apache.druid.query.search.SearchQueryRunnerFactory)2 SearchStrategySelector (org.apache.druid.query.search.SearchStrategySelector)2 Test (org.junit.Test)2 Closer (org.apache.druid.java.util.common.io.Closer)1 MapQueryToolChestWarehouse (org.apache.druid.query.MapQueryToolChestWarehouse)1 Query (org.apache.druid.query.Query)1 QueryToolChest (org.apache.druid.query.QueryToolChest)1 HyperUniquesSerde (org.apache.druid.query.aggregation.hyperloglog.HyperUniquesSerde)1 ResponseContext (org.apache.druid.query.context.ResponseContext)1 DefaultDimensionSpec (org.apache.druid.query.dimension.DefaultDimensionSpec)1 GroupByQuery (org.apache.druid.query.groupby.GroupByQuery)1 GroupByQueryConfig (org.apache.druid.query.groupby.GroupByQueryConfig)1 GroupByQueryRunnerFactory (org.apache.druid.query.groupby.GroupByQueryRunnerFactory)1 TimeBoundaryQuery (org.apache.druid.query.timeboundary.TimeBoundaryQuery)1