Search in sources :

Example 1 with MapQueryToolChestWarehouse

use of org.apache.druid.query.MapQueryToolChestWarehouse in project druid by druid-io.

the class CachingClusteredClientTestUtils method createWarehouse.

/**
 * Returns a new {@link QueryToolChestWarehouse} for unit tests and a resourceCloser which should be closed at the end
 * of the test.
 */
public static Pair<QueryToolChestWarehouse, Closer> createWarehouse(ObjectMapper objectMapper) {
    final Pair<GroupByQueryRunnerFactory, Closer> factoryCloserPair = GroupByQueryRunnerTest.makeQueryRunnerFactory(new GroupByQueryConfig());
    final GroupByQueryRunnerFactory factory = factoryCloserPair.lhs;
    final Closer resourceCloser = factoryCloserPair.rhs;
    return Pair.of(new MapQueryToolChestWarehouse(ImmutableMap.<Class<? extends Query>, QueryToolChest>builder().put(TimeseriesQuery.class, new TimeseriesQueryQueryToolChest()).put(TopNQuery.class, new TopNQueryQueryToolChest(new TopNQueryConfig())).put(SearchQuery.class, new SearchQueryQueryToolChest(new SearchQueryConfig())).put(GroupByQuery.class, factory.getToolchest()).put(TimeBoundaryQuery.class, new TimeBoundaryQueryQueryToolChest()).build()), resourceCloser);
}
Also used : Closer(org.apache.druid.java.util.common.io.Closer) SearchQuery(org.apache.druid.query.search.SearchQuery) GroupByQueryRunnerFactory(org.apache.druid.query.groupby.GroupByQueryRunnerFactory) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) TopNQuery(org.apache.druid.query.topn.TopNQuery) Query(org.apache.druid.query.Query) TimeBoundaryQuery(org.apache.druid.query.timeboundary.TimeBoundaryQuery) SearchQuery(org.apache.druid.query.search.SearchQuery) GroupByQuery(org.apache.druid.query.groupby.GroupByQuery) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) SearchQueryConfig(org.apache.druid.query.search.SearchQueryConfig) GroupByQueryConfig(org.apache.druid.query.groupby.GroupByQueryConfig) TopNQueryQueryToolChest(org.apache.druid.query.topn.TopNQueryQueryToolChest) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) QueryToolChest(org.apache.druid.query.QueryToolChest) TimeBoundaryQueryQueryToolChest(org.apache.druid.query.timeboundary.TimeBoundaryQueryQueryToolChest) SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) TimeBoundaryQuery(org.apache.druid.query.timeboundary.TimeBoundaryQuery) SearchQueryQueryToolChest(org.apache.druid.query.search.SearchQueryQueryToolChest) TopNQueryConfig(org.apache.druid.query.topn.TopNQueryConfig) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) TimeBoundaryQueryQueryToolChest(org.apache.druid.query.timeboundary.TimeBoundaryQueryQueryToolChest) TopNQueryQueryToolChest(org.apache.druid.query.topn.TopNQueryQueryToolChest)

Example 2 with MapQueryToolChestWarehouse

use of org.apache.druid.query.MapQueryToolChestWarehouse in project druid by druid-io.

the class AsyncQueryForwardingServletTest method testHandleExceptionWithFilterDisabled.

@Test
public void testHandleExceptionWithFilterDisabled() throws Exception {
    String errorMessage = "test exception message";
    ObjectMapper mockMapper = Mockito.mock(ObjectMapper.class);
    HttpServletResponse response = Mockito.mock(HttpServletResponse.class);
    ServletOutputStream outputStream = Mockito.mock(ServletOutputStream.class);
    Mockito.when(response.getOutputStream()).thenReturn(outputStream);
    final AsyncQueryForwardingServlet servlet = new AsyncQueryForwardingServlet(new MapQueryToolChestWarehouse(ImmutableMap.of()), mockMapper, TestHelper.makeSmileMapper(), null, null, null, new NoopServiceEmitter(), new NoopRequestLogger(), new DefaultGenericQueryMetricsFactory(), new AuthenticatorMapper(ImmutableMap.of()), new Properties(), new ServerConfig());
    Exception testException = new IllegalStateException(errorMessage);
    servlet.handleException(response, mockMapper, testException);
    ArgumentCaptor<Exception> captor = ArgumentCaptor.forClass(Exception.class);
    Mockito.verify(mockMapper).writeValue(ArgumentMatchers.eq(outputStream), captor.capture());
    Assert.assertTrue(captor.getValue() instanceof QueryException);
    Assert.assertEquals(QueryInterruptedException.UNKNOWN_EXCEPTION, ((QueryException) captor.getValue()).getErrorCode());
    Assert.assertEquals(errorMessage, captor.getValue().getMessage());
    Assert.assertEquals(IllegalStateException.class.getName(), ((QueryException) captor.getValue()).getErrorClass());
}
Also used : ServletOutputStream(javax.servlet.ServletOutputStream) HttpServletResponse(javax.servlet.http.HttpServletResponse) NoopRequestLogger(org.apache.druid.server.log.NoopRequestLogger) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) Properties(java.util.Properties) QueryException(org.apache.druid.query.QueryException) QueryInterruptedException(org.apache.druid.query.QueryInterruptedException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) AuthenticatorMapper(org.apache.druid.server.security.AuthenticatorMapper) ServerConfig(org.apache.druid.server.initialization.ServerConfig) QueryException(org.apache.druid.query.QueryException) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) DefaultGenericQueryMetricsFactory(org.apache.druid.query.DefaultGenericQueryMetricsFactory) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) BaseJettyTest(org.apache.druid.server.initialization.BaseJettyTest) Test(org.junit.Test)

Example 3 with MapQueryToolChestWarehouse

use of org.apache.druid.query.MapQueryToolChestWarehouse in project druid by druid-io.

the class AsyncQueryForwardingServletTest method testHandleExceptionWithFilterEnabledButMessageMatchAllowedRegex.

@Test
public void testHandleExceptionWithFilterEnabledButMessageMatchAllowedRegex() throws Exception {
    String errorMessage = "test exception message";
    ObjectMapper mockMapper = Mockito.mock(ObjectMapper.class);
    HttpServletResponse response = Mockito.mock(HttpServletResponse.class);
    ServletOutputStream outputStream = Mockito.mock(ServletOutputStream.class);
    Mockito.when(response.getOutputStream()).thenReturn(outputStream);
    final AsyncQueryForwardingServlet servlet = new AsyncQueryForwardingServlet(new MapQueryToolChestWarehouse(ImmutableMap.of()), mockMapper, TestHelper.makeSmileMapper(), null, null, null, new NoopServiceEmitter(), new NoopRequestLogger(), new DefaultGenericQueryMetricsFactory(), new AuthenticatorMapper(ImmutableMap.of()), new Properties(), new ServerConfig() {

        @Override
        public boolean isShowDetailedJettyErrors() {
            return true;
        }

        @Override
        public ErrorResponseTransformStrategy getErrorResponseTransformStrategy() {
            return new AllowedRegexErrorResponseTransformStrategy(ImmutableList.of("test .*"));
        }
    });
    Exception testException = new IllegalStateException(errorMessage);
    servlet.handleException(response, mockMapper, testException);
    ArgumentCaptor<Exception> captor = ArgumentCaptor.forClass(Exception.class);
    Mockito.verify(mockMapper).writeValue(ArgumentMatchers.eq(outputStream), captor.capture());
    Assert.assertTrue(captor.getValue() instanceof QueryException);
    Assert.assertEquals(QueryInterruptedException.UNKNOWN_EXCEPTION, ((QueryException) captor.getValue()).getErrorCode());
    Assert.assertEquals(errorMessage, captor.getValue().getMessage());
    Assert.assertNull(((QueryException) captor.getValue()).getErrorClass());
    Assert.assertNull(((QueryException) captor.getValue()).getHost());
}
Also used : ServletOutputStream(javax.servlet.ServletOutputStream) HttpServletResponse(javax.servlet.http.HttpServletResponse) NoopRequestLogger(org.apache.druid.server.log.NoopRequestLogger) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) Properties(java.util.Properties) QueryException(org.apache.druid.query.QueryException) QueryInterruptedException(org.apache.druid.query.QueryInterruptedException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) AuthenticatorMapper(org.apache.druid.server.security.AuthenticatorMapper) ServerConfig(org.apache.druid.server.initialization.ServerConfig) QueryException(org.apache.druid.query.QueryException) ErrorResponseTransformStrategy(org.apache.druid.common.exception.ErrorResponseTransformStrategy) AllowedRegexErrorResponseTransformStrategy(org.apache.druid.common.exception.AllowedRegexErrorResponseTransformStrategy) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) AllowedRegexErrorResponseTransformStrategy(org.apache.druid.common.exception.AllowedRegexErrorResponseTransformStrategy) DefaultGenericQueryMetricsFactory(org.apache.druid.query.DefaultGenericQueryMetricsFactory) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) BaseJettyTest(org.apache.druid.server.initialization.BaseJettyTest) Test(org.junit.Test)

Example 4 with MapQueryToolChestWarehouse

use of org.apache.druid.query.MapQueryToolChestWarehouse in project druid by druid-io.

the class AsyncQueryForwardingServletTest method verifyServletCallsForQuery.

/**
 * Verifies that the Servlet calls the right methods the right number of times.
 */
private void verifyServletCallsForQuery(Object query, boolean isSql, QueryHostFinder hostFinder, Properties properties) throws Exception {
    final ObjectMapper jsonMapper = TestHelper.makeJsonMapper();
    final HttpServletRequest requestMock = EasyMock.createMock(HttpServletRequest.class);
    final ByteArrayInputStream inputStream = new ByteArrayInputStream(jsonMapper.writeValueAsBytes(query));
    final ServletInputStream servletInputStream = new ServletInputStream() {

        private boolean finished;

        @Override
        public boolean isFinished() {
            return finished;
        }

        @Override
        public boolean isReady() {
            return true;
        }

        @Override
        public void setReadListener(final ReadListener readListener) {
        // do nothing
        }

        @Override
        public int read() {
            final int b = inputStream.read();
            if (b < 0) {
                finished = true;
            }
            return b;
        }
    };
    EasyMock.expect(requestMock.getContentType()).andReturn("application/json").times(2);
    requestMock.setAttribute("org.apache.druid.proxy.objectMapper", jsonMapper);
    EasyMock.expectLastCall();
    EasyMock.expect(requestMock.getRequestURI()).andReturn(isSql ? "/druid/v2/sql" : "/druid/v2/");
    EasyMock.expect(requestMock.getMethod()).andReturn("POST");
    EasyMock.expect(requestMock.getInputStream()).andReturn(servletInputStream);
    requestMock.setAttribute(isSql ? "org.apache.druid.proxy.sqlQuery" : "org.apache.druid.proxy.query", query);
    requestMock.setAttribute("org.apache.druid.proxy.to.host", "1.2.3.4:9999");
    requestMock.setAttribute("org.apache.druid.proxy.to.host.scheme", "http");
    EasyMock.expectLastCall();
    EasyMock.replay(requestMock);
    final AtomicLong didService = new AtomicLong();
    final AsyncQueryForwardingServlet servlet = new AsyncQueryForwardingServlet(new MapQueryToolChestWarehouse(ImmutableMap.of()), jsonMapper, TestHelper.makeSmileMapper(), hostFinder, null, null, new NoopServiceEmitter(), new NoopRequestLogger(), new DefaultGenericQueryMetricsFactory(), new AuthenticatorMapper(ImmutableMap.of()), properties, new ServerConfig()) {

        @Override
        protected void doService(final HttpServletRequest request, final HttpServletResponse response) {
            didService.incrementAndGet();
        }
    };
    servlet.service(requestMock, null);
    // This test is mostly about verifying that the servlet calls the right methods the right number of times.
    EasyMock.verify(hostFinder, requestMock);
    Assert.assertEquals(1, didService.get());
}
Also used : NoopRequestLogger(org.apache.druid.server.log.NoopRequestLogger) HttpServletResponse(javax.servlet.http.HttpServletResponse) NoopServiceEmitter(org.apache.druid.server.metrics.NoopServiceEmitter) ReadListener(javax.servlet.ReadListener) HttpServletRequest(javax.servlet.http.HttpServletRequest) AuthenticatorMapper(org.apache.druid.server.security.AuthenticatorMapper) ServerConfig(org.apache.druid.server.initialization.ServerConfig) AtomicLong(java.util.concurrent.atomic.AtomicLong) ServletInputStream(javax.servlet.ServletInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) DefaultGenericQueryMetricsFactory(org.apache.druid.query.DefaultGenericQueryMetricsFactory) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 5 with MapQueryToolChestWarehouse

use of org.apache.druid.query.MapQueryToolChestWarehouse in project druid by druid-io.

the class MaterializedViewQueryQueryToolChestTest method testDecorateObjectMapper.

@Test
public void testDecorateObjectMapper() throws IOException {
    GroupByQuery realQuery = GroupByQuery.builder().setDataSource(QueryRunnerTestHelper.DATA_SOURCE).setQuerySegmentSpec(QueryRunnerTestHelper.FIRST_TO_THIRD).setDimensions(new DefaultDimensionSpec("quality", "alias")).setAggregatorSpecs(QueryRunnerTestHelper.ROWS_COUNT, new LongSumAggregatorFactory("idx", "index")).setGranularity(QueryRunnerTestHelper.DAY_GRAN).setContext(ImmutableMap.of(GroupByQueryConfig.CTX_KEY_ARRAY_RESULT_ROWS, false)).build();
    QueryToolChest queryToolChest = new MaterializedViewQueryQueryToolChest(new MapQueryToolChestWarehouse(ImmutableMap.<Class<? extends Query>, QueryToolChest>builder().put(GroupByQuery.class, new GroupByQueryQueryToolChest(null)).build()));
    ObjectMapper objectMapper = queryToolChest.decorateObjectMapper(JSON_MAPPER, realQuery);
    List<ResultRow> results = Arrays.asList(GroupByQueryRunnerTestHelper.createExpectedRow(realQuery, "2011-04-01", "alias", "automotive", "rows", 1L, "idx", 135L), GroupByQueryRunnerTestHelper.createExpectedRow(realQuery, "2011-04-01", "alias", "business", "rows", 1L, "idx", 118L));
    List<MapBasedRow> expectedResults = results.stream().map(resultRow -> resultRow.toMapBasedRow(realQuery)).collect(Collectors.toList());
    Assert.assertEquals("decorate-object-mapper", JSON_MAPPER.writerFor(new TypeReference<List<MapBasedRow>>() {
    }).writeValueAsString(expectedResults), objectMapper.writeValueAsString(results));
}
Also used : ResultRow(org.apache.druid.query.groupby.ResultRow) Arrays(java.util.Arrays) MapBasedRow(org.apache.druid.data.input.MapBasedRow) TimeseriesResultValue(org.apache.druid.query.timeseries.TimeseriesResultValue) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) Druids(org.apache.druid.query.Druids) Query(org.apache.druid.query.Query) LongSumAggregatorFactory(org.apache.druid.query.aggregation.LongSumAggregatorFactory) Map(java.util.Map) GroupByQuery(org.apache.druid.query.groupby.GroupByQuery) TypeReference(com.fasterxml.jackson.core.type.TypeReference) MetricManipulationFn(org.apache.druid.query.aggregation.MetricManipulationFn) DateTimes(org.apache.druid.java.util.common.DateTimes) Function(com.google.common.base.Function) ImmutableMap(com.google.common.collect.ImmutableMap) ResultRow(org.apache.druid.query.groupby.ResultRow) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) AggregatorFactory(org.apache.druid.query.aggregation.AggregatorFactory) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) GroupByQueryConfig(org.apache.druid.query.groupby.GroupByQueryConfig) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) QueryToolChest(org.apache.druid.query.QueryToolChest) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) Test(org.junit.Test) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) GroupByQueryRunnerTestHelper(org.apache.druid.query.groupby.GroupByQueryRunnerTestHelper) Result(org.apache.druid.query.Result) List(java.util.List) QueryRunnerTestHelper(org.apache.druid.query.QueryRunnerTestHelper) GroupByQueryQueryToolChest(org.apache.druid.query.groupby.GroupByQueryQueryToolChest) Assert(org.junit.Assert) Query(org.apache.druid.query.Query) GroupByQuery(org.apache.druid.query.groupby.GroupByQuery) TimeseriesQuery(org.apache.druid.query.timeseries.TimeseriesQuery) LongSumAggregatorFactory(org.apache.druid.query.aggregation.LongSumAggregatorFactory) TimeseriesQueryQueryToolChest(org.apache.druid.query.timeseries.TimeseriesQueryQueryToolChest) QueryToolChest(org.apache.druid.query.QueryToolChest) GroupByQueryQueryToolChest(org.apache.druid.query.groupby.GroupByQueryQueryToolChest) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) GroupByQueryQueryToolChest(org.apache.druid.query.groupby.GroupByQueryQueryToolChest) MapBasedRow(org.apache.druid.data.input.MapBasedRow) GroupByQuery(org.apache.druid.query.groupby.GroupByQuery) MapQueryToolChestWarehouse(org.apache.druid.query.MapQueryToolChestWarehouse) List(java.util.List) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) Test(org.junit.Test)

Aggregations

MapQueryToolChestWarehouse (org.apache.druid.query.MapQueryToolChestWarehouse)12 Test (org.junit.Test)10 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)9 IOException (java.io.IOException)8 HttpServletResponse (javax.servlet.http.HttpServletResponse)7 DefaultGenericQueryMetricsFactory (org.apache.druid.query.DefaultGenericQueryMetricsFactory)7 ServerConfig (org.apache.druid.server.initialization.ServerConfig)7 NoopRequestLogger (org.apache.druid.server.log.NoopRequestLogger)7 NoopServiceEmitter (org.apache.druid.server.metrics.NoopServiceEmitter)7 AuthenticatorMapper (org.apache.druid.server.security.AuthenticatorMapper)7 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)6 Properties (java.util.Properties)6 ServletOutputStream (javax.servlet.ServletOutputStream)6 QueryException (org.apache.druid.query.QueryException)6 QueryInterruptedException (org.apache.druid.query.QueryInterruptedException)6 BaseJettyTest (org.apache.druid.server.initialization.BaseJettyTest)6 Query (org.apache.druid.query.Query)5 QueryToolChest (org.apache.druid.query.QueryToolChest)5 GroupByQuery (org.apache.druid.query.groupby.GroupByQuery)5 TimeseriesQuery (org.apache.druid.query.timeseries.TimeseriesQuery)5