Search in sources :

Example 1 with ISO_TIME_FORMAT

use of org.apache.hadoop.hive.druid.serde.DruidSerDeUtils.ISO_TIME_FORMAT in project hive by apache.

the class DruidGroupByQueryRecordReader method initDimensionTypes.

private void initDimensionTypes() throws IOException {
    // @TODO move this out of here to org.apache.hadoop.hive.druid.serde.DruidSerDe
    List<DimensionSpec> dimensionSpecList = ((GroupByQuery) query).getDimensions();
    List<DimensionSpec> extractionDimensionSpecList = dimensionSpecList.stream().filter(dimensionSpecs -> dimensionSpecs instanceof ExtractionDimensionSpec).collect(Collectors.toList());
    extractionDimensionSpecList.stream().forEach(dimensionSpec -> {
        ExtractionDimensionSpec extractionDimensionSpec = (ExtractionDimensionSpec) dimensionSpec;
        if (extractionDimensionSpec.getExtractionFn() instanceof TimeFormatExtractionFn) {
            final TimeFormatExtractionFn timeFormatExtractionFn = (TimeFormatExtractionFn) extractionDimensionSpec.getExtractionFn();
            if (timeFormatExtractionFn == null || timeFormatExtractionFn.getFormat().equals(ISO_TIME_FORMAT)) {
                timeExtractionFields.add(extractionDimensionSpec.getOutputName());
            } else {
                intFormattedTimeExtractionFields.add(extractionDimensionSpec.getOutputName());
            }
        }
    });
}
Also used : ISODateTimeFormat(org.joda.time.format.ISODateTimeFormat) NullWritable(org.apache.hadoop.io.NullWritable) InputSplit(org.apache.hadoop.mapreduce.InputSplit) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DimensionSpec(io.druid.query.dimension.DimensionSpec) IOException(java.io.IOException) GroupByQuery(io.druid.query.groupby.GroupByQuery) Collectors(java.util.stream.Collectors) Maps(com.google.common.collect.Maps) List(java.util.List) Lists(com.google.common.collect.Lists) Row(io.druid.data.input.Row) MapBasedRow(io.druid.data.input.MapBasedRow) Map(java.util.Map) Configuration(org.apache.hadoop.conf.Configuration) ISO_TIME_FORMAT(org.apache.hadoop.hive.druid.serde.DruidSerDeUtils.ISO_TIME_FORMAT) ExtractionDimensionSpec(io.druid.query.dimension.ExtractionDimensionSpec) JavaType(com.fasterxml.jackson.databind.JavaType) DruidStorageHandlerUtils(org.apache.hadoop.hive.druid.DruidStorageHandlerUtils) TypeReference(com.fasterxml.jackson.core.type.TypeReference) TimeFormatExtractionFn(io.druid.query.extraction.TimeFormatExtractionFn) HttpClient(com.metamx.http.client.HttpClient) DimensionSpec(io.druid.query.dimension.DimensionSpec) ExtractionDimensionSpec(io.druid.query.dimension.ExtractionDimensionSpec) TimeFormatExtractionFn(io.druid.query.extraction.TimeFormatExtractionFn) GroupByQuery(io.druid.query.groupby.GroupByQuery) ExtractionDimensionSpec(io.druid.query.dimension.ExtractionDimensionSpec)

Aggregations

TypeReference (com.fasterxml.jackson.core.type.TypeReference)1 JavaType (com.fasterxml.jackson.databind.JavaType)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 Lists (com.google.common.collect.Lists)1 Maps (com.google.common.collect.Maps)1 HttpClient (com.metamx.http.client.HttpClient)1 MapBasedRow (io.druid.data.input.MapBasedRow)1 Row (io.druid.data.input.Row)1 DimensionSpec (io.druid.query.dimension.DimensionSpec)1 ExtractionDimensionSpec (io.druid.query.dimension.ExtractionDimensionSpec)1 TimeFormatExtractionFn (io.druid.query.extraction.TimeFormatExtractionFn)1 GroupByQuery (io.druid.query.groupby.GroupByQuery)1 IOException (java.io.IOException)1 List (java.util.List)1 Map (java.util.Map)1 Collectors (java.util.stream.Collectors)1 Configuration (org.apache.hadoop.conf.Configuration)1 DruidStorageHandlerUtils (org.apache.hadoop.hive.druid.DruidStorageHandlerUtils)1 ISO_TIME_FORMAT (org.apache.hadoop.hive.druid.serde.DruidSerDeUtils.ISO_TIME_FORMAT)1 NullWritable (org.apache.hadoop.io.NullWritable)1