Search in sources :

Example 16 with IncrementalIndexAdapter

use of io.druid.segment.incremental.IncrementalIndexAdapter in project druid by druid-io.

the class IndexMerger method persist.

public File persist(final IncrementalIndex index, final Interval dataInterval, File outDir, IndexSpec indexSpec, ProgressIndicator progress) throws IOException {
    if (index.isEmpty()) {
        throw new IAE("Trying to persist an empty index!");
    }
    final long firstTimestamp = index.getMinTime().getMillis();
    final long lastTimestamp = index.getMaxTime().getMillis();
    if (!(dataInterval.contains(firstTimestamp) && dataInterval.contains(lastTimestamp))) {
        throw new IAE("interval[%s] does not encapsulate the full range of timestamps[%s, %s]", dataInterval, new DateTime(firstTimestamp), new DateTime(lastTimestamp));
    }
    FileUtils.forceMkdir(outDir);
    log.info("Starting persist for interval[%s], rows[%,d]", dataInterval, index.size());
    return merge(Arrays.<IndexableAdapter>asList(new IncrementalIndexAdapter(dataInterval, index, indexSpec.getBitmapSerdeFactory().getBitmapFactory())), //                     while merging a single iterable
    false, index.getMetricAggs(), outDir, indexSpec, progress);
}
Also used : IncrementalIndexAdapter(io.druid.segment.incremental.IncrementalIndexAdapter) IAE(io.druid.java.util.common.IAE) DateTime(org.joda.time.DateTime)

Aggregations

IncrementalIndexAdapter (io.druid.segment.incremental.IncrementalIndexAdapter)16 Test (org.junit.Test)13 IncrementalIndex (io.druid.segment.incremental.IncrementalIndex)12 OnheapIncrementalIndex (io.druid.segment.incremental.OnheapIncrementalIndex)12 File (java.io.File)11 IncrementalIndexTest (io.druid.segment.data.IncrementalIndexTest)10 LongSumAggregatorFactory (io.druid.query.aggregation.LongSumAggregatorFactory)9 CountAggregatorFactory (io.druid.query.aggregation.CountAggregatorFactory)6 DateTime (org.joda.time.DateTime)6 Interval (org.joda.time.Interval)6 RoaringBitmapFactory (io.druid.collections.bitmap.RoaringBitmapFactory)5 MapBasedInputRow (io.druid.data.input.MapBasedInputRow)5 AggregatorFactory (io.druid.query.aggregation.AggregatorFactory)4 ConciseBitmapFactory (io.druid.collections.bitmap.ConciseBitmapFactory)1 DimensionsSpec (io.druid.data.input.impl.DimensionsSpec)1 IAE (io.druid.java.util.common.IAE)1 Pair (io.druid.java.util.common.Pair)1 Before (org.junit.Before)1