Search in sources :

Example 11 with StringTuple

use of org.apache.druid.data.input.StringTuple in project druid by druid-io.

the class RangePartitionMultiPhaseParallelIndexingTest method assertRangePartitions.

private void assertRangePartitions(Set<DataSegment> publishedSegments) throws IOException {
    Multimap<Interval, DataSegment> intervalToSegments = ArrayListMultimap.create();
    publishedSegments.forEach(s -> intervalToSegments.put(s.getInterval(), s));
    Set<Interval> publishedIntervals = intervalToSegments.keySet();
    assertHasExpectedIntervals(publishedIntervals);
    File tempSegmentDir = temporaryFolder.newFolder();
    intervalToSegments.asMap().forEach((interval, segments) -> {
        assertNumPartition(segments);
        List<StringTuple> allValues = new ArrayList<>(NUM_ROW);
        for (DataSegment segment : segments) {
            List<StringTuple> values = getColumnValues(segment, tempSegmentDir);
            assertValuesInRange(values, segment);
            allValues.addAll(values);
        }
        assertIntervalHasAllExpectedValues(interval, allValues);
    });
}
Also used : ArrayList(java.util.ArrayList) DataSegment(org.apache.druid.timeline.DataSegment) File(java.io.File) StringTuple(org.apache.druid.data.input.StringTuple) Interval(org.joda.time.Interval)

Aggregations

StringTuple (org.apache.druid.data.input.StringTuple)11 HashMap (java.util.HashMap)5 Test (org.junit.Test)5 RangeSet (com.google.common.collect.RangeSet)4 TreeRangeSet (com.google.common.collect.TreeRangeSet)4 Interval (org.joda.time.Interval)2 File (java.io.File)1 ArrayList (java.util.ArrayList)1 InputRow (org.apache.druid.data.input.InputRow)1 StringDistribution (org.apache.druid.indexing.common.task.batch.parallel.distribution.StringDistribution)1 StringSketch (org.apache.druid.indexing.common.task.batch.parallel.distribution.StringSketch)1 DataSegment (org.apache.druid.timeline.DataSegment)1 DimensionRangeShardSpec (org.apache.druid.timeline.partition.DimensionRangeShardSpec)1 PartitionBoundaries (org.apache.druid.timeline.partition.PartitionBoundaries)1