Search in sources :

Example 6 with RealtimeInvertedIndex

use of com.linkedin.pinot.core.realtime.impl.invertedIndex.RealtimeInvertedIndex in project pinot by linkedin.

the class RealtimeSegmentImpl method getSortedBitmapIntIteratorsForIntegerColumn.

private IntIterator[] getSortedBitmapIntIteratorsForIntegerColumn(final String columnToSortOn) {
    final RealtimeInvertedIndex index = invertedIndexMap.get(columnToSortOn);
    final MutableDictionaryReader dictionary = dictionaryMap.get(columnToSortOn);
    final IntIterator[] intIterators = new IntIterator[dictionary.length()];
    int[] rawValuesArr = new int[dictionary.length()];
    for (int i = 0; i < dictionary.length(); i++) {
        rawValuesArr[i] = (Integer) dictionary.get(i);
    }
    Arrays.sort(rawValuesArr);
    long start = System.currentTimeMillis();
    LOGGER.info("Column {}, dictionary len : {}, time to sort : {} ", columnToSortOn, dictionary.length(), (System.currentTimeMillis() - start));
    for (int i = 0; i < rawValuesArr.length; i++) {
        intIterators[i] = index.getDocIdSetFor(dictionary.indexOf(rawValuesArr[i])).getIntIterator();
    }
    return intIterators;
}
Also used : IntIterator(org.roaringbitmap.IntIterator) MutableDictionaryReader(com.linkedin.pinot.core.realtime.impl.dictionary.MutableDictionaryReader) RealtimeInvertedIndex(com.linkedin.pinot.core.realtime.impl.invertedIndex.RealtimeInvertedIndex)

Aggregations

RealtimeInvertedIndex (com.linkedin.pinot.core.realtime.impl.invertedIndex.RealtimeInvertedIndex)6 MutableDictionaryReader (com.linkedin.pinot.core.realtime.impl.dictionary.MutableDictionaryReader)5 IntIterator (org.roaringbitmap.IntIterator)5 ArrayList (java.util.ArrayList)4 DataFileReader (com.linkedin.pinot.core.io.reader.DataFileReader)1 IOException (java.io.IOException)1