Search in sources :

Example 1 with IntDoublePair

use of com.linkedin.pinot.common.utils.Pairs.IntDoublePair in project pinot by linkedin.

the class IntDoubleIndexedPriorityQueue method poll.

/**
   * Returns the key+value pair with the max priority (min for minHeap mode)
   * <ul>
   *   <li> key+value pair is removed from the priority queue. </li>
   *   <li> Returns null if the priority queue is empty. </li>
   *   <li> Runtime complexity of O(1). </li>
   * </ul>
   *
   * @return Key+Value pair
   */
public IntDoublePair poll() {
    if (isEmpty()) {
        return null;
    }
    IntDoublePair poll = peek();
    int lastIndex = _values.size() - 1;
    swapValues(0, lastIndex);
    _values.removeDouble(lastIndex);
    _keyToIndexMap.remove(_indexToKeyMap.get(lastIndex));
    _indexToKeyMap.remove(lastIndex);
    if (!_values.isEmpty()) {
        siftDown(0);
    }
    return poll;
}
Also used : IntDoublePair(com.linkedin.pinot.common.utils.Pairs.IntDoublePair)

Example 2 with IntDoublePair

use of com.linkedin.pinot.common.utils.Pairs.IntDoublePair in project pinot by linkedin.

the class DoubleGroupByResultHolderTest method testTrimResults.

/**
   * Helper method for testing trim results.
   * Sets a max size on the result holder and adds more values than the max size of the result holder.
   * Then asserts that the right results survive after trimming.
   *
   * @param minOrder
   */
void testTrimResults(final boolean minOrder) {
    GroupByResultHolder resultHolder = new DoubleGroupByResultHolder(INITIAL_CAPACITY, MAX_CAPACITY, INITIAL_CAPACITY, DEFAULT_VALUE, minOrder);
    List<IntDoublePair> expected = new ArrayList<>(MAX_CAPACITY);
    for (int i = 0; i < INITIAL_CAPACITY; i++) {
        resultHolder.setValueForKey(i, _expected[i]);
        expected.add(new IntDoublePair(i, _expected[i]));
    }
    // This will trigger the transition from array based to map based storage within the result holder.
    resultHolder.ensureCapacity(MAX_CAPACITY);
    for (int i = INITIAL_CAPACITY; i < MAX_CAPACITY; i++) {
        resultHolder.setValueForKey(i, _expected[i]);
        expected.add(new IntDoublePair(i, _expected[i]));
    }
    // Trim the results.
    resultHolder.trimResults();
    // Sort the input
    Collections.sort(expected, new IntDoubleComparator(!minOrder));
    // Ensure that all the correct group keys survive after trimming.
    for (int i = 0; i < INITIAL_CAPACITY; i++) {
        IntDoublePair pair = expected.get(i);
        Assert.assertEquals(resultHolder.getDoubleResult(pair.getIntValue()), pair.getDoubleValue());
    }
}
Also used : IntDoublePair(com.linkedin.pinot.common.utils.Pairs.IntDoublePair) GroupByResultHolder(com.linkedin.pinot.core.query.aggregation.groupby.GroupByResultHolder) DoubleGroupByResultHolder(com.linkedin.pinot.core.query.aggregation.groupby.DoubleGroupByResultHolder) DoubleGroupByResultHolder(com.linkedin.pinot.core.query.aggregation.groupby.DoubleGroupByResultHolder) ArrayList(java.util.ArrayList) IntDoubleComparator(com.linkedin.pinot.common.utils.Pairs.IntDoubleComparator)

Example 3 with IntDoublePair

use of com.linkedin.pinot.common.utils.Pairs.IntDoublePair in project pinot by linkedin.

the class DoubleGroupByResultHolder method trimResults.

/**
   * {@inheritDoc}
   *
   * Keys with 'lowest' values (as per the sort order) are trimmed away to reduce the size to _trimSize.
   *
   * @return Array of keys that were trimmed out.
   */
@Override
public int[] trimResults() {
    if (_storageMode == StorageMode.ARRAY_STORAGE) {
        // Still in array mode, trimming has not kicked in yet.
        return EMPTY_ARRAY;
    }
    int numKeysToRemove = _resultMap.size() - _trimSize;
    int[] removedGroupKeys = new int[numKeysToRemove];
    for (int i = 0; i < numKeysToRemove; i++) {
        IntDoublePair pair = _priorityQueue.poll();
        int groupKey = pair.getIntValue();
        _resultMap.remove(groupKey);
        removedGroupKeys[i] = groupKey;
    }
    return removedGroupKeys;
}
Also used : IntDoublePair(com.linkedin.pinot.common.utils.Pairs.IntDoublePair)

Aggregations

IntDoublePair (com.linkedin.pinot.common.utils.Pairs.IntDoublePair)3 IntDoubleComparator (com.linkedin.pinot.common.utils.Pairs.IntDoubleComparator)1 DoubleGroupByResultHolder (com.linkedin.pinot.core.query.aggregation.groupby.DoubleGroupByResultHolder)1 GroupByResultHolder (com.linkedin.pinot.core.query.aggregation.groupby.GroupByResultHolder)1 ArrayList (java.util.ArrayList)1