Search in sources :

Example 11 with Rectangle

use of org.locationtech.spatial4j.shape.Rectangle in project lucene-solr by apache.

the class HeatmapFacetCounter method calcFacets.

/**
   * Calculates spatial 2D facets (aggregated counts) in a grid, sometimes called a heatmap.
   * Facet computation is implemented by navigating the underlying indexed terms efficiently. If you don't know exactly
   * what facetLevel to go to for a given input box but you have some sense of how many cells there should be relative
   * to the size of the shape, then consider using the logic that {@link org.apache.lucene.spatial.prefix.PrefixTreeStrategy}
   * uses when approximating what level to go to when indexing a shape given a distErrPct.
   *
   * @param context the IndexReader's context
   * @param topAcceptDocs a Bits to limit counted docs.  If null, live docs are counted.
   * @param inputShape the shape to gather grid squares for; typically a {@link Rectangle}.
   *                   The <em>actual</em> heatmap area will usually be larger since the cells on the edge that overlap
   *                   are returned. We always return a rectangle of integers even if the inputShape isn't a rectangle
   *                   -- the non-intersecting cells will all be 0.
   *                   If null is given, the entire world is assumed.
   * @param facetLevel the target depth (detail) of cells.
   * @param maxCells the maximum number of cells to return. If the cells exceed this count, an
   */
public static Heatmap calcFacets(PrefixTreeStrategy strategy, IndexReaderContext context, Bits topAcceptDocs, Shape inputShape, final int facetLevel, int maxCells) throws IOException {
    if (maxCells > (MAX_ROWS_OR_COLUMNS * MAX_ROWS_OR_COLUMNS)) {
        throw new IllegalArgumentException("maxCells (" + maxCells + ") should be <= " + MAX_ROWS_OR_COLUMNS);
    }
    if (inputShape == null) {
        inputShape = strategy.getSpatialContext().getWorldBounds();
    }
    final Rectangle inputRect = inputShape.getBoundingBox();
    //First get the rect of the cell at the bottom-left at depth facetLevel
    final SpatialPrefixTree grid = strategy.getGrid();
    final SpatialContext ctx = grid.getSpatialContext();
    final Point cornerPt = ctx.makePoint(inputRect.getMinX(), inputRect.getMinY());
    final CellIterator cellIterator = grid.getTreeCellIterator(cornerPt, facetLevel);
    Cell cornerCell = null;
    while (cellIterator.hasNext()) {
        cornerCell = cellIterator.next();
    }
    assert cornerCell != null && cornerCell.getLevel() == facetLevel : "Cell not at target level: " + cornerCell;
    final Rectangle cornerRect = (Rectangle) cornerCell.getShape();
    assert cornerRect.hasArea();
    //Now calculate the number of columns and rows necessary to cover the inputRect
    //note: we might change this below...
    double heatMinX = cornerRect.getMinX();
    final double cellWidth = cornerRect.getWidth();
    final Rectangle worldRect = ctx.getWorldBounds();
    final int columns = calcRowsOrCols(cellWidth, heatMinX, inputRect.getWidth(), inputRect.getMinX(), worldRect.getWidth());
    final double heatMinY = cornerRect.getMinY();
    final double cellHeight = cornerRect.getHeight();
    final int rows = calcRowsOrCols(cellHeight, heatMinY, inputRect.getHeight(), inputRect.getMinY(), worldRect.getHeight());
    assert rows > 0 && columns > 0;
    if (columns > MAX_ROWS_OR_COLUMNS || rows > MAX_ROWS_OR_COLUMNS || columns * rows > maxCells) {
        throw new IllegalArgumentException("Too many cells (" + columns + " x " + rows + ") for level " + facetLevel + " shape " + inputRect);
    }
    //Create resulting heatmap bounding rectangle & Heatmap object.
    final double halfCellWidth = cellWidth / 2.0;
    // if X world-wraps, use world bounds' range
    if (columns * cellWidth + halfCellWidth > worldRect.getWidth()) {
        heatMinX = worldRect.getMinX();
    }
    double heatMaxX = heatMinX + columns * cellWidth;
    if (Math.abs(heatMaxX - worldRect.getMaxX()) < halfCellWidth) {
        //numeric conditioning issue
        heatMaxX = worldRect.getMaxX();
    } else if (heatMaxX > worldRect.getMaxX()) {
        //wraps dateline (won't happen if !geo)
        heatMaxX = heatMaxX - worldRect.getMaxX() + worldRect.getMinX();
    }
    final double halfCellHeight = cellHeight / 2.0;
    double heatMaxY = heatMinY + rows * cellHeight;
    if (Math.abs(heatMaxY - worldRect.getMaxY()) < halfCellHeight) {
        //numeric conditioning issue
        heatMaxY = worldRect.getMaxY();
    }
    final Heatmap heatmap = new Heatmap(columns, rows, ctx.makeRectangle(heatMinX, heatMaxX, heatMinY, heatMaxY));
    if (topAcceptDocs instanceof Bits.MatchNoBits) {
        // short-circuit
        return heatmap;
    }
    //All ancestor cell counts (of facetLevel) will be captured during facet visiting and applied later. If the data is
    // just points then there won't be any ancestors.
    //Facet count of ancestors covering all of the heatmap:
    // single-element array so it can be accumulated in the inner class
    int[] allCellsAncestorCount = new int[1];
    //All other ancestors:
    Map<Rectangle, Integer> ancestors = new HashMap<>();
    //Now lets count some facets!
    PrefixTreeFacetCounter.compute(strategy, context, topAcceptDocs, inputShape, facetLevel, new PrefixTreeFacetCounter.FacetVisitor() {

        @Override
        public void visit(Cell cell, int count) {
            final double heatMinX = heatmap.region.getMinX();
            final Rectangle rect = (Rectangle) cell.getShape();
            if (cell.getLevel() == facetLevel) {
                //heatmap level; count it directly
                //convert to col & row
                int column;
                if (rect.getMinX() >= heatMinX) {
                    column = (int) Math.round((rect.getMinX() - heatMinX) / cellWidth);
                } else {
                    // due to dateline wrap
                    column = (int) Math.round((rect.getMinX() + 360 - heatMinX) / cellWidth);
                }
                int row = (int) Math.round((rect.getMinY() - heatMinY) / cellHeight);
                // allows adjacent cells to overlap on the seam), so we need to skip them
                if (column < 0 || column >= heatmap.columns || row < 0 || row >= heatmap.rows) {
                    return;
                }
                // increment
                heatmap.counts[column * heatmap.rows + row] += count;
            } else if (rect.relate(heatmap.region) == SpatialRelation.CONTAINS) {
                //containing ancestor
                allCellsAncestorCount[0] += count;
            } else {
                // ancestor
                // note: not particularly efficient (possible put twice, and Integer wrapper); oh well
                Integer existingCount = ancestors.put(rect, count);
                if (existingCount != null) {
                    ancestors.put(rect, count + existingCount);
                }
            }
        }
    });
    // Apply allCellsAncestorCount
    if (allCellsAncestorCount[0] > 0) {
        for (int i = 0; i < heatmap.counts.length; i++) {
            heatmap.counts[i] += allCellsAncestorCount[0];
        }
    }
    // Apply ancestors
    //  note: This approach isn't optimized for a ton of ancestor cells. We'll potentially increment the same cells
    //    multiple times in separate passes if any ancestors overlap. IF this poses a problem, we could optimize it
    //    with additional complication by keeping track of intervals in a sorted tree structure (possible TreeMap/Set)
    //    and iterate them cleverly such that we just make one pass at this stage.
    //output of intersectInterval
    int[] pair = new int[2];
    for (Map.Entry<Rectangle, Integer> entry : ancestors.entrySet()) {
        // from a cell (thus doesn't cross DL)
        Rectangle rect = entry.getKey();
        final int count = entry.getValue();
        //note: we approach this in a way that eliminates int overflow/underflow (think huge cell, tiny heatmap)
        intersectInterval(heatMinY, heatMaxY, cellHeight, rows, rect.getMinY(), rect.getMaxY(), pair);
        final int startRow = pair[0];
        final int endRow = pair[1];
        if (!heatmap.region.getCrossesDateLine()) {
            intersectInterval(heatMinX, heatMaxX, cellWidth, columns, rect.getMinX(), rect.getMaxX(), pair);
            final int startCol = pair[0];
            final int endCol = pair[1];
            incrementRange(heatmap, startCol, endCol, startRow, endRow, count);
        } else {
            // note: the cell rect might intersect 2 disjoint parts of the heatmap, so we do the left & right separately
            final int leftColumns = (int) Math.round((180 - heatMinX) / cellWidth);
            final int rightColumns = heatmap.columns - leftColumns;
            //left half of dateline:
            if (rect.getMaxX() > heatMinX) {
                intersectInterval(heatMinX, 180, cellWidth, leftColumns, rect.getMinX(), rect.getMaxX(), pair);
                final int startCol = pair[0];
                final int endCol = pair[1];
                incrementRange(heatmap, startCol, endCol, startRow, endRow, count);
            }
            //right half of dateline
            if (rect.getMinX() < heatMaxX) {
                intersectInterval(-180, heatMaxX, cellWidth, rightColumns, rect.getMinX(), rect.getMaxX(), pair);
                final int startCol = pair[0] + leftColumns;
                final int endCol = pair[1] + leftColumns;
                incrementRange(heatmap, startCol, endCol, startRow, endRow, count);
            }
        }
    }
    return heatmap;
}
Also used : SpatialContext(org.locationtech.spatial4j.context.SpatialContext) HashMap(java.util.HashMap) Rectangle(org.locationtech.spatial4j.shape.Rectangle) SpatialPrefixTree(org.apache.lucene.spatial.prefix.tree.SpatialPrefixTree) Point(org.locationtech.spatial4j.shape.Point) Point(org.locationtech.spatial4j.shape.Point) CellIterator(org.apache.lucene.spatial.prefix.tree.CellIterator) Cell(org.apache.lucene.spatial.prefix.tree.Cell) HashMap(java.util.HashMap) Map(java.util.Map)

Example 12 with Rectangle

use of org.locationtech.spatial4j.shape.Rectangle in project lucene-solr by apache.

the class BBoxStrategy method makeQuery.

//---------------------------------
// Query Building
//---------------------------------
//  Utility on SpatialStrategy?
//  public Query makeQueryWithValueSource(SpatialArgs args, ValueSource valueSource) {
//    return new CustomScoreQuery(makeQuery(args), new FunctionQuery(valueSource));
//or...
//  return new BooleanQuery.Builder()
//      .add(new FunctionQuery(valueSource), BooleanClause.Occur.MUST)//matches everything and provides score
//      .add(filterQuery, BooleanClause.Occur.FILTER)//filters (score isn't used)
//  .build();
//  }
@Override
public Query makeQuery(SpatialArgs args) {
    Shape shape = args.getShape();
    if (!(shape instanceof Rectangle))
        throw new UnsupportedOperationException("Can only query by Rectangle, not " + shape);
    Rectangle bbox = (Rectangle) shape;
    Query spatial;
    // Useful for understanding Relations:
    // http://edndoc.esri.com/arcsde/9.1/general_topics/understand_spatial_relations.htm
    SpatialOperation op = args.getOperation();
    if (op == SpatialOperation.BBoxIntersects)
        spatial = makeIntersects(bbox);
    else if (op == SpatialOperation.BBoxWithin)
        spatial = makeWithin(bbox);
    else if (op == SpatialOperation.Contains)
        spatial = makeContains(bbox);
    else if (op == SpatialOperation.Intersects)
        spatial = makeIntersects(bbox);
    else if (op == SpatialOperation.IsEqualTo)
        spatial = makeEquals(bbox);
    else if (op == SpatialOperation.IsDisjointTo)
        spatial = makeDisjoint(bbox);
    else if (op == SpatialOperation.IsWithin)
        spatial = makeWithin(bbox);
    else {
        //no Overlaps support yet
        throw new UnsupportedSpatialOperation(op);
    }
    return new ConstantScoreQuery(spatial);
}
Also used : UnsupportedSpatialOperation(org.apache.lucene.spatial.query.UnsupportedSpatialOperation) Shape(org.locationtech.spatial4j.shape.Shape) Query(org.apache.lucene.search.Query) TermQuery(org.apache.lucene.search.TermQuery) BooleanQuery(org.apache.lucene.search.BooleanQuery) ConstantScoreQuery(org.apache.lucene.search.ConstantScoreQuery) Rectangle(org.locationtech.spatial4j.shape.Rectangle) ConstantScoreQuery(org.apache.lucene.search.ConstantScoreQuery) UnsupportedSpatialOperation(org.apache.lucene.spatial.query.UnsupportedSpatialOperation) SpatialOperation(org.apache.lucene.spatial.query.SpatialOperation)

Example 13 with Rectangle

use of org.locationtech.spatial4j.shape.Rectangle in project lucene-solr by apache.

the class BBoxSimilarityValueSource method getValues.

@Override
public FunctionValues getValues(Map context, LeafReaderContext readerContext) throws IOException {
    final FunctionValues shapeValues = bboxValueSource.getValues(context, readerContext);
    return new DoubleDocValues(this) {

        @Override
        public double doubleVal(int doc) throws IOException {
            //? limit to Rect or call getBoundingBox()? latter would encourage bad practice
            final Rectangle rect = (Rectangle) shapeValues.objectVal(doc);
            return rect == null ? 0 : score(rect, null);
        }

        @Override
        public boolean exists(int doc) throws IOException {
            return shapeValues.exists(doc);
        }

        @Override
        public Explanation explain(int doc) throws IOException {
            final Rectangle rect = (Rectangle) shapeValues.objectVal(doc);
            if (rect == null) {
                return Explanation.noMatch("no rect");
            }
            AtomicReference<Explanation> explanation = new AtomicReference<>();
            score(rect, explanation);
            return explanation.get();
        }
    };
}
Also used : Explanation(org.apache.lucene.search.Explanation) DoubleDocValues(org.apache.lucene.queries.function.docvalues.DoubleDocValues) Rectangle(org.locationtech.spatial4j.shape.Rectangle) FunctionValues(org.apache.lucene.queries.function.FunctionValues) AtomicReference(java.util.concurrent.atomic.AtomicReference)

Example 14 with Rectangle

use of org.locationtech.spatial4j.shape.Rectangle in project lucene-solr by apache.

the class RandomSpatialOpFuzzyPrefixTreeTest method randomShapePairRect.

private Shape randomShapePairRect(boolean biasContains) {
    Rectangle shape1 = randomRectangle();
    Rectangle shape2 = randomRectangle();
    return new ShapePair(shape1, shape2, biasContains);
}
Also used : Rectangle(org.locationtech.spatial4j.shape.Rectangle)

Example 15 with Rectangle

use of org.locationtech.spatial4j.shape.Rectangle in project lucene-solr by apache.

the class RandomSpatialOpFuzzyPrefixTreeTest method doTest.

@SuppressWarnings("fallthrough")
private void doTest(final SpatialOperation operation) throws IOException {
    //first show that when there's no data, a query will result in no results
    {
        Query query = strategy.makeQuery(new SpatialArgs(operation, randomRectangle()));
        SearchResults searchResults = executeQuery(query, 1);
        assertEquals(0, searchResults.numFound);
    }
    final boolean biasContains = (operation == SpatialOperation.Contains);
    //Main index loop:
    Map<String, Shape> indexedShapes = new LinkedHashMap<>();
    //grid snapped
    Map<String, Shape> indexedShapesGS = new LinkedHashMap<>();
    final int numIndexedShapes = randomIntBetween(1, 6);
    boolean indexedAtLeastOneShapePair = false;
    final boolean pointsOnly = ((PrefixTreeStrategy) strategy).isPointsOnly();
    for (int i = 0; i < numIndexedShapes; i++) {
        String id = "" + i;
        Shape indexedShape;
        int R = random().nextInt(12);
        if (R == 0) {
            //1 in 12
            indexedShape = null;
        } else if (R == 1 || pointsOnly) {
            //1 in 12
            //just one point
            indexedShape = randomPoint();
        } else if (R <= 4) {
            //3 in 12
            //comprised of more than one shape
            indexedShape = randomShapePairRect(biasContains);
            indexedAtLeastOneShapePair = true;
        } else {
            //just one rect
            indexedShape = randomRectangle();
        }
        indexedShapes.put(id, indexedShape);
        indexedShapesGS.put(id, gridSnap(indexedShape));
        adoc(id, indexedShape);
        if (random().nextInt(10) == 0)
            //intermediate commit, produces extra segments
            commit();
    }
    //delete some documents randomly
    Iterator<String> idIter = indexedShapes.keySet().iterator();
    while (idIter.hasNext()) {
        String id = idIter.next();
        if (random().nextInt(10) == 0) {
            deleteDoc(id);
            idIter.remove();
            indexedShapesGS.remove(id);
        }
    }
    commit();
    //Main query loop:
    final int numQueryShapes = atLeast(20);
    for (int i = 0; i < numQueryShapes; i++) {
        int scanLevel = randomInt(grid.getMaxLevels());
        ((RecursivePrefixTreeStrategy) strategy).setPrefixGridScanLevel(scanLevel);
        final Shape queryShape;
        switch(randomInt(10)) {
            case 0:
                queryShape = randomPoint();
                break;
            case 4:
                //choose an existing indexed shape
                if (!indexedShapes.isEmpty()) {
                    Shape tmp = indexedShapes.values().iterator().next();
                    if (tmp instanceof Point || tmp instanceof Rectangle) {
                        //avoids null and shapePair
                        queryShape = tmp;
                        break;
                    }
                }
            default:
                queryShape = randomRectangle();
        }
        final Shape queryShapeGS = gridSnap(queryShape);
        final boolean opIsDisjoint = operation == SpatialOperation.IsDisjointTo;
        //Generate truth via brute force:
        // We ensure true-positive matches (if the predicate on the raw shapes match
        //  then the search should find those same matches).
        // approximations, false-positive matches
        //true-positives
        Set<String> expectedIds = new LinkedHashSet<>();
        //false-positives (unless disjoint)
        Set<String> secondaryIds = new LinkedHashSet<>();
        for (Map.Entry<String, Shape> entry : indexedShapes.entrySet()) {
            String id = entry.getKey();
            Shape indexedShapeCompare = entry.getValue();
            if (indexedShapeCompare == null)
                continue;
            Shape queryShapeCompare = queryShape;
            if (operation.evaluate(indexedShapeCompare, queryShapeCompare)) {
                expectedIds.add(id);
                if (opIsDisjoint) {
                    //if no longer intersect after buffering them, for disjoint, remember this
                    indexedShapeCompare = indexedShapesGS.get(id);
                    queryShapeCompare = queryShapeGS;
                    if (!operation.evaluate(indexedShapeCompare, queryShapeCompare))
                        secondaryIds.add(id);
                }
            } else if (!opIsDisjoint) {
                //buffer either the indexed or query shape (via gridSnap) and try again
                if (operation == SpatialOperation.Intersects) {
                    indexedShapeCompare = indexedShapesGS.get(id);
                    queryShapeCompare = queryShapeGS;
                //TODO Unfortunately, grid-snapping both can result in intersections that otherwise
                // wouldn't happen when the grids are adjacent. Not a big deal but our test is just a
                // bit more lenient.
                } else if (operation == SpatialOperation.Contains) {
                    indexedShapeCompare = indexedShapesGS.get(id);
                } else if (operation == SpatialOperation.IsWithin) {
                    queryShapeCompare = queryShapeGS;
                }
                if (operation.evaluate(indexedShapeCompare, queryShapeCompare))
                    secondaryIds.add(id);
            }
        }
        //Search and verify results
        SpatialArgs args = new SpatialArgs(operation, queryShape);
        if (queryShape instanceof ShapePair)
            //a hack; we want to be more detailed than gridSnap(queryShape)
            args.setDistErrPct(0.0);
        Query query = strategy.makeQuery(args);
        SearchResults got = executeQuery(query, 100);
        Set<String> remainingExpectedIds = new LinkedHashSet<>(expectedIds);
        for (SearchResult result : got.results) {
            String id = result.getId();
            boolean removed = remainingExpectedIds.remove(id);
            if (!removed && (!opIsDisjoint && !secondaryIds.contains(id))) {
                fail("Shouldn't match", id, indexedShapes, indexedShapesGS, queryShape);
            }
        }
        if (opIsDisjoint)
            remainingExpectedIds.removeAll(secondaryIds);
        if (!remainingExpectedIds.isEmpty()) {
            String id = remainingExpectedIds.iterator().next();
            fail("Should have matched", id, indexedShapes, indexedShapesGS, queryShape);
        }
    }
}
Also used : LinkedHashSet(java.util.LinkedHashSet) SpatialArgs(org.apache.lucene.spatial.query.SpatialArgs) Shape(org.locationtech.spatial4j.shape.Shape) Query(org.apache.lucene.search.Query) Rectangle(org.locationtech.spatial4j.shape.Rectangle) Point(org.locationtech.spatial4j.shape.Point) Point(org.locationtech.spatial4j.shape.Point) LinkedHashMap(java.util.LinkedHashMap) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map)

Aggregations

Rectangle (org.locationtech.spatial4j.shape.Rectangle)36 Point (org.locationtech.spatial4j.shape.Point)10 Test (org.junit.Test)8 Shape (org.locationtech.spatial4j.shape.Shape)7 ArrayList (java.util.ArrayList)5 Query (org.apache.lucene.search.Query)5 Coordinate (com.vividsolutions.jts.geom.Coordinate)3 FunctionValues (org.apache.lucene.queries.function.FunctionValues)3 BooleanQuery (org.apache.lucene.search.BooleanQuery)3 SpatialArgs (org.apache.lucene.spatial.query.SpatialArgs)3 SpatialOperation (org.apache.lucene.spatial.query.SpatialOperation)3 GeoShape (org.apache.lucene.spatial3d.geom.GeoShape)3 SpatialContext (org.locationtech.spatial4j.context.SpatialContext)3 Map (java.util.Map)2 LeafReader (org.apache.lucene.index.LeafReader)2 NumericDocValues (org.apache.lucene.index.NumericDocValues)2 ConstantScoreQuery (org.apache.lucene.search.ConstantScoreQuery)2 TermQuery (org.apache.lucene.search.TermQuery)2 UnsupportedSpatialOperation (org.apache.lucene.spatial.query.UnsupportedSpatialOperation)2 GeoPoint (org.apache.lucene.spatial3d.geom.GeoPoint)2