Search in sources :

Example 1 with ExplicitSpatialQuery

use of org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery in project geowave by locationtech.

the class GeoWaveVisibilityIT method testQuery.

public static void testQuery(final DataStore store, final DataStatisticsStore statsStore, final short internalAdapterId, final String[] auths, final boolean spatial, final int expectedResultCount, final int expectedNonNullFieldCount) {
    try (CloseableIterator<SimpleFeature> it = (CloseableIterator) store.query(QueryBuilder.newBuilder().setAuthorizations(auths).constraints(spatial ? new ExplicitSpatialQuery(new GeometryFactory().toGeometry(new Envelope(-1, 1, -1, 1))) : null).build())) {
        int resultCount = 0;
        int nonNullFieldsCount = 0;
        while (it.hasNext()) {
            final SimpleFeature feature = it.next();
            for (int a = 0; a < feature.getAttributeCount(); a++) {
                if (feature.getAttribute(a) != null) {
                    nonNullFieldsCount++;
                }
            }
            resultCount++;
        }
        Assert.assertEquals("Unexpected result count for " + (spatial ? "spatial query" : "full table scan") + " with auths " + Arrays.toString(auths), expectedResultCount, resultCount);
        Assert.assertEquals("Unexpected non-null field count for " + (spatial ? "spatial query" : "full table scan") + " with auths " + Arrays.toString(auths), expectedNonNullFieldCount, nonNullFieldsCount);
    }
    final Long count = (Long) store.aggregate(AggregationQueryBuilder.newBuilder().count(getType().getTypeName()).setAuthorizations(auths).constraints(spatial ? new ExplicitSpatialQuery(new GeometryFactory().toGeometry(new Envelope(-1, 1, -1, 1))) : null).build());
    Assert.assertEquals("Unexpected aggregation result count for " + (spatial ? "spatial query" : "full table scan") + " with auths " + Arrays.toString(auths), expectedResultCount, count.intValue());
    final CountValue countStat = store.aggregateStatistics(StatisticQueryBuilder.newBuilder(CountStatistic.STATS_TYPE).typeName(getType().getTypeName()).authorizations(auths).build());
    assertNotNull(countStat);
    Assert.assertEquals("Unexpected stats result count for " + (spatial ? "spatial query" : "full table scan") + " with auths " + Arrays.toString(auths), expectedResultCount, countStat.getValue().intValue());
}
Also used : CloseableIterator(org.locationtech.geowave.core.store.CloseableIterator) ExplicitSpatialQuery(org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery) GeometryFactory(org.locationtech.jts.geom.GeometryFactory) DifferingVisibilityCountValue(org.locationtech.geowave.core.store.statistics.index.DifferingVisibilityCountStatistic.DifferingVisibilityCountValue) CountValue(org.locationtech.geowave.core.store.statistics.adapter.CountStatistic.CountValue) Envelope(org.locationtech.jts.geom.Envelope) SimpleFeature(org.opengis.feature.simple.SimpleFeature) Point(org.locationtech.jts.geom.Point)

Example 2 with ExplicitSpatialQuery

use of org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery in project geowave by locationtech.

the class GeoWaveSparkIngestIT method testBasicSparkIngest.

@Test
public void testBasicSparkIngest() throws Exception {
    // ingest test points
    TestUtils.testSparkIngest(dataStore, DimensionalityType.SPATIAL, S3URL, GDELT_INPUT_FILES, "gdelt");
    final DataStatisticsStore statsStore = dataStore.createDataStatisticsStore();
    final PersistentAdapterStore adapterStore = dataStore.createAdapterStore();
    final InternalDataAdapter<?>[] adapters = adapterStore.getAdapters();
    for (final InternalDataAdapter<?> internalDataAdapter : adapters) {
        final FeatureDataAdapter adapter = (FeatureDataAdapter) internalDataAdapter.getAdapter();
        // query by the full bounding box, make sure there is more than
        // 0 count and make sure the count matches the number of results
        final BoundingBoxValue bboxValue = InternalStatisticsHelper.getFieldStatistic(statsStore, BoundingBoxStatistic.STATS_TYPE, adapter.getTypeName(), adapter.getFeatureType().getGeometryDescriptor().getLocalName());
        final CountValue count = InternalStatisticsHelper.getDataTypeStatistic(statsStore, CountStatistic.STATS_TYPE, adapter.getTypeName());
        // then query it
        final GeometryFactory factory = new GeometryFactory();
        final Envelope env = new Envelope(bboxValue.getMinX(), bboxValue.getMaxX(), bboxValue.getMinY(), bboxValue.getMaxY());
        final Geometry spatialFilter = factory.toGeometry(env);
        final QueryConstraints query = new ExplicitSpatialQuery(spatialFilter);
        final int resultCount = testQuery(adapter, query);
        assertTrue("'" + adapter.getTypeName() + "' adapter must have at least one element in its statistic", count.getValue() > 0);
        assertEquals("'" + adapter.getTypeName() + "' adapter should have the same results from a spatial query of '" + env + "' as its total count statistic", count.getValue().intValue(), resultCount);
        assertEquals("'" + adapter.getTypeName() + "' adapter entries ingested does not match expected count", new Integer(GDELT_COUNT), new Integer(resultCount));
    }
    // Clean up
    TestUtils.deleteAll(dataStore);
}
Also used : GeometryFactory(org.locationtech.jts.geom.GeometryFactory) QueryConstraints(org.locationtech.geowave.core.store.query.constraints.QueryConstraints) BoundingBoxValue(org.locationtech.geowave.core.geotime.store.statistics.BoundingBoxStatistic.BoundingBoxValue) Envelope(org.locationtech.jts.geom.Envelope) DataStatisticsStore(org.locationtech.geowave.core.store.statistics.DataStatisticsStore) Geometry(org.locationtech.jts.geom.Geometry) PersistentAdapterStore(org.locationtech.geowave.core.store.adapter.PersistentAdapterStore) ExplicitSpatialQuery(org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery) CountValue(org.locationtech.geowave.core.store.statistics.adapter.CountStatistic.CountValue) InternalDataAdapter(org.locationtech.geowave.core.store.adapter.InternalDataAdapter) FeatureDataAdapter(org.locationtech.geowave.adapter.vector.FeatureDataAdapter) Test(org.junit.Test)

Example 3 with ExplicitSpatialQuery

use of org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery in project geowave by locationtech.

the class BasicKafkaIT method testBasicIngestGpx.

@Test
public void testBasicIngestGpx() throws Exception {
    KafkaTestUtils.testKafkaStage(OSM_GPX_INPUT_DIR);
    KafkaTestUtils.testKafkaIngest(dataStorePluginOptions, false, OSM_GPX_INPUT_DIR);
    final DataStatisticsStore statsStore = dataStorePluginOptions.createDataStatisticsStore();
    final PersistentAdapterStore adapterStore = dataStorePluginOptions.createAdapterStore();
    int adapterCount = 0;
    final InternalDataAdapter<?>[] adapters = adapterStore.getAdapters();
    for (final InternalDataAdapter<?> internalDataAdapter : adapters) {
        final FeatureDataAdapter adapter = (FeatureDataAdapter) internalDataAdapter.getAdapter();
        final BoundingBoxValue bboxValue = InternalStatisticsHelper.getFieldStatistic(statsStore, BoundingBoxStatistic.STATS_TYPE, adapter.getTypeName(), adapter.getFeatureType().getGeometryDescriptor().getLocalName());
        final CountValue count = InternalStatisticsHelper.getDataTypeStatistic(statsStore, CountStatistic.STATS_TYPE, adapter.getTypeName());
        // then query it
        final GeometryFactory factory = new GeometryFactory();
        final Envelope env = new Envelope(bboxValue.getMinX(), bboxValue.getMaxX(), bboxValue.getMinY(), bboxValue.getMaxY());
        final Geometry spatialFilter = factory.toGeometry(env);
        final QueryConstraints query = new ExplicitSpatialQuery(spatialFilter);
        final int resultCount = testQuery(adapter, query);
        assertTrue("'" + adapter.getTypeName() + "' adapter must have at least one element in its statistic", count.getValue() > 0);
        assertEquals("'" + adapter.getTypeName() + "' adapter should have the same results from a spatial query of '" + env + "' as its total count statistic", count.getValue().intValue(), resultCount);
        assertEquals("'" + adapter.getTypeName() + "' adapter entries ingested does not match expected count", EXPECTED_COUNT_PER_ADAPTER_ID.get(adapter.getTypeName()), new Integer(resultCount));
        adapterCount++;
    }
    assertTrue("There should be exactly two adapters", (adapterCount == 2));
}
Also used : GeometryFactory(org.locationtech.jts.geom.GeometryFactory) QueryConstraints(org.locationtech.geowave.core.store.query.constraints.QueryConstraints) BoundingBoxValue(org.locationtech.geowave.core.geotime.store.statistics.BoundingBoxStatistic.BoundingBoxValue) Envelope(org.locationtech.jts.geom.Envelope) DataStatisticsStore(org.locationtech.geowave.core.store.statistics.DataStatisticsStore) Geometry(org.locationtech.jts.geom.Geometry) PersistentAdapterStore(org.locationtech.geowave.core.store.adapter.PersistentAdapterStore) ExplicitSpatialQuery(org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery) CountValue(org.locationtech.geowave.core.store.statistics.adapter.CountStatistic.CountValue) InternalDataAdapter(org.locationtech.geowave.core.store.adapter.InternalDataAdapter) FeatureDataAdapter(org.locationtech.geowave.adapter.vector.FeatureDataAdapter) Test(org.junit.Test)

Example 4 with ExplicitSpatialQuery

use of org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery in project geowave by locationtech.

the class DBScanIT method testDBScan.

@Test
public void testDBScan() {
    dataGenerator.setIncludePolygons(false);
    try {
        ingest(dataStorePluginOptions.createDataStore());
    } catch (final IOException e1) {
        e1.printStackTrace();
        TestUtils.deleteAll(dataStorePluginOptions);
        Assert.fail("Unable to ingest data in DBScanIT");
    }
    try {
        runScan(new ExplicitSpatialQuery(dataGenerator.getBoundingRegion()));
    } catch (final Exception e) {
        e.printStackTrace();
        TestUtils.deleteAll(dataStorePluginOptions);
        Assert.fail("Exception during scan of DBScanIT");
    }
    TestUtils.deleteAll(dataStorePluginOptions);
}
Also used : ExplicitSpatialQuery(org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery) IOException(java.io.IOException) FactoryException(org.opengis.referencing.FactoryException) IOException(java.io.IOException) Test(org.junit.Test)

Example 5 with ExplicitSpatialQuery

use of org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery in project geowave by locationtech.

the class GeoWaveBasicURLIngestIT method testBasicURLIngest.

@Test
public void testBasicURLIngest() throws Exception {
    TestUtils.testS3LocalIngest(dataStore, DimensionalityType.SPATIAL, S3URL, GDELT_INPUT_FILE_URL, "gdelt", 4);
    final PersistentAdapterStore adapterStore = dataStore.createAdapterStore();
    final DataStore ds = dataStore.createDataStore();
    final InternalDataAdapter<?>[] adapters = adapterStore.getAdapters();
    for (final InternalDataAdapter<?> internalDataAdapter : adapters) {
        final FeatureDataAdapter adapter = (FeatureDataAdapter) internalDataAdapter.getAdapter();
        // query by the full bounding box, make sure there is more than
        // 0 count and make sure the count matches the number of results
        final BoundingBoxValue bbox = ds.aggregateStatistics(StatisticQueryBuilder.newBuilder(BoundingBoxStatistic.STATS_TYPE).typeName(internalDataAdapter.getTypeName()).fieldName(adapter.getFeatureType().getGeometryDescriptor().getLocalName()).build());
        assertNotNull(bbox);
        final CountValue count = ds.aggregateStatistics(StatisticQueryBuilder.newBuilder(CountStatistic.STATS_TYPE).typeName(internalDataAdapter.getTypeName()).build());
        assertNotNull(count);
        // then query it
        final GeometryFactory factory = new GeometryFactory();
        final Envelope env = new Envelope(bbox.getMinX(), bbox.getMaxX(), bbox.getMinY(), bbox.getMaxY());
        final Geometry spatialFilter = factory.toGeometry(env);
        final QueryConstraints query = new ExplicitSpatialQuery(spatialFilter);
        final int resultCount = testQuery(adapter, query);
        assertTrue("'" + adapter.getTypeName() + "' adapter must have at least one element in its statistic", count.getValue() > 0);
        assertEquals("'" + adapter.getTypeName() + "' adapter should have the same results from a spatial query of '" + env + "' as its total count statistic", count.getValue().intValue(), resultCount);
        assertEquals("'" + adapter.getTypeName() + "' adapter entries ingested does not match expected count", new Integer(GDELT_URL_COUNT), new Integer(resultCount));
    }
    // Clean up
    TestUtils.deleteAll(dataStore);
}
Also used : GeometryFactory(org.locationtech.jts.geom.GeometryFactory) QueryConstraints(org.locationtech.geowave.core.store.query.constraints.QueryConstraints) BoundingBoxValue(org.locationtech.geowave.core.geotime.store.statistics.BoundingBoxStatistic.BoundingBoxValue) Envelope(org.locationtech.jts.geom.Envelope) Geometry(org.locationtech.jts.geom.Geometry) PersistentAdapterStore(org.locationtech.geowave.core.store.adapter.PersistentAdapterStore) ExplicitSpatialQuery(org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery) CountValue(org.locationtech.geowave.core.store.statistics.adapter.CountStatistic.CountValue) DataStore(org.locationtech.geowave.core.store.api.DataStore) InternalDataAdapter(org.locationtech.geowave.core.store.adapter.InternalDataAdapter) FeatureDataAdapter(org.locationtech.geowave.adapter.vector.FeatureDataAdapter) Test(org.junit.Test)

Aggregations

ExplicitSpatialQuery (org.locationtech.geowave.core.geotime.store.query.ExplicitSpatialQuery)16 Test (org.junit.Test)11 QueryConstraints (org.locationtech.geowave.core.store.query.constraints.QueryConstraints)9 Envelope (org.locationtech.jts.geom.Envelope)8 GeometryFactory (org.locationtech.jts.geom.GeometryFactory)8 Geometry (org.locationtech.jts.geom.Geometry)7 FeatureDataAdapter (org.locationtech.geowave.adapter.vector.FeatureDataAdapter)4 CountValue (org.locationtech.geowave.core.store.statistics.adapter.CountStatistic.CountValue)4 BoundingBoxValue (org.locationtech.geowave.core.geotime.store.statistics.BoundingBoxStatistic.BoundingBoxValue)3 InternalDataAdapter (org.locationtech.geowave.core.store.adapter.InternalDataAdapter)3 PersistentAdapterStore (org.locationtech.geowave.core.store.adapter.PersistentAdapterStore)3 SimpleFeature (org.opengis.feature.simple.SimpleFeature)3 IOException (java.io.IOException)2 Date (java.util.Date)2 DataStatisticsStore (org.locationtech.geowave.core.store.statistics.DataStatisticsStore)2 Coordinate (org.locationtech.jts.geom.Coordinate)2 BigDecimal (java.math.BigDecimal)1 Calendar (java.util.Calendar)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1