Search in sources :

Example 1 with ClientVisibilityFilter

use of org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter in project geowave by locationtech.

the class CassandraOperations method createReader.

@Override
public RowReader<GeoWaveRow> createReader(final DataIndexReaderParams readerParams) {
    final byte[][] dataIds;
    Iterator<GeoWaveRow> iterator;
    if (readerParams.getDataIds() == null) {
        if ((readerParams.getStartInclusiveDataId() != null) || (readerParams.getEndInclusiveDataId() != null)) {
            final List<byte[]> intermediaries = new ArrayList<>();
            ByteArrayUtils.addAllIntermediaryByteArrays(intermediaries, new ByteArrayRange(readerParams.getStartInclusiveDataId(), readerParams.getEndInclusiveDataId()));
            dataIds = intermediaries.toArray(new byte[0][]);
            iterator = getRows(dataIds, readerParams.getAdapterId());
        } else {
            iterator = getRows(readerParams.getAdapterId());
        }
    } else {
        dataIds = readerParams.getDataIds();
        iterator = getRows(dataIds, readerParams.getAdapterId());
    }
    if (options.isVisibilityEnabled()) {
        Stream<GeoWaveRow> stream = Streams.stream(iterator);
        final Set<String> authorizations = Sets.newHashSet(readerParams.getAdditionalAuthorizations());
        stream = stream.filter(new ClientVisibilityFilter(authorizations));
        iterator = stream.iterator();
    }
    return new RowReaderWrapper<>(new CloseableIterator.Wrapper(iterator));
}
Also used : GeoWaveRow(org.locationtech.geowave.core.store.entities.GeoWaveRow) CloseableIterator(org.locationtech.geowave.core.store.CloseableIterator) RowReaderWrapper(org.locationtech.geowave.core.store.operations.RowReaderWrapper) ArrayList(java.util.ArrayList) ByteArrayRange(org.locationtech.geowave.core.index.ByteArrayRange) ClientVisibilityFilter(org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)

Example 2 with ClientVisibilityFilter

use of org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter in project geowave by locationtech.

the class CassandraReader method initScanner.

protected void initScanner() {
    final Collection<SinglePartitionQueryRanges> ranges = readerParams.getQueryRanges().getPartitionQueryRanges();
    if ((ranges != null) && !ranges.isEmpty()) {
        iterator = operations.getBatchedRangeRead(readerParams.getIndex().getName(), readerParams.getAdapterIds(), ranges, DataStoreUtils.isMergingIteratorRequired(readerParams, visibilityEnabled), rowTransformer, new ClientVisibilityFilter(Sets.newHashSet(readerParams.getAdditionalAuthorizations()))).results();
    } else {
        // TODO figure out the query select by adapter IDs here
        final Select select = operations.getSelect(readerParams.getIndex().getName());
        CloseableIterator<CassandraRow> results = operations.executeQuery(select.build());
        if ((readerParams.getAdapterIds() != null) && (readerParams.getAdapterIds().length > 0)) {
            // TODO because we aren't filtering server-side by adapter ID,
            // we will need to filter here on the client
            results = new CloseableIteratorWrapper<>(results, Iterators.filter(results, input -> ArrayUtils.contains(readerParams.getAdapterIds(), input.getAdapterId())));
        }
        iterator = wrapResults(results, readerParams);
    }
}
Also used : SinglePartitionQueryRanges(org.locationtech.geowave.core.index.SinglePartitionQueryRanges) Select(com.datastax.oss.driver.api.querybuilder.select.Select) CassandraRow(org.locationtech.geowave.datastore.cassandra.CassandraRow) ClientVisibilityFilter(org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)

Example 3 with ClientVisibilityFilter

use of org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter in project geowave by locationtech.

the class CassandraReader method wrapResults.

@SuppressWarnings("unchecked")
private CloseableIterator<T> wrapResults(final CloseableIterator<CassandraRow> results, final RangeReaderParams<T> readerParams) {
    final Set<String> authorizations = Sets.newHashSet(readerParams.getAdditionalAuthorizations());
    final Iterator<GeoWaveRow> iterator = (Iterator) Streams.stream(results).filter(new ClientVisibilityFilter(authorizations)).iterator();
    return new CloseableIteratorWrapper<>(results, rowTransformer.apply(DataStoreUtils.isMergingIteratorRequired(readerParams, visibilityEnabled) ? new GeoWaveRowMergingIterator(iterator) : iterator));
}
Also used : GeoWaveRow(org.locationtech.geowave.core.store.entities.GeoWaveRow) Iterator(java.util.Iterator) GeoWaveRowMergingIterator(org.locationtech.geowave.core.store.entities.GeoWaveRowMergingIterator) CloseableIterator(org.locationtech.geowave.core.store.CloseableIterator) GeoWaveRowMergingIterator(org.locationtech.geowave.core.store.entities.GeoWaveRowMergingIterator) CloseableIteratorWrapper(org.locationtech.geowave.core.store.CloseableIteratorWrapper) ClientVisibilityFilter(org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)

Example 4 with ClientVisibilityFilter

use of org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter in project geowave by locationtech.

the class DynamoDBOperations method createReader.

@Override
public RowReader<GeoWaveRow> createReader(final DataIndexReaderParams readerParams) {
    final String typeName = readerParams.getInternalAdapterStore().getTypeName(readerParams.getAdapterId());
    if (typeName == null) {
        return new RowReaderWrapper<>(new CloseableIterator.Empty<GeoWaveRow>());
    }
    byte[][] dataIds;
    Iterator<GeoWaveRow> iterator;
    if (readerParams.getDataIds() != null) {
        dataIds = readerParams.getDataIds();
        iterator = getRowsFromDataIndex(dataIds, readerParams.getAdapterId(), typeName);
    } else {
        if ((readerParams.getStartInclusiveDataId() != null) || (readerParams.getEndInclusiveDataId() != null)) {
            final List<byte[]> intermediaries = new ArrayList<>();
            ByteArrayUtils.addAllIntermediaryByteArrays(intermediaries, new ByteArrayRange(readerParams.getStartInclusiveDataId(), readerParams.getEndInclusiveDataId()));
            dataIds = intermediaries.toArray(new byte[0][]);
            iterator = getRowsFromDataIndex(dataIds, readerParams.getAdapterId(), typeName);
        } else {
            iterator = getRowsFromDataIndex(readerParams.getAdapterId(), typeName);
        }
    }
    if (options.getBaseOptions().isVisibilityEnabled()) {
        Stream<GeoWaveRow> stream = Streams.stream(iterator);
        final Set<String> authorizations = Sets.newHashSet(readerParams.getAdditionalAuthorizations());
        stream = stream.filter(new ClientVisibilityFilter(authorizations));
        iterator = stream.iterator();
    }
    return new RowReaderWrapper<>(new CloseableIterator.Wrapper<>(iterator));
}
Also used : CloseableIterator(org.locationtech.geowave.core.store.CloseableIterator) GeoWaveRow(org.locationtech.geowave.core.store.entities.GeoWaveRow) RowReaderWrapper(org.locationtech.geowave.core.store.operations.RowReaderWrapper) ArrayList(java.util.ArrayList) ByteArrayRange(org.locationtech.geowave.core.index.ByteArrayRange) ClientVisibilityFilter(org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)

Example 5 with ClientVisibilityFilter

use of org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter in project geowave by locationtech.

the class FileSystemReader method createIteratorForDataIndexReader.

private Iterator<GeoWaveRow> createIteratorForDataIndexReader(final FileSystemClient client, final DataIndexReaderParams dataIndexReaderParams) {
    final FileSystemDataIndexTable dataIndexTable = FileSystemUtils.getDataIndexTable(client, dataIndexReaderParams.getAdapterId(), dataIndexReaderParams.getInternalAdapterStore().getTypeName(dataIndexReaderParams.getAdapterId()));
    Iterator<GeoWaveRow> iterator;
    if (dataIndexReaderParams.getDataIds() != null) {
        iterator = dataIndexTable.dataIndexIterator(dataIndexReaderParams.getDataIds());
    } else {
        iterator = dataIndexTable.dataIndexIterator(dataIndexReaderParams.getStartInclusiveDataId(), dataIndexReaderParams.getEndInclusiveDataId());
    }
    if (client.isVisibilityEnabled()) {
        Stream<GeoWaveRow> stream = Streams.stream(iterator);
        final Set<String> authorizations = Sets.newHashSet(dataIndexReaderParams.getAdditionalAuthorizations());
        stream = stream.filter(new ClientVisibilityFilter(authorizations));
        iterator = stream.iterator();
    }
    return iterator;
}
Also used : GeoWaveRow(org.locationtech.geowave.core.store.entities.GeoWaveRow) FileSystemDataIndexTable(org.locationtech.geowave.datastore.filesystem.util.FileSystemDataIndexTable) ClientVisibilityFilter(org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)

Aggregations

ClientVisibilityFilter (org.locationtech.geowave.core.store.query.filter.ClientVisibilityFilter)12 ByteArrayRange (org.locationtech.geowave.core.index.ByteArrayRange)8 GeoWaveRow (org.locationtech.geowave.core.store.entities.GeoWaveRow)8 SinglePartitionQueryRanges (org.locationtech.geowave.core.index.SinglePartitionQueryRanges)6 CloseableIterator (org.locationtech.geowave.core.store.CloseableIterator)6 ArrayList (java.util.ArrayList)5 GeoWaveRowRange (org.locationtech.geowave.mapreduce.splits.GeoWaveRowRange)5 Iterator (java.util.Iterator)4 CloseableIteratorWrapper (org.locationtech.geowave.core.store.CloseableIteratorWrapper)4 GeoWaveRowMergingIterator (org.locationtech.geowave.core.store.entities.GeoWaveRowMergingIterator)4 Iterators (com.google.common.collect.Iterators)3 Sets (com.google.common.collect.Sets)3 Streams (com.google.common.collect.Streams)3 Closeable (java.io.Closeable)3 IOException (java.io.IOException)3 Arrays (java.util.Arrays)3 Collection (java.util.Collection)3 Collections (java.util.Collections)3 Set (java.util.Set)3 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3