Search in sources :

Example 1 with GeoWaveKey

use of org.locationtech.geowave.core.store.entities.GeoWaveKey in project geowave by locationtech.

the class DataStoreUtils method keysToInsertionIds.

public static InsertionIds keysToInsertionIds(final GeoWaveKey... geoWaveKeys) {
    final Map<ByteArray, List<byte[]>> sortKeysPerPartition = new HashMap<>();
    for (final GeoWaveKey key : geoWaveKeys) {
        final ByteArray partitionKey = new ByteArray(key.getPartitionKey());
        List<byte[]> sortKeys = sortKeysPerPartition.get(partitionKey);
        if (sortKeys == null) {
            sortKeys = new ArrayList<>();
            sortKeysPerPartition.put(partitionKey, sortKeys);
        }
        sortKeys.add(key.getSortKey());
    }
    final Set<SinglePartitionInsertionIds> insertionIds = new HashSet<>();
    for (final Entry<ByteArray, List<byte[]>> e : sortKeysPerPartition.entrySet()) {
        insertionIds.add(new SinglePartitionInsertionIds(e.getKey().getBytes(), e.getValue()));
    }
    return new InsertionIds(insertionIds);
}
Also used : SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) HashMap(java.util.HashMap) InsertionIds(org.locationtech.geowave.core.index.InsertionIds) SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey) ByteArray(org.locationtech.geowave.core.index.ByteArray) List(java.util.List) ArrayList(java.util.ArrayList) LinkedList(java.util.LinkedList) HashSet(java.util.HashSet)

Example 2 with GeoWaveKey

use of org.locationtech.geowave.core.store.entities.GeoWaveKey in project geowave by locationtech.

the class AccumuloReader method entryRowIdsMatch.

private boolean entryRowIdsMatch(final Entry<Key, Value> nextEntry, final Entry<Key, Value> peekedEntry) {
    final GeoWaveKey nextKey = new GeoWaveKeyImpl(nextEntry.getKey().getRow().copyBytes(), partitionKeyLength);
    final GeoWaveKey peekedKey = new GeoWaveKeyImpl(peekedEntry.getKey().getRow().copyBytes(), partitionKeyLength);
    return DataStoreUtils.rowIdsMatch(nextKey, peekedKey);
}
Also used : GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey) GeoWaveKeyImpl(org.locationtech.geowave.core.store.entities.GeoWaveKeyImpl)

Example 3 with GeoWaveKey

use of org.locationtech.geowave.core.store.entities.GeoWaveKey in project geowave by locationtech.

the class RasterTileResizeMapper method mapNativeValue.

@Override
protected void mapNativeValue(final GeoWaveInputKey key, final GridCoverage value, final MapContext<GeoWaveInputKey, GridCoverage, GeoWaveInputKey, Object> context) throws IOException, InterruptedException {
    if (helper.isOriginalCoverage(key.getInternalAdapterId())) {
        final InternalDataAdapter<?> adapter = super.serializationTool.getInternalAdapter(key.getInternalAdapterId());
        if ((adapter != null) && (adapter.getAdapter() != null) && (adapter.getAdapter() instanceof RasterDataAdapter)) {
            final Iterator<GridCoverage> coverages = helper.getCoveragesForIndex(value);
            if (coverages == null) {
                LOGGER.error("Couldn't get coverages instance, getCoveragesForIndex returned null");
                throw new IOException("Couldn't get coverages instance, getCoveragesForIndex returned null");
            }
            while (coverages.hasNext()) {
                final GridCoverage c = coverages.next();
                // converted above
                if (c instanceof FitToIndexGridCoverage) {
                    final byte[] partitionKey = ((FitToIndexGridCoverage) c).getPartitionKey();
                    final byte[] sortKey = ((FitToIndexGridCoverage) c).getSortKey();
                    final GeoWaveKey geowaveKey = new GeoWaveKeyImpl(helper.getNewDataId(c), key.getInternalAdapterId(), partitionKey, sortKey, 0);
                    final GeoWaveInputKey inputKey = new GeoWaveInputKey(helper.getNewAdapterId(), geowaveKey, helper.getIndexName());
                    context.write(inputKey, c);
                }
            }
        }
    }
}
Also used : RasterDataAdapter(org.locationtech.geowave.adapter.raster.adapter.RasterDataAdapter) FitToIndexGridCoverage(org.locationtech.geowave.adapter.raster.FitToIndexGridCoverage) GridCoverage(org.opengis.coverage.grid.GridCoverage) FitToIndexGridCoverage(org.locationtech.geowave.adapter.raster.FitToIndexGridCoverage) GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey) GeoWaveKeyImpl(org.locationtech.geowave.core.store.entities.GeoWaveKeyImpl) IOException(java.io.IOException) GeoWaveInputKey(org.locationtech.geowave.mapreduce.input.GeoWaveInputKey)

Example 4 with GeoWaveKey

use of org.locationtech.geowave.core.store.entities.GeoWaveKey in project geowave by locationtech.

the class QueryFilterIterator method aggregateFieldData.

protected FlattenedUnreadData aggregateFieldData(final Key key, final Value value, final PersistentDataset<Object> commonData) {
    final GeoWaveKey gwKey = new GeoWaveKeyImpl(key.getRow().copyBytes(), partitionKeyLength);
    final GeoWaveValue gwValue = new GeoWaveValueImpl(key.getColumnQualifier().getBytes(), key.getColumnVisibilityData().getBackingArray(), value.get());
    return DataStoreUtils.aggregateFieldData(gwKey, gwValue, commonData, model, commonIndexFieldNames);
}
Also used : GeoWaveValueImpl(org.locationtech.geowave.core.store.entities.GeoWaveValueImpl) GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey) GeoWaveKeyImpl(org.locationtech.geowave.core.store.entities.GeoWaveKeyImpl) GeoWaveValue(org.locationtech.geowave.core.store.entities.GeoWaveValue)

Example 5 with GeoWaveKey

use of org.locationtech.geowave.core.store.entities.GeoWaveKey in project geowave by locationtech.

the class PersistentDataFormatter method formatEntry.

public String formatEntry(final Entry<Key, Value> entry, final DateFormat timestampFormat) {
    final StringBuilder sb = new StringBuilder();
    final StringBuilder sbInsertion = new StringBuilder();
    final Key key = entry.getKey();
    final GeoWaveKey rowId = new GeoWaveKeyImpl(key.getRow().copyBytes(), 0);
    byte[] insertionIdBytes;
    insertionIdBytes = rowId.getSortKey();
    for (final byte b : insertionIdBytes) {
        sbInsertion.append(String.format("%02x", b));
    }
    final Text insertionIdText = new Text(sbInsertion.toString());
    final Text adapterIdText = new Text((Short.toString(rowId.getAdapterId())));
    final Text dataIdText = new Text(StringUtils.stringFromBinary(rowId.getDataId()));
    final Text duplicatesText = new Text(Integer.toString(rowId.getNumberOfDuplicates()));
    // append insertion Id
    appendText(sb, insertionIdText).append(" ");
    // append adapterId
    appendText(sb, adapterIdText).append(" ");
    // append dataId
    appendText(sb, dataIdText).append(" ");
    // append numberOfDuplicates
    appendText(sb, duplicatesText).append(" ");
    // append column family
    appendText(sb, key.getColumnFamily()).append(":");
    // append column qualifier
    appendText(sb, key.getColumnQualifier()).append(" ");
    // append visibility expression
    sb.append(new ColumnVisibility(key.getColumnVisibility()));
    // append timestamp
    if (timestampFormat != null) {
        tmpDate.get().setTime(entry.getKey().getTimestamp());
        sb.append(" ").append(timestampFormat.format(tmpDate.get()));
    }
    final Value value = entry.getValue();
    // append value
    if ((value != null) && (value.getSize() > 0)) {
        sb.append("\t");
        appendValue(sb, value);
    }
    return sb.toString();
}
Also used : GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey) GeoWaveKeyImpl(org.locationtech.geowave.core.store.entities.GeoWaveKeyImpl) Value(org.apache.accumulo.core.data.Value) Text(org.apache.hadoop.io.Text) ColumnVisibility(org.apache.accumulo.core.security.ColumnVisibility) Key(org.apache.accumulo.core.data.Key) GeoWaveKey(org.locationtech.geowave.core.store.entities.GeoWaveKey)

Aggregations

GeoWaveKey (org.locationtech.geowave.core.store.entities.GeoWaveKey)7 GeoWaveKeyImpl (org.locationtech.geowave.core.store.entities.GeoWaveKeyImpl)4 GeoWaveRowImpl (org.locationtech.geowave.core.store.entities.GeoWaveRowImpl)2 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 LinkedList (java.util.LinkedList)1 List (java.util.List)1 Key (org.apache.accumulo.core.data.Key)1 Value (org.apache.accumulo.core.data.Value)1 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)1 Text (org.apache.hadoop.io.Text)1 FitToIndexGridCoverage (org.locationtech.geowave.adapter.raster.FitToIndexGridCoverage)1 RasterDataAdapter (org.locationtech.geowave.adapter.raster.adapter.RasterDataAdapter)1 ByteArray (org.locationtech.geowave.core.index.ByteArray)1 InsertionIds (org.locationtech.geowave.core.index.InsertionIds)1 SinglePartitionInsertionIds (org.locationtech.geowave.core.index.SinglePartitionInsertionIds)1 GeoWaveValue (org.locationtech.geowave.core.store.entities.GeoWaveValue)1 GeoWaveValueImpl (org.locationtech.geowave.core.store.entities.GeoWaveValueImpl)1