Search in sources :

Example 21 with InsertionIds

use of org.locationtech.geowave.core.index.InsertionIds in project geowave by locationtech.

the class RoundRobinKeyIndexStrategyTest method testGetInsertionIds.

@Test
public void testGetInsertionIds() {
    final List<ByteArray> ids = new ArrayList<>();
    final InsertionIds ids2 = sfcIndexStrategy.getInsertionIds(sfcIndexedRange, 1);
    final List<byte[]> compositeIds = ids2.getCompositeInsertionIds();
    for (int i = 0; i < 3; i++) {
        for (final byte[] id2 : compositeIds) {
            ids.add(new ByteArray(ByteArrayUtils.combineArrays(new byte[] { (byte) i }, id2)));
        }
    }
    final Set<ByteArray> testIds = new HashSet<>(ids);
    final Set<ByteArray> compoundIndexIds = compoundIndexStrategy.getInsertionIds(sfcIndexedRange, 8).getCompositeInsertionIds().stream().map(i -> new ByteArray(i)).collect(Collectors.toSet());
    Assert.assertTrue(testIds.containsAll(compoundIndexIds));
    final SinglePartitionInsertionIds id2 = ids2.getPartitionKeys().iterator().next();
    final MultiDimensionalCoordinates sfcIndexCoordinatesPerDim = sfcIndexStrategy.getCoordinatesPerDimension(id2.getPartitionKey(), id2.getSortKeys().get(0));
    // the first 2 bytes are the partition keys
    final MultiDimensionalCoordinates coordinatesPerDim = compoundIndexStrategy.getCoordinatesPerDimension(Arrays.copyOfRange(ids.get(0).getBytes(), 0, 2), Arrays.copyOfRange(ids.get(0).getBytes(), 2, ids.get(0).getBytes().length));
    Assert.assertTrue(sfcIndexCoordinatesPerDim.equals(coordinatesPerDim));
}
Also used : ByteArray(org.locationtech.geowave.core.index.ByteArray) CompoundIndexStrategy(org.locationtech.geowave.core.index.CompoundIndexStrategy) Arrays(java.util.Arrays) BasicNumericDataset(org.locationtech.geowave.core.index.numeric.BasicNumericDataset) PersistenceUtils(org.locationtech.geowave.core.index.persist.PersistenceUtils) HashMap(java.util.HashMap) NumericDimensionDefinition(org.locationtech.geowave.core.index.dimension.NumericDimensionDefinition) NumericData(org.locationtech.geowave.core.index.numeric.NumericData) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Map(java.util.Map) ByteArrayRange(org.locationtech.geowave.core.index.ByteArrayRange) BasicDimensionDefinition(org.locationtech.geowave.core.index.dimension.BasicDimensionDefinition) NumericRange(org.locationtech.geowave.core.index.numeric.NumericRange) SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) Set(java.util.Set) TieredSFCIndexFactory(org.locationtech.geowave.core.index.sfc.tiered.TieredSFCIndexFactory) Test(org.junit.Test) Collectors(java.util.stream.Collectors) List(java.util.List) InsertionIds(org.locationtech.geowave.core.index.InsertionIds) SFCType(org.locationtech.geowave.core.index.sfc.SFCFactory.SFCType) ByteArrayUtils(org.locationtech.geowave.core.index.ByteArrayUtils) MultiDimensionalNumericData(org.locationtech.geowave.core.index.numeric.MultiDimensionalNumericData) Assert(org.junit.Assert) NumericIndexStrategy(org.locationtech.geowave.core.index.NumericIndexStrategy) Assert.assertEquals(org.junit.Assert.assertEquals) MultiDimensionalCoordinates(org.locationtech.geowave.core.index.MultiDimensionalCoordinates) SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) InsertionIds(org.locationtech.geowave.core.index.InsertionIds) ArrayList(java.util.ArrayList) ByteArray(org.locationtech.geowave.core.index.ByteArray) MultiDimensionalCoordinates(org.locationtech.geowave.core.index.MultiDimensionalCoordinates) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 22 with InsertionIds

use of org.locationtech.geowave.core.index.InsertionIds in project geowave by locationtech.

the class TextIndexUtils method getInsertionIds.

public static InsertionIds getInsertionIds(final String entry, final EnumSet<TextSearchType> supportedSearchTypes, final EnumSet<CaseSensitivity> supportedCaseSensitivities, final int nGramCharacters) {
    if ((entry == null) || entry.isEmpty()) {
        LOGGER.info("Cannot index null string, skipping entry");
        return new InsertionIds();
    }
    final Set<TextIndexType> indexTypes = supportedSearchTypes.stream().map(TextSearchType::getIndexType).collect(Collectors.toSet());
    final List<SinglePartitionInsertionIds> retVal = new ArrayList<>(indexTypes.size());
    for (final TextIndexType indexType : indexTypes) {
        for (final CaseSensitivity caseSensitivity : supportedCaseSensitivities) {
            final boolean caseSensitive = CaseSensitivity.CASE_SENSITIVE.equals(caseSensitivity);
            switch(indexType) {
                case FORWARD:
                    retVal.add(getForwardInsertionIds(entry, caseSensitive));
                    break;
                case REVERSE:
                    retVal.add(getReverseInsertionIds(entry, caseSensitive));
                    break;
                case NGRAM:
                    final SinglePartitionInsertionIds i = getNGramInsertionIds(entry, nGramCharacters, indexTypes.contains(TextIndexType.FORWARD), caseSensitive);
                    if (i != null) {
                        retVal.add(i);
                    }
                    break;
            }
        }
    }
    return new InsertionIds(retVal);
}
Also used : SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) SinglePartitionInsertionIds(org.locationtech.geowave.core.index.SinglePartitionInsertionIds) InsertionIds(org.locationtech.geowave.core.index.InsertionIds) ArrayList(java.util.ArrayList)

Aggregations

InsertionIds (org.locationtech.geowave.core.index.InsertionIds)22 SinglePartitionInsertionIds (org.locationtech.geowave.core.index.SinglePartitionInsertionIds)16 MultiDimensionalNumericData (org.locationtech.geowave.core.index.numeric.MultiDimensionalNumericData)9 ArrayList (java.util.ArrayList)8 HashSet (java.util.HashSet)7 Test (org.junit.Test)7 NumericIndexStrategy (org.locationtech.geowave.core.index.NumericIndexStrategy)7 BasicNumericDataset (org.locationtech.geowave.core.index.numeric.BasicNumericDataset)7 ByteArray (org.locationtech.geowave.core.index.ByteArray)6 NumericData (org.locationtech.geowave.core.index.numeric.NumericData)6 HashMap (java.util.HashMap)5 BinnedNumericDataset (org.locationtech.geowave.core.index.numeric.BinnedNumericDataset)5 NumericRange (org.locationtech.geowave.core.index.numeric.NumericRange)5 List (java.util.List)4 Map (java.util.Map)4 Index (org.locationtech.geowave.core.store.api.Index)3 Maps (com.google.common.collect.Maps)2 Arrays (java.util.Arrays)2 Calendar (java.util.Calendar)2 Function (java.util.function.Function)2