Search in sources :

Example 36 with Uid

use of datawave.ingest.protobuf.Uid in project datawave by NationalSecurityAgency.

the class ShardedDataTypeHandlerTest method testCreateTermReverseIndex.

@Test
public void testCreateTermReverseIndex() {
    Type dataType = new Type(DATA_TYPE_NAME, TestIngestHelper.class, null, null, 10, null);
    String entry = "testingtesting";
    RawRecordContainer record = new RawRecordContainerImpl();
    record.setDataType(dataType);
    record.setRawFileName("data_" + 0 + ".dat");
    record.setRawRecordNumber(1);
    record.setRawData(entry.getBytes(StandardCharsets.UTF_8));
    Uid.List uid = Uid.List.newBuilder().setIGNORE(false).setCOUNT(1).addUID("d8zay2.-3pnndm.-anolok").build();
    byte[] visibility = new byte[] { 65, 76, 76 };
    byte[] maskVisibility = new byte[] { 67, 76, 76 };
    byte[] shardId = new byte[] { 50, 48, 48, 48, 48, 49, 48, 49, 95, 54, 57 };
    Multimap<BulkIngestKey, Value> termIndex = handler.createTermIndexColumn(record, "TEST_COL", "FIELD_VALUE", visibility, maskVisibility, maskedFieldHelper, shardId, handler.getShardIndexTableName(), new Value(uid.toByteArray()), Direction.REVERSE);
    assertTrue(termIndex.size() == 2);
    boolean foundValue = false;
    for (BulkIngestKey k : termIndex.keySet()) {
        Text row = k.getKey().getRow();
        if (row.toString().contains("EULAV_DEKSAM")) {
            foundValue = true;
        }
    }
    assertTrue(foundValue);
}
Also used : Uid(datawave.ingest.protobuf.Uid) GeometryType(datawave.data.type.GeometryType) NumberType(datawave.data.type.NumberType) Type(datawave.ingest.data.Type) RawRecordContainer(datawave.ingest.data.RawRecordContainer) NormalizedFieldAndValue(datawave.ingest.data.config.NormalizedFieldAndValue) Value(org.apache.accumulo.core.data.Value) BulkIngestKey(datawave.ingest.mapreduce.job.BulkIngestKey) Text(org.apache.hadoop.io.Text) RawRecordContainerImpl(datawave.ingest.config.RawRecordContainerImpl) Test(org.junit.Test)

Aggregations

Uid (datawave.ingest.protobuf.Uid)36 Value (org.apache.accumulo.core.data.Value)36 Test (org.junit.Test)29 Key (org.apache.accumulo.core.data.Key)10 BulkIngestKey (datawave.ingest.mapreduce.job.BulkIngestKey)7 Text (org.apache.hadoop.io.Text)7 NormalizedFieldAndValue (datawave.ingest.data.config.NormalizedFieldAndValue)6 GeometryType (datawave.data.type.GeometryType)5 NumberType (datawave.data.type.NumberType)5 RawRecordContainerImpl (datawave.ingest.config.RawRecordContainerImpl)5 RawRecordContainer (datawave.ingest.data.RawRecordContainer)5 Type (datawave.ingest.data.Type)5 InvalidProtocolBufferException (com.google.protobuf.InvalidProtocolBufferException)3 Builder (datawave.ingest.protobuf.Uid.List.Builder)3 ArrayList (java.util.ArrayList)3 IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)3 CacheBuilder (com.google.common.cache.CacheBuilder)2 List (java.util.List)2 Entry (java.util.Map.Entry)2 IngestHelperInterface (datawave.ingest.data.config.ingest.IngestHelperInterface)1