Search in sources :

Example 1 with SortedListKeyValueIterator

use of datawave.query.iterator.SortedListKeyValueIterator in project datawave by NationalSecurityAgency.

the class TermFrequencyIteratorTest method buildIterAcrossValuesWithNulls.

// Build some basic TF keys with values
public SortedListKeyValueIterator buildIterAcrossValuesWithNulls() {
    List<Map.Entry<Key, Value>> baseSource = new ArrayList<>();
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456", "FIELD_A", "value_a"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456", "FIELD_A", "value_b"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456", "FIELD_A", "value_c"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456", "FIELD_A", "val\0ue_d"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FIELD_A", "value_a"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FIELD_A", "value_b"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FIELD_A", "val\0ue_c"), new Value()));
    baseSource.add(new SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FIELD_A", "value_d"), new Value()));
    return new SortedListKeyValueIterator(baseSource);
}
Also used : SortedListKeyValueIterator(datawave.query.iterator.SortedListKeyValueIterator) SimpleEntry(java.util.AbstractMap.SimpleEntry) SimpleEntry(java.util.AbstractMap.SimpleEntry) ArrayList(java.util.ArrayList) Value(org.apache.accumulo.core.data.Value)

Example 2 with SortedListKeyValueIterator

use of datawave.query.iterator.SortedListKeyValueIterator in project datawave by NationalSecurityAgency.

the class TermFrequencyIndexIteratorTest method setup.

@Before
public void setup() throws ParseException {
    List<Map.Entry<Key, Value>> baseSource = new ArrayList<>();
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456", "FOO", "bar"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456", "FOO", "baz"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FOO", "buf"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.1", "FOO", "buz"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.2", "FOO", "alf"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.2", "FOO", "arm"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.2", "FOOT", "armfoot"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.3", "AFOO", "alfa"), new Value()));
    baseSource.add(new AbstractMap.SimpleEntry(getTfKey("row", "type1", "123.345.456.3", "ZFOO", "alfz"), new Value()));
    source = new SortedListKeyValueIterator(baseSource);
    String lcNoDiacritics = LcNoDiacriticsType.class.getName();
    typeMetadata = new TypeMetadata();
    typeMetadata.put("FOO", "type1", lcNoDiacritics);
    fieldsToKeep = new HashSet<>();
    fieldsToKeep.add("FOO");
    filter = new EventDataQueryExpressionFilter(JexlASTHelper.parseJexlQuery("FOO=='bar' || FOO=='baz' || FOO=='buf' || FOO=='buz' || FOO=='alf' || FOO=='arm'"), typeMetadata, fieldsToKeep);
    aggregator = new TermFrequencyAggregator(fieldsToKeep, filter);
}
Also used : AbstractMap(java.util.AbstractMap) SortedListKeyValueIterator(datawave.query.iterator.SortedListKeyValueIterator) TypeMetadata(datawave.query.util.TypeMetadata) EventDataQueryExpressionFilter(datawave.query.predicate.EventDataQueryExpressionFilter) ArrayList(java.util.ArrayList) Value(org.apache.accumulo.core.data.Value) TermFrequencyAggregator(datawave.query.jexl.functions.TermFrequencyAggregator) TLDTermFrequencyAggregator(datawave.query.tld.TLDTermFrequencyAggregator) Before(org.junit.Before)

Aggregations

SortedListKeyValueIterator (datawave.query.iterator.SortedListKeyValueIterator)2 ArrayList (java.util.ArrayList)2 Value (org.apache.accumulo.core.data.Value)2 TermFrequencyAggregator (datawave.query.jexl.functions.TermFrequencyAggregator)1 EventDataQueryExpressionFilter (datawave.query.predicate.EventDataQueryExpressionFilter)1 TLDTermFrequencyAggregator (datawave.query.tld.TLDTermFrequencyAggregator)1 TypeMetadata (datawave.query.util.TypeMetadata)1 AbstractMap (java.util.AbstractMap)1 SimpleEntry (java.util.AbstractMap.SimpleEntry)1 Before (org.junit.Before)1