Search in sources :

Example 31 with Key

use of org.apache.accumulo.core.data.Key in project Gaffer by gchq.

the class ElementConverterFunction method call.

@Override
public Iterator<Tuple2<Key, Value>> call(final Element e) throws Exception {
    final List<Tuple2<Key, Value>> tuples = new ArrayList<>(2);
    final Pair<Key> keys = converterBroadcast.value().getKeysFromElement(e);
    final Value value = converterBroadcast.value().getValueFromElement(e);
    tuples.add(new Tuple2<>(keys.getFirst(), value));
    final Key second = keys.getSecond();
    if (second != null) {
        tuples.add(new Tuple2<>(second, value));
    }
    return tuples.listIterator();
}
Also used : Tuple2(scala.Tuple2) ArrayList(java.util.ArrayList) Value(org.apache.accumulo.core.data.Value) Key(org.apache.accumulo.core.data.Key)

Example 32 with Key

use of org.apache.accumulo.core.data.Key in project Gaffer by gchq.

the class ElementConverterFunction method apply.

@Override
public TraversableOnce<Tuple2<Key, Value>> apply(final Element element) {
    final ArrayBuffer<Tuple2<Key, Value>> buf = new ArrayBuffer<>();
    Pair<Key> keys = new Pair<>();
    Value value = null;
    try {
        keys = converterBroadcast.value().getKeysFromElement(element);
        value = converterBroadcast.value().getValueFromElement(element);
    } catch (final AccumuloElementConversionException e) {
        LOGGER.error(e.getMessage(), e);
    }
    final Key first = keys.getFirst();
    if (first != null) {
        buf.$plus$eq(new Tuple2<>(first, value));
    }
    final Key second = keys.getSecond();
    if (second != null) {
        buf.$plus$eq(new Tuple2<>(second, value));
    }
    return buf;
}
Also used : Tuple2(scala.Tuple2) Value(org.apache.accumulo.core.data.Value) ArrayBuffer(scala.collection.mutable.ArrayBuffer) Key(org.apache.accumulo.core.data.Key) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair) AccumuloElementConversionException(uk.gov.gchq.gaffer.accumulostore.key.exception.AccumuloElementConversionException)

Example 33 with Key

use of org.apache.accumulo.core.data.Key in project Gaffer by gchq.

the class RowIDAggregator method deepCopy.

@Override
public SortedKeyValueIterator<Key, Value> deepCopy(final IteratorEnvironment env) {
    RowIDAggregator rowIDAggregator = new RowIDAggregator();
    rowIDAggregator.topKey = this.topKey;
    rowIDAggregator.topValue = this.topValue;
    rowIDAggregator.schema = this.schema;
    rowIDAggregator.aggregator = this.aggregator;
    rowIDAggregator.elementConverter = this.elementConverter;
    Key newWorkKey = new Key();
    newWorkKey.set(workKey);
    rowIDAggregator.workKey = newWorkKey;
    return rowIDAggregator;
}
Also used : Key(org.apache.accumulo.core.data.Key) PartialKey(org.apache.accumulo.core.data.PartialKey)

Example 34 with Key

use of org.apache.accumulo.core.data.Key in project Gaffer by gchq.

the class RowIDAggregator method seek.

@Override
public void seek(final Range range, final Collection<ByteSequence> columnFamilies, final boolean inclusive) throws IOException {
    topKey = null;
    workKey = new Key();
    super.seek(range, columnFamilies, inclusive);
    currentRange = range;
    currentColumnFamilies = columnFamilies;
    currentColumnFamiliesInclusive = inclusive;
    findTop();
}
Also used : Key(org.apache.accumulo.core.data.Key) PartialKey(org.apache.accumulo.core.data.PartialKey)

Example 35 with Key

use of org.apache.accumulo.core.data.Key in project Gaffer by gchq.

the class RowIDAggregator method findTop.

/**
     * Given the current position in the source}, filter to only the columns specified. Sets topKey and topValue to non-null on success
     *
     * @throws IOException Failure to seek
     */
protected void findTop() throws IOException {
    if (!source.hasTop()) {
        return;
    }
    PropertiesIterator iter = new PropertiesIterator(source, currentRange, currentColumnFamilies, currentColumnFamiliesInclusive, group, workKey, elementConverter);
    Properties topProperties = reduce(iter);
    try {
        topValue = elementConverter.getValueFromProperties(group, topProperties);
        topKey = new Key(workKey.getRowData().getBackingArray(), group.getBytes(CommonConstants.UTF_8), elementConverter.buildColumnQualifier(group, topProperties), elementConverter.buildColumnVisibility(group, topProperties), elementConverter.buildTimestamp(topProperties));
    } catch (AccumuloElementConversionException e) {
        throw new RuntimeException(e);
    }
}
Also used : Properties(uk.gov.gchq.gaffer.data.element.Properties) Key(org.apache.accumulo.core.data.Key) PartialKey(org.apache.accumulo.core.data.PartialKey) AccumuloElementConversionException(uk.gov.gchq.gaffer.accumulostore.key.exception.AccumuloElementConversionException)

Aggregations

Key (org.apache.accumulo.core.data.Key)115 Value (org.apache.accumulo.core.data.Value)68 Test (org.junit.Test)66 Edge (uk.gov.gchq.gaffer.data.element.Edge)44 Range (org.apache.accumulo.core.data.Range)35 HashMap (java.util.HashMap)29 Text (org.apache.hadoop.io.Text)23 Element (uk.gov.gchq.gaffer.data.element.Element)23 Scanner (org.apache.accumulo.core.client.Scanner)19 Authorizations (org.apache.accumulo.core.security.Authorizations)18 Mutation (org.apache.accumulo.core.data.Mutation)17 Entity (uk.gov.gchq.gaffer.data.element.Entity)15 Entry (java.util.Map.Entry)14 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)13 AccumuloException (org.apache.accumulo.core.client.AccumuloException)11 Connector (org.apache.accumulo.core.client.Connector)11 IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)11 ExprNodeColumnDesc (org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc)11 ExprNodeConstantDesc (org.apache.hadoop.hive.ql.plan.ExprNodeConstantDesc)11 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)11