Search in sources :

Example 6 with Pair

use of uk.gov.gchq.gaffer.accumulostore.utils.Pair in project Gaffer by gchq.

the class GetElementsinRangesHandlerTest method shouldHaveNoIncomingEdges.

private void shouldHaveNoIncomingEdges(final AccumuloStore store) throws OperationException {
    // Create set to query for
    final Set<Pair<ElementSeed>> simpleEntityRanges = new HashSet<>();
    final User user = new User();
    //get Everything between 0 and 1 (Note we are using strings and string serialisers, with this ordering 0999 is before 1)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("1")));
    final View view = new View.Builder(defaultView).entity(TestGroups.ENTITY, new ViewElementDefinition.Builder().groupBy().build()).edge(TestGroups.EDGE, new ViewElementDefinition.Builder().groupBy().build()).build();
    final GetElementsInRanges<Pair<ElementSeed>, Element> operation = new GetElementsInRanges<>(view, simpleEntityRanges);
    //All Edges stored should be outgoing from our provided seeds.
    operation.setIncludeIncomingOutGoing(IncludeIncomingOutgoingType.INCOMING);
    final GetElementsInRangesHandler handler = new GetElementsInRangesHandler();
    final CloseableIterable<Element> elements = handler.doOperation(operation, user, store);
    final int count = Iterables.size(elements);
    //There should be no incoming edges to the provided range
    assertEquals(0, count);
    elements.close();
}
Also used : User(uk.gov.gchq.gaffer.user.User) GetElementsInRanges(uk.gov.gchq.gaffer.accumulostore.operation.impl.GetElementsInRanges) Element(uk.gov.gchq.gaffer.data.element.Element) ViewElementDefinition(uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) HashSet(java.util.HashSet) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair)

Example 7 with Pair

use of uk.gov.gchq.gaffer.accumulostore.utils.Pair in project Gaffer by gchq.

the class ElementConverterFunction method apply.

@Override
public TraversableOnce<Tuple2<Key, Value>> apply(final Element element) {
    final ArrayBuffer<Tuple2<Key, Value>> buf = new ArrayBuffer<>();
    Pair<Key> keys = new Pair<>();
    Value value = null;
    try {
        keys = converterBroadcast.value().getKeysFromElement(element);
        value = converterBroadcast.value().getValueFromElement(element);
    } catch (final AccumuloElementConversionException e) {
        LOGGER.error(e.getMessage(), e);
    }
    final Key first = keys.getFirst();
    if (first != null) {
        buf.$plus$eq(new Tuple2<>(first, value));
    }
    final Key second = keys.getSecond();
    if (second != null) {
        buf.$plus$eq(new Tuple2<>(second, value));
    }
    return buf;
}
Also used : Tuple2(scala.Tuple2) Value(org.apache.accumulo.core.data.Value) ArrayBuffer(scala.collection.mutable.ArrayBuffer) Key(org.apache.accumulo.core.data.Key) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair) AccumuloElementConversionException(uk.gov.gchq.gaffer.accumulostore.key.exception.AccumuloElementConversionException)

Example 8 with Pair

use of uk.gov.gchq.gaffer.accumulostore.utils.Pair in project Gaffer by gchq.

the class AbstractCoreKeyAccumuloElementConverter method getKeysFromEdge.

@Override
public Pair<Key> getKeysFromEdge(final Edge edge) throws AccumuloElementConversionException {
    // Get pair of row keys
    final Pair<byte[]> rowKeys = getRowKeysFromEdge(edge);
    final byte[] columnFamily = buildColumnFamily(edge.getGroup());
    final byte[] columnQualifier = buildColumnQualifier(edge.getGroup(), edge.getProperties());
    final byte[] columnVisibility = buildColumnVisibility(edge.getGroup(), edge.getProperties());
    final long timeStamp = buildTimestamp(edge.getProperties());
    // Create Accumulo keys - note that second row key may be null (if it's
    // a self-edge) and
    // in that case we should return null second key
    final Key key1 = new Key(rowKeys.getFirst(), columnFamily, columnQualifier, columnVisibility, timeStamp);
    final Key key2 = rowKeys.getSecond() != null ? new Key(rowKeys.getSecond(), columnFamily, columnQualifier, columnVisibility, timeStamp) : null;
    // Return pair of keys
    return new Pair<>(key1, key2);
}
Also used : Key(org.apache.accumulo.core.data.Key) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair)

Example 9 with Pair

use of uk.gov.gchq.gaffer.accumulostore.utils.Pair in project Gaffer by gchq.

the class GetElementsinRangesHandlerTest method shouldSummariseOutGoingEdgesOnly.

private void shouldSummariseOutGoingEdgesOnly(final AccumuloStore store) throws OperationException {
    // Create set to query for
    final Set<Pair<ElementSeed>> simpleEntityRanges = new HashSet<>();
    //get Everything between 0 and 1 (Note we are using strings and string serialisers, with this ordering 0999 is before 1)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("C")));
    final View view = new View.Builder(defaultView).entity(TestGroups.ENTITY, new ViewElementDefinition.Builder().groupBy().build()).edge(TestGroups.EDGE, new ViewElementDefinition.Builder().groupBy().build()).build();
    final GetElementsInRanges<Pair<ElementSeed>, Element> operation = new GetElementsInRanges<>(view, simpleEntityRanges);
    //All Edges stored should be outgoing from our provided seeds.
    operation.setIncludeIncomingOutGoing(IncludeIncomingOutgoingType.OUTGOING);
    final GetElementsInRangesHandler handler = new GetElementsInRangesHandler();
    final CloseableIterable<Element> rangeElements = handler.doOperation(operation, user, store);
    int count = 0;
    for (final Element elm : rangeElements) {
        //Make sure every element has been summarised
        assertEquals(9, elm.getProperty(AccumuloPropertyNames.COLUMN_QUALIFIER));
        count++;
    }
    assertEquals(1000, count);
    rangeElements.close();
    simpleEntityRanges.clear();
    //This should get everything between 0 and 0799 (again being string ordering 0800 is more than 08)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("08")));
    final CloseableIterable<Element> elements = handler.doOperation(operation, user, store);
    count = 0;
    for (final Element elm : elements) {
        //Make sure every element has been summarised
        assertEquals(9, elm.getProperty(AccumuloPropertyNames.COLUMN_QUALIFIER));
        count++;
    }
    assertEquals(800, count);
    elements.close();
}
Also used : GetElementsInRanges(uk.gov.gchq.gaffer.accumulostore.operation.impl.GetElementsInRanges) Element(uk.gov.gchq.gaffer.data.element.Element) ViewElementDefinition(uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) HashSet(java.util.HashSet) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair)

Example 10 with Pair

use of uk.gov.gchq.gaffer.accumulostore.utils.Pair in project Gaffer by gchq.

the class GetElementsinRangesHandlerTest method shouldReturnElementsNoSummarisation.

private void shouldReturnElementsNoSummarisation(final AccumuloStore store) throws OperationException {
    // Create set to query for
    final Set<Pair<ElementSeed>> simpleEntityRanges = new HashSet<>();
    final User user = new User();
    //get Everything between 0 and 1 (Note we are using strings and string serialisers, with this ordering 0999 is before 1)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("1")));
    final GetElementsInRanges<Pair<ElementSeed>, Element> operation = new GetElementsInRanges<>(defaultView, simpleEntityRanges);
    final GetElementsInRangesHandler handler = new GetElementsInRangesHandler();
    CloseableIterable<Element> elementsInRanges = handler.doOperation(operation, user, store);
    final int elementsInRangesCount = Iterables.size(elementsInRanges);
    //Each Edge was put in 3 times with different col qualifiers, without summarisation we expect this number
    assertEquals(1000 * 3, elementsInRangesCount);
    elementsInRanges.close();
    simpleEntityRanges.clear();
    //This should get everything between 0 and 0799 (again being string ordering 0800 is more than 08)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("08")));
    final CloseableIterable<Element> elements = handler.doOperation(operation, user, store);
    final int count = Iterables.size(elements);
    //Each Edge was put in 3 times with different col qualifiers, without summarisation we expect this number
    assertEquals(800 * 3, count);
    elements.close();
}
Also used : User(uk.gov.gchq.gaffer.user.User) GetElementsInRanges(uk.gov.gchq.gaffer.accumulostore.operation.impl.GetElementsInRanges) Element(uk.gov.gchq.gaffer.data.element.Element) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) HashSet(java.util.HashSet) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair)

Aggregations

Pair (uk.gov.gchq.gaffer.accumulostore.utils.Pair)13 EntitySeed (uk.gov.gchq.gaffer.operation.data.EntitySeed)9 HashSet (java.util.HashSet)6 GetElementsInRanges (uk.gov.gchq.gaffer.accumulostore.operation.impl.GetElementsInRanges)6 ElementSeed (uk.gov.gchq.gaffer.operation.data.ElementSeed)6 Element (uk.gov.gchq.gaffer.data.element.Element)5 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)4 ViewElementDefinition (uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition)4 ArrayList (java.util.ArrayList)3 Iterator (java.util.Iterator)3 Key (org.apache.accumulo.core.data.Key)3 Test (org.junit.Test)3 OperationTest (uk.gov.gchq.gaffer.operation.OperationTest)3 User (uk.gov.gchq.gaffer.user.User)3 Range (org.apache.accumulo.core.data.Range)1 Value (org.apache.accumulo.core.data.Value)1 Tuple2 (scala.Tuple2)1 ArrayBuffer (scala.collection.mutable.ArrayBuffer)1 AccumuloElementConversionException (uk.gov.gchq.gaffer.accumulostore.key.exception.AccumuloElementConversionException)1 IteratorSettingException (uk.gov.gchq.gaffer.accumulostore.key.exception.IteratorSettingException)1