Search in sources :

Example 11 with LazyElementCell

use of uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell in project Gaffer by gchq.

the class QueryAggregationProcessor method process.

@Override
public List<LazyElementCell> process(final List<LazyElementCell> elementCells) {
    if (elementCells.size() <= 1) {
        return elementCells;
    }
    final List<LazyElementCell> output = new ArrayList<>();
    ElementAggregator aggregator = null;
    Properties aggregatedProperties = null;
    LazyElementCell firstElementCell = null;
    for (final LazyElementCell elementCell : elementCells) {
        if (elementCell.isDeleted()) {
            continue;
        }
        if (!aggregatedGroups.contains(elementCell.getGroup())) {
            if (null != firstElementCell) {
                output(firstElementCell, aggregatedProperties, output);
                firstElementCell = null;
            }
            output(elementCell, null, output);
            aggregatedProperties = null;
            aggregator = null;
        } else if (null == firstElementCell) {
            firstElementCell = elementCell;
            aggregatedProperties = null;
            aggregator = null;
        } else {
            final String group = elementCell.getGroup();
            final Set<String> schemaGroupBy = schema.getElement(group).getGroupBy();
            final ViewElementDefinition elementDef = view.getElement(group);
            final Set<String> groupBy = null != elementDef ? elementDef.getGroupBy() : null;
            if (!compareGroupByKeys(firstElementCell.getCell(), elementCell.getCell(), group, schemaGroupBy, groupBy)) {
                output(firstElementCell, aggregatedProperties, output);
                firstElementCell = elementCell;
                aggregatedProperties = null;
                aggregator = null;
            } else {
                if (null == aggregator) {
                    final ElementAggregator viewAggregator = null != elementDef ? elementDef.getAggregator() : null;
                    aggregator = schema.getElement(group).getQueryAggregator(groupBy, viewAggregator);
                    aggregatedProperties = firstElementCell.getElement().getProperties();
                }
                final Properties properties = elementCell.getElement().getProperties();
                aggregatedProperties = aggregator.apply(properties, aggregatedProperties);
            }
        }
    }
    output(firstElementCell, aggregatedProperties, output);
    return output;
}
Also used : Set(java.util.Set) LazyElementCell(uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell) ArrayList(java.util.ArrayList) ViewElementDefinition(uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition) Properties(uk.gov.gchq.gaffer.data.element.Properties) ElementAggregator(uk.gov.gchq.gaffer.data.element.function.ElementAggregator)

Example 12 with LazyElementCell

use of uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell in project Gaffer by gchq.

the class QueryAggregationProcessor method output.

private void output(final LazyElementCell elementCell, final Properties aggregatedProperties, final List<LazyElementCell> output) {
    if (null == aggregatedProperties) {
        if (null != elementCell) {
            output.add(elementCell);
        }
    } else {
        try {
            final Cell firstCell = elementCell.getCell();
            final Element element = elementCell.getElement();
            element.copyProperties(aggregatedProperties);
            final Cell aggregatedCell = CellUtil.createCell(CellUtil.cloneRow(firstCell), CellUtil.cloneFamily(firstCell), serialisation.getColumnQualifier(element), serialisation.getTimestamp(element), firstCell.getTypeByte(), serialisation.getValue(element), CellUtil.getTagArray(firstCell), 0);
            elementCell.setCell(aggregatedCell);
            elementCell.setElement(element);
            output.add(elementCell);
        } catch (final SerialisationException e) {
            throw new RuntimeException(e);
        }
    }
}
Also used : SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) Element(uk.gov.gchq.gaffer.data.element.Element) LazyElementCell(uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell) Cell(org.apache.hadoop.hbase.Cell)

Example 13 with LazyElementCell

use of uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell in project Gaffer by gchq.

the class StoreAggregationProcessor method output.

private void output(final LazyElementCell elementCell, final Properties aggregatedProperties, final List<LazyElementCell> output) {
    if (null == aggregatedProperties) {
        if (null != elementCell) {
            output.add(elementCell);
        }
    } else {
        try {
            final Cell firstCell = elementCell.getCell();
            final Element element = elementCell.getElement();
            element.copyProperties(aggregatedProperties);
            final Cell aggregatedCell = CellUtil.createCell(CellUtil.cloneRow(firstCell), CellUtil.cloneFamily(firstCell), CellUtil.cloneQualifier(firstCell), serialisation.getTimestamp(element), firstCell.getTypeByte(), serialisation.getValue(element), CellUtil.getTagArray(firstCell), 0);
            elementCell.setCell(aggregatedCell);
            elementCell.setElement(element);
            output.add(elementCell);
        } catch (final SerialisationException e) {
            throw new RuntimeException(e);
        }
    }
}
Also used : SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) Element(uk.gov.gchq.gaffer.data.element.Element) LazyElementCell(uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell) Cell(org.apache.hadoop.hbase.Cell)

Aggregations

LazyElementCell (uk.gov.gchq.gaffer.hbasestore.serialisation.LazyElementCell)13 Element (uk.gov.gchq.gaffer.data.element.Element)7 Cell (org.apache.hadoop.hbase.Cell)6 Test (org.junit.jupiter.api.Test)6 ArrayList (java.util.ArrayList)4 Edge (uk.gov.gchq.gaffer.data.element.Edge)3 Entity (uk.gov.gchq.gaffer.data.element.Entity)2 Properties (uk.gov.gchq.gaffer.data.element.Properties)2 ElementAggregator (uk.gov.gchq.gaffer.data.element.function.ElementAggregator)2 SerialisationException (uk.gov.gchq.gaffer.exception.SerialisationException)2 List (java.util.List)1 Set (java.util.Set)1 InternalScanner (org.apache.hadoop.hbase.regionserver.InternalScanner)1 ScannerContext (org.apache.hadoop.hbase.regionserver.ScannerContext)1 Pair (uk.gov.gchq.gaffer.commonutil.pair.Pair)1 ViewElementDefinition (uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition)1 GafferScannerProcessor (uk.gov.gchq.gaffer.hbasestore.coprocessor.processor.GafferScannerProcessor)1