Search in sources :

Example 31 with Node

use of org.openforis.idm.model.Node in project collect by openforis.

the class DataTableDataExtractor method extractDataInternal.

private void extractDataInternal(DataTable table, Node<?> source, Dataset data) {
    // Extract data from this node
    Row row = extractRow(source);
    data.addRow(row);
    // Extract data from descendants
    for (DataTable childTable : table.getChildTables()) {
        Path path = childTable.getRelativePath();
        List<Node<?>> children = path.evaluate(source);
        for (Node<?> child : children) {
            extractDataInternal(childTable, child, data);
        }
    }
}
Also used : Path(org.openforis.idm.path.Path) DataTable(org.openforis.collect.relational.model.DataTable) Node(org.openforis.idm.model.Node) Row(org.openforis.collect.relational.data.Row)

Example 32 with Node

use of org.openforis.idm.model.Node in project collect by openforis.

the class DataQueryExectutorTask method execute.

@Override
@Transactional
protected void execute() throws Throwable {
    CollectSurvey survey = input.query.getSurvey();
    DataQueryEvaluator queryEvaluator = createQueryEvaluator(input.query);
    RecordFilter filter = createRecordsFilter(true);
    List<CollectRecordSummary> recordSummaries = recordManager.loadSummaries(filter);
    Iterator<CollectRecordSummary> it = recordSummaries.iterator();
    while (it.hasNext() && isRunning()) {
        CollectRecordSummary recordSummary = (CollectRecordSummary) it.next();
        CollectRecord record = recordManager.load(survey, recordSummary.getId(), input.step, false);
        List<Node<?>> nodes = queryEvaluator.evaluate(record);
        for (Node<?> node : nodes) {
            processNode(node);
        }
        incrementProcessedItems();
    }
}
Also used : CollectRecord(org.openforis.collect.model.CollectRecord) CollectRecordSummary(org.openforis.collect.model.CollectRecordSummary) Node(org.openforis.idm.model.Node) CollectSurvey(org.openforis.collect.model.CollectSurvey) XPathDataQueryEvaluator(org.openforis.collect.datacleansing.xpath.XPathDataQueryEvaluator) RecordFilter(org.openforis.collect.model.RecordFilter) Transactional(org.springframework.transaction.annotation.Transactional)

Example 33 with Node

use of org.openforis.idm.model.Node in project collect by openforis.

the class DataQueryGroupExectutorTask method execute.

@Override
@Transactional
protected void execute() throws Throwable {
    CollectSurvey survey = input.getSurvey();
    RecordFilter filter = createRecordsFilter();
    List<CollectRecordSummary> recordSummaries = recordManager.loadSummaries(filter);
    Iterator<CollectRecordSummary> it = recordSummaries.iterator();
    while (it.hasNext() && isRunning()) {
        CollectRecordSummary recordSummary = (CollectRecordSummary) it.next();
        Date modifiedDate = recordSummary.getModifiedDate();
        if (lastRecordModifiedDate == null) {
            lastRecordModifiedDate = modifiedDate;
        } else if (modifiedDate.compareTo(lastRecordModifiedDate) > 0) {
            lastRecordModifiedDate = modifiedDate;
        }
        CollectRecord record = recordManager.load(survey, recordSummary.getId(), input.step, false);
        List<DataQuery> queries = input.getQueries();
        for (DataQuery query : queries) {
            DataQueryEvaluator queryEvaluator = createQueryEvaluator(query);
            List<Node<?>> nodes = queryEvaluator.evaluate(record);
            for (Node<?> node : nodes) {
                processNode(query, node);
            }
        }
        incrementProcessedItems();
    }
}
Also used : CollectRecord(org.openforis.collect.model.CollectRecord) Node(org.openforis.idm.model.Node) Date(java.util.Date) CollectRecordSummary(org.openforis.collect.model.CollectRecordSummary) CollectSurvey(org.openforis.collect.model.CollectSurvey) RecordFilter(org.openforis.collect.model.RecordFilter) XPathDataQueryEvaluator(org.openforis.collect.datacleansing.xpath.XPathDataQueryEvaluator) Transactional(org.springframework.transaction.annotation.Transactional)

Example 34 with Node

use of org.openforis.idm.model.Node in project collect by openforis.

the class DataQueryResultIterator method fetchNextRecord.

private void fetchNextRecord() {
    currentRecord = null;
    currentNodes = null;
    currentNodeIndex = 0;
    while (active && currentRecordIndex < recordSummaries.size()) {
        CollectRecord summary = recordSummaries.get(currentRecordIndex++);
        CollectRecord record = recordManager.load((CollectSurvey) summary.getSurvey(), summary.getId());
        List<Node<?>> nodes = queryEvaluator.evaluate(record);
        if (!nodes.isEmpty()) {
            currentRecord = record;
            currentNodes = nodes;
        }
    }
}
Also used : CollectRecord(org.openforis.collect.model.CollectRecord) Node(org.openforis.idm.model.Node)

Example 35 with Node

use of org.openforis.idm.model.Node in project collect by openforis.

the class RecordUpdater method deleteNode.

/**
 * Deletes a node from the record.
 *
 * @param node
 * @return
 */
public NodeChangeSet deleteNode(Node<?> node) {
    Record record = node.getRecord();
    NodeChangeMap changeMap = new NodeChangeMap();
    Set<Node<?>> nodesToBeDeleted = node.getDescendantsAndSelf();
    Set<NodePointer> pointersToBeDeleted = nodesToPointers(nodesToBeDeleted);
    NodePointer nodePointer = new NodePointer(node);
    List<NodePointer> ancestorPointers = getAncestorPointers(node);
    Entity parentEntity = node.getParent();
    // calculated attributes
    List<Attribute<?, ?>> dependentCalculatedAttributes = record.determineCalculatedAttributes(nodesToBeDeleted);
    dependentCalculatedAttributes.removeAll(nodesToBeDeleted);
    if (validateAfterUpdate) {
        // relevance
        List<NodePointer> relevanceDependenciesToDeleted = record.determineRelevanceDependentNodes(nodesToBeDeleted);
        // min/max
        Collection<NodePointer> preDeletionMinMaxDependenciesToCheck = new HashSet<NodePointer>(pointersToBeDeleted);
        preDeletionMinMaxDependenciesToCheck.addAll(getAncestorsAndSelfPointers(node));
        Collection<NodePointer> minCountDependenciesToDelete = record.determineMinCountDependentNodes(preDeletionMinMaxDependenciesToCheck);
        Collection<NodePointer> maxCountDependenciesToDelete = record.determineMaxCountDependentNodes(preDeletionMinMaxDependenciesToCheck);
        // validation
        Set<Attribute<?, ?>> validationDependenciesToDeleted = record.determineValidationDependentNodes(nodesToBeDeleted);
        validationDependenciesToDeleted.removeAll(nodesToBeDeleted);
        List<Integer> ancestorIds = node.getAncestorIds();
        performNodeDeletion(node);
        changeMap.addNodeDeleteChange(record.getId(), ((CollectRecord) record).getStep(), ancestorIds, node);
        // calculated attributes
        List<Attribute<?, ?>> updatedCalculatedAttributes = recalculateValues(dependentCalculatedAttributes);
        changeMap.addValueChanges(updatedCalculatedAttributes);
        // relevance
        Set<NodePointer> pointersToRecalculateRelevanceFor = new HashSet<NodePointer>();
        pointersToRecalculateRelevanceFor.addAll(record.determineRelevanceDependentNodes(updatedCalculatedAttributes));
        pointersToRecalculateRelevanceFor.addAll(relevanceDependenciesToDeleted);
        if (parentEntity != null) {
            pointersToRecalculateRelevanceFor.addAll(record.determineRelevanceDependentNodePointers(Arrays.asList(new NodePointer(parentEntity, node.getDefinition()))));
        }
        List<NodePointer> relevanceToUpdate = record.determineRelevanceDependentNodePointers(pointersToRecalculateRelevanceFor);
        // check relevance update with detached entity in node pointer
        RelevanceUpdater relevanceUpdater = new RelevanceUpdater(relevanceToUpdate);
        Set<NodePointer> updatedRelevancePointers = relevanceUpdater.update();
        changeMap.addRelevanceChanges(updatedRelevancePointers);
        // min count
        Collection<NodePointer> pointersToCheckMinCountFor = new HashSet<NodePointer>(updatedRelevancePointers);
        pointersToCheckMinCountFor.addAll(minCountDependenciesToDelete);
        pointersToCheckMinCountFor.addAll(nodesToPointers(updatedCalculatedAttributes));
        Collection<NodePointer> minCountPointersToUpdate = record.determineMinCountDependentNodes(pointersToCheckMinCountFor);
        Collection<NodePointer> updatedMinCountPointers = updateMinCount(minCountPointersToUpdate);
        changeMap.addMinCountChanges(updatedMinCountPointers);
        // max count
        Collection<NodePointer> pointersToCheckMaxCountFor = new HashSet<NodePointer>(updatedRelevancePointers);
        pointersToCheckMaxCountFor.addAll(maxCountDependenciesToDelete);
        pointersToCheckMaxCountFor.addAll(nodesToPointers(updatedCalculatedAttributes));
        Collection<NodePointer> maxCountPointersToUpdate = record.determineMaxCountDependentNodes(pointersToCheckMaxCountFor);
        Collection<NodePointer> updatedMaxCountPointers = updateMaxCount(maxCountPointersToUpdate);
        changeMap.addMinCountChanges(updatedMaxCountPointers);
        Set<NodePointer> updatedCardinalityPointers = new HashSet<NodePointer>(updatedMinCountPointers);
        updatedCardinalityPointers.addAll(updatedMaxCountPointers);
        // validate cardinality
        Set<NodePointer> pointersToValidateCardinalityFor = new HashSet<NodePointer>(updatedMinCountPointers.size() + updatedMaxCountPointers.size());
        pointersToValidateCardinalityFor.addAll(updatedMinCountPointers);
        pointersToValidateCardinalityFor.addAll(updatedMaxCountPointers);
        // validate cardinality on ancestor node pointers because we are considering empty nodes as missing nodes
        pointersToValidateCardinalityFor.add(nodePointer);
        pointersToValidateCardinalityFor.addAll(ancestorPointers);
        validateCardinality(record, pointersToValidateCardinalityFor, changeMap);
        // validate attributes
        Set<Node<?>> nodesToCheckValidationFor = new HashSet<Node<?>>(validationDependenciesToDeleted);
        nodesToCheckValidationFor.addAll(updatedCalculatedAttributes);
        nodesToCheckValidationFor.addAll(pointersToNodes(updatedRelevancePointers));
        nodesToCheckValidationFor.addAll(pointersToNodes(updatedCardinalityPointers));
        Set<Attribute<?, ?>> attributesToRevalidate = record.determineValidationDependentNodes(nodesToCheckValidationFor);
        validateAttributes(record, attributesToRevalidate, changeMap);
    } else {
        performNodeDeletion(node);
        // calculated attributes
        List<Attribute<?, ?>> updatedCalculatedAttributes = recalculateValues(dependentCalculatedAttributes);
        changeMap.addValueChanges(updatedCalculatedAttributes);
    }
    return changeMap;
}
Also used : Entity(org.openforis.idm.model.Entity) CodeAttribute(org.openforis.idm.model.CodeAttribute) Attribute(org.openforis.idm.model.Attribute) BooleanAttribute(org.openforis.idm.model.BooleanAttribute) Node(org.openforis.idm.model.Node) NodePointer(org.openforis.idm.model.NodePointer) Record(org.openforis.idm.model.Record) HashSet(java.util.HashSet) LinkedHashSet(java.util.LinkedHashSet)

Aggregations

Node (org.openforis.idm.model.Node)37 Entity (org.openforis.idm.model.Entity)19 Test (org.junit.Test)13 ArrayList (java.util.ArrayList)11 AbstractTest (org.openforis.idm.AbstractTest)11 NodeDefinition (org.openforis.idm.metamodel.NodeDefinition)8 CollectRecord (org.openforis.collect.model.CollectRecord)6 RealAttribute (org.openforis.idm.model.RealAttribute)6 Attribute (org.openforis.idm.model.Attribute)5 CodeAttribute (org.openforis.idm.model.CodeAttribute)5 Code (org.openforis.idm.model.Code)4 Record (org.openforis.idm.model.Record)4 HashSet (java.util.HashSet)3 LinkedHashSet (java.util.LinkedHashSet)3 XPathDataQueryEvaluator (org.openforis.collect.datacleansing.xpath.XPathDataQueryEvaluator)3 CollectRecordSummary (org.openforis.collect.model.CollectRecordSummary)3 CollectSurvey (org.openforis.collect.model.CollectSurvey)3 RecordFilter (org.openforis.collect.model.RecordFilter)3 AttributeDefinition (org.openforis.idm.metamodel.AttributeDefinition)3 EntityDefinition (org.openforis.idm.metamodel.EntityDefinition)3