Search in sources :

Example 1 with RecordFilter

use of org.openforis.collect.model.RecordFilter in project collect by openforis.

the class DataQueryExectutorTask method createRecordsFilter.

private RecordFilter createRecordsFilter(boolean limitResults) {
    CollectSurvey survey = input.query.getSurvey();
    EntityDefinition entityDef = (EntityDefinition) survey.getSchema().getDefinitionById(input.query.getEntityDefinitionId());
    EntityDefinition rootEntityDef = entityDef.getRootEntity();
    Integer rootEntityId = rootEntityDef.getId();
    RecordFilter filter = new RecordFilter(survey);
    filter.setStep(input.step);
    filter.setRootEntityId(rootEntityId);
    if (limitResults) {
        filter.setOffset(0);
        filter.setMaxNumberOfRecords(input.maxRecords);
    }
    return filter;
}
Also used : EntityDefinition(org.openforis.idm.metamodel.EntityDefinition) CollectSurvey(org.openforis.collect.model.CollectSurvey) RecordFilter(org.openforis.collect.model.RecordFilter)

Example 2 with RecordFilter

use of org.openforis.collect.model.RecordFilter in project collect by openforis.

the class DataQueryGroupExectutorTask method createRecordsFilter.

private RecordFilter createRecordsFilter() {
    CollectSurvey survey = input.getSurvey();
    RecordFilter filter = new RecordFilter(survey);
    filter.setStep(input.step);
    filter.setOffset(0);
    filter.setMaxNumberOfRecords(input.maxRecords);
    return filter;
}
Also used : CollectSurvey(org.openforis.collect.model.CollectSurvey) RecordFilter(org.openforis.collect.model.RecordFilter)

Example 3 with RecordFilter

use of org.openforis.collect.model.RecordFilter in project collect by openforis.

the class DataQueryGroupExectutorTask method countTotalItems.

@Override
protected long countTotalItems() {
    RecordFilter recordsFilter = createRecordsFilter();
    int count = recordManager.countRecords(recordsFilter);
    return count;
}
Also used : RecordFilter(org.openforis.collect.model.RecordFilter)

Example 4 with RecordFilter

use of org.openforis.collect.model.RecordFilter in project collect by openforis.

the class DataCleansingChainExectutorTask method execute.

@Override
@Transactional
protected void execute() throws Throwable {
    CollectSurvey survey = input.chain.getSurvey();
    RecordFilter filter = createRecordsFilter();
    List<CollectRecordSummary> recordSummaries = recordManager.loadSummaries(filter);
    datasetSize = recordSummaries.size();
    lastRecordModifiedDate = null;
    // Delete records in bulk if needed
    Set<Integer> recordIdsToBeDeleted = new TreeSet<Integer>();
    Iterator<CollectRecordSummary> it = recordSummaries.iterator();
    while (it.hasNext() && isRunning()) {
        CollectRecordSummary recordSummary = (CollectRecordSummary) it.next();
        udpateLastRecordModifiedDate(recordSummary);
        CollectRecord record = recordManager.load(survey, recordSummary.getId(), input.step, false);
        boolean recordCleansed = false;
        for (DataCleansingStep step : input.chain.getSteps()) {
            DataCleansingStepExecutionResult stepExecutionResult = executeStep(record, step);
            if (stepExecutionResult != null) {
                recordCleansed = true;
                if (stepExecutionResult == DataCleansingStepExecutionResult.RECORD_TO_BE_DELETED) {
                    recordIdsToBeDeleted.add(record.getId());
                    break;
                }
            }
        }
        if (recordCleansed) {
            cleansedRecords++;
        }
        incrementProcessedItems();
    }
    recordManager.deleteByIds(recordIdsToBeDeleted);
}
Also used : CollectRecord(org.openforis.collect.model.CollectRecord) TreeSet(java.util.TreeSet) CollectRecordSummary(org.openforis.collect.model.CollectRecordSummary) CollectSurvey(org.openforis.collect.model.CollectSurvey) RecordFilter(org.openforis.collect.model.RecordFilter) Transactional(org.springframework.transaction.annotation.Transactional)

Example 5 with RecordFilter

use of org.openforis.collect.model.RecordFilter in project collect by openforis.

the class DataExportService method fullExport.

@Transactional
public Proxy fullExport(CollectSurvey survey, boolean includeRecordFiles, boolean onlyOwnedRecords, String[] rootEntityKeyValues, boolean full) {
    if (backupJob == null || !backupJob.isRunning()) {
        resetJobs();
        RecordFilter filter = createRecordFilter(survey, null, onlyOwnedRecords, rootEntityKeyValues);
        SurveyBackupJob job = jobManager.createJob(SurveyBackupJob.class);
        job.setFull(full);
        if (full) {
            job.setOutputFormat(OutputFormat.DESKTOP_FULL);
        } else {
            job.setOutputFormat(OutputFormat.ONLY_DATA);
        }
        job.setSurvey(survey);
        job.setIncludeData(true);
        job.setIncludeRecordFiles(includeRecordFiles);
        job.setRecordFilter(filter);
        backupJob = job;
        jobManager.start(job);
    }
    return getCurrentJob();
}
Also used : SurveyBackupJob(org.openforis.collect.io.SurveyBackupJob) RecordFilter(org.openforis.collect.model.RecordFilter) Transactional(org.springframework.transaction.annotation.Transactional)

Aggregations

RecordFilter (org.openforis.collect.model.RecordFilter)66 CollectRecordSummary (org.openforis.collect.model.CollectRecordSummary)33 CollectSurvey (org.openforis.collect.model.CollectSurvey)27 CollectRecord (org.openforis.collect.model.CollectRecord)14 EntityDefinition (org.openforis.idm.metamodel.EntityDefinition)11 RequestMapping (org.springframework.web.bind.annotation.RequestMapping)11 Date (java.util.Date)10 User (org.openforis.collect.model.User)10 SessionState (org.openforis.collect.web.session.SessionState)7 Transactional (org.springframework.transaction.annotation.Transactional)7 ResponseBody (org.springframework.web.bind.annotation.ResponseBody)6 File (java.io.File)5 HashMap (java.util.HashMap)5 CSVDataExportParameters (org.openforis.collect.io.data.csv.CSVDataExportParameters)5 Step (org.openforis.collect.model.CollectRecord.Step)5 IOException (java.io.IOException)4 SurveyBackupJob (org.openforis.collect.io.SurveyBackupJob)4 CSVDataExportJob (org.openforis.collect.io.data.CSVDataExportJob)4 Schema (org.openforis.idm.metamodel.Schema)4 Test (org.junit.Test)3