Search in sources :

Example 11 with FormStatus

use of org.opendatakit.briefcase.model.FormStatus in project briefcase by opendatakit.

the class ServerUploader method uploadFormAndSubmissionFiles.

public boolean uploadFormAndSubmissionFiles(List<FormStatus> formsToTransfer) {
    boolean allSuccessful = true;
    for (FormStatus formToTransfer : formsToTransfer) {
        BriefcaseFormDefinition briefcaseLfd = (BriefcaseFormDefinition) formToTransfer.getFormDefinition();
        boolean thisFormSuccessful = true;
        if (isCancelled()) {
            formToTransfer.setStatusString("Aborted upload.", true);
            EventBus.publish(new FormStatusEvent(formToTransfer));
            return false;
        }
        if (!formToTransfer.isSuccessful()) {
            formToTransfer.setStatusString("Skipping upload -- download failed", false);
            EventBus.publish(new FormStatusEvent(formToTransfer));
            continue;
        }
        File briefcaseFormDefFile = FileSystemUtils.getFormDefinitionFileIfExists(briefcaseLfd.getFormDirectory());
        if (briefcaseFormDefFile == null) {
            formToTransfer.setStatusString("Form does not exist", true);
            EventBus.publish(new FormStatusEvent(formToTransfer));
            continue;
        }
        File briefcaseFormMediaDir = FileSystemUtils.getMediaDirectoryIfExists(briefcaseLfd.getFormDirectory());
        boolean outcome;
        outcome = uploadForm(formToTransfer, briefcaseFormDefFile, briefcaseFormMediaDir);
        thisFormSuccessful = thisFormSuccessful & outcome;
        allSuccessful = allSuccessful & outcome;
        URI u = getUploadSubmissionUri(formToTransfer);
        if (u == null) {
            // error already logged...
            continue;
        }
        Set<File> briefcaseInstances = FileSystemUtils.getFormSubmissionDirectories(briefcaseLfd.getFormDirectory());
        DatabaseUtils formDatabase = null;
        try {
            formDatabase = DatabaseUtils.newInstance(briefcaseLfd.getFormDirectory());
            // make sure all the local instances are in the database...
            formDatabase.updateInstanceLists(briefcaseInstances);
            // exclude submissions the server reported as already submitted
            subtractServerInstances(formToTransfer, formDatabase, briefcaseInstances);
            int i = 1;
            for (File briefcaseInstance : briefcaseInstances) {
                outcome = uploadSubmission(formDatabase, formToTransfer, u, i++, briefcaseInstances.size(), briefcaseInstance);
                thisFormSuccessful = thisFormSuccessful & outcome;
                allSuccessful = allSuccessful & outcome;
                // and stop this loop quickly if we're cancelled...
                if (isCancelled()) {
                    break;
                }
            }
        } catch (SQLException | FileSystemException e) {
            thisFormSuccessful = false;
            allSuccessful = false;
            String msg = "unable to open form database";
            log.error(msg, e);
            formToTransfer.setStatusString(msg + ": " + e.getMessage(), false);
            EventBus.publish(new FormStatusEvent(formToTransfer));
        } finally {
            if (formDatabase != null) {
                try {
                    formDatabase.close();
                } catch (SQLException e) {
                    thisFormSuccessful = false;
                    allSuccessful = false;
                    String msg = "unable to close form database";
                    log.warn(msg, e);
                    formToTransfer.setStatusString(msg + ": " + e.getMessage(), false);
                    EventBus.publish(new FormStatusEvent(formToTransfer));
                }
            }
        }
        if (isCancelled()) {
            formToTransfer.setStatusString("Aborted upload.", true);
            EventBus.publish(new FormStatusEvent(formToTransfer));
        } else if (thisFormSuccessful) {
            formToTransfer.setStatusString("Successful upload!", true);
            EventBus.publish(new FormStatusEvent(formToTransfer));
        } else {
            formToTransfer.setStatusString("Partially successful upload...", true);
            EventBus.publish(new FormStatusEvent(formToTransfer));
        }
    }
    return allSuccessful;
}
Also used : SQLException(java.sql.SQLException) FormStatusEvent(org.opendatakit.briefcase.model.FormStatusEvent) URI(java.net.URI) FileSystemException(org.opendatakit.briefcase.model.FileSystemException) FormStatus(org.opendatakit.briefcase.model.FormStatus) BriefcaseFormDefinition(org.opendatakit.briefcase.model.BriefcaseFormDefinition) File(java.io.File)

Example 12 with FormStatus

use of org.opendatakit.briefcase.model.FormStatus in project briefcase by opendatakit.

the class TransferToServer method push.

public static void push(ServerConnectionInfo transferSettings, FormStatus... forms) {
    List<FormStatus> formList = Arrays.asList(forms);
    TransferToServer action = new TransferToServer(transferSettings, new TerminationFuture(), formList);
    try {
        boolean allSuccessful = action.doAction();
        if (allSuccessful)
            EventBus.publish(new TransferSucceededEvent(false, formList, transferSettings));
        if (!allSuccessful)
            throw new PushFromServerException(formList);
    } catch (Exception e) {
        EventBus.publish(new TransferFailedEvent(false, formList));
        throw new PushFromServerException(formList);
    }
}
Also used : TransferFailedEvent(org.opendatakit.briefcase.model.TransferFailedEvent) FormStatus(org.opendatakit.briefcase.model.FormStatus) TerminationFuture(org.opendatakit.briefcase.model.TerminationFuture) TransferSucceededEvent(org.opendatakit.briefcase.model.TransferSucceededEvent)

Example 13 with FormStatus

use of org.opendatakit.briefcase.model.FormStatus in project briefcase by opendatakit.

the class ServerFetcher method downloadFormAndSubmissionFiles.

public boolean downloadFormAndSubmissionFiles(List<FormStatus> formsToTransfer) {
    boolean allSuccessful = true;
    // boolean error = false;
    int total = formsToTransfer.size();
    for (int i = 0; i < total; i++) {
        FormStatus fs = formsToTransfer.get(i);
        if (isCancelled()) {
            fs.setStatusString("Aborted. Skipping fetch of form and submissions...", true);
            EventBus.publish(new FormStatusEvent(fs));
            return false;
        }
        RemoteFormDefinition fd = getRemoteFormDefinition(fs);
        fs.setStatusString("Fetching form definition", true);
        EventBus.publish(new FormStatusEvent(fs));
        try {
            File tmpdl = FileSystemUtils.getTempFormDefinitionFile();
            AggregateUtils.commonDownloadFile(serverInfo, tmpdl, fd.getDownloadUrl());
            fs.setStatusString("resolving against briefcase form definitions", true);
            EventBus.publish(new FormStatusEvent(fs));
            boolean successful = false;
            BriefcaseFormDefinition briefcaseLfd;
            DatabaseUtils formDatabase = null;
            try {
                try {
                    briefcaseLfd = BriefcaseFormDefinition.resolveAgainstBriefcaseDefn(tmpdl);
                    if (briefcaseLfd.needsMediaUpdate()) {
                        if (fd.getManifestUrl() != null) {
                            File mediaDir = FileSystemUtils.getMediaDirectory(briefcaseLfd.getFormDirectory());
                            String error = downloadManifestAndMediaFiles(mediaDir, fs);
                            if (error != null) {
                                allSuccessful = false;
                                fs.setStatusString("Error fetching form definition: " + error, false);
                                EventBus.publish(new FormStatusEvent(fs));
                                continue;
                            }
                        }
                    }
                    formDatabase = DatabaseUtils.newInstance(briefcaseLfd.getFormDirectory());
                } catch (BadFormDefinition e) {
                    allSuccessful = false;
                    String msg = "Error parsing form definition";
                    log.error(msg, e);
                    fs.setStatusString(msg + ": " + e.getMessage(), false);
                    EventBus.publish(new FormStatusEvent(fs));
                    continue;
                }
                fs.setStatusString("preparing to retrieve instance data", true);
                EventBus.publish(new FormStatusEvent(fs));
                File formInstancesDir = FileSystemUtils.getFormInstancesDirectory(briefcaseLfd.getFormDirectory());
                // this will publish events
                successful = downloadAllSubmissionsForForm(formInstancesDir, formDatabase, briefcaseLfd, fs);
            } catch (SQLException | FileSystemException e) {
                allSuccessful = false;
                String msg = "unable to open form database";
                log.error(msg, e);
                fs.setStatusString(msg + ": " + e.getMessage(), false);
                EventBus.publish(new FormStatusEvent(fs));
                continue;
            } finally {
                if (formDatabase != null) {
                    try {
                        formDatabase.close();
                    } catch (SQLException e) {
                        allSuccessful = false;
                        String msg = "unable to close form database";
                        log.error(msg, e);
                        fs.setStatusString(msg + ": " + e.getMessage(), false);
                        EventBus.publish(new FormStatusEvent(fs));
                        continue;
                    }
                }
            }
            allSuccessful = allSuccessful && successful;
            // on success, we haven't actually set a success event (because we don't know we're done)
            if (successful) {
                fs.setStatusString(SUCCESS_STATUS, true);
                EventBus.publish(new FormStatusEvent(fs));
            } else {
                fs.setStatusString(FAILED_STATUS, true);
                EventBus.publish(new FormStatusEvent(fs));
            }
        } catch (SocketTimeoutException se) {
            allSuccessful = false;
            log.error("error accessing URL", se);
            fs.setStatusString("Communications to the server timed out. Detailed message: " + se.getLocalizedMessage() + " while accessing: " + fd.getDownloadUrl() + " A network login screen may be interfering with the transmission to the server.", false);
            EventBus.publish(new FormStatusEvent(fs));
        } catch (IOException e) {
            allSuccessful = false;
            log.error("error accessing form download URL", e);
            fs.setStatusString("Unexpected error: " + e.getLocalizedMessage() + " while accessing: " + fd.getDownloadUrl() + " A network login screen may be interfering with the transmission to the server.", false);
            EventBus.publish(new FormStatusEvent(fs));
        } catch (FileSystemException | TransmissionException | URISyntaxException e) {
            allSuccessful = false;
            log.error("error accessing form download URL", e);
            fs.setStatusString("Unexpected error: " + e.getLocalizedMessage() + " while accessing: " + fd.getDownloadUrl(), false);
            EventBus.publish(new FormStatusEvent(fs));
        }
    }
    return allSuccessful;
}
Also used : SQLException(java.sql.SQLException) FormStatusEvent(org.opendatakit.briefcase.model.FormStatusEvent) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) FileSystemException(org.opendatakit.briefcase.model.FileSystemException) SocketTimeoutException(java.net.SocketTimeoutException) RemoteFormDefinition(org.opendatakit.briefcase.model.RemoteFormDefinition) TransmissionException(org.opendatakit.briefcase.model.TransmissionException) FormStatus(org.opendatakit.briefcase.model.FormStatus) BriefcaseFormDefinition(org.opendatakit.briefcase.model.BriefcaseFormDefinition) File(java.io.File)

Example 14 with FormStatus

use of org.opendatakit.briefcase.model.FormStatus in project briefcase by opendatakit.

the class ExportPanelUnitTest method saves_to_user_preferences_the_last_successful_export_date_for_a_form.

@Test
public void saves_to_user_preferences_the_last_successful_export_date_for_a_form() {
    BriefcasePreferences exportPreferences = new BriefcasePreferences(InMemoryPreferences.empty());
    BriefcasePreferences appPreferences = new BriefcasePreferences(InMemoryPreferences.empty());
    List<FormStatus> formsList = FormStatusBuilder.buildFormStatusList(10);
    initialDefaultConf = ExportConfiguration.empty();
    ExportForms forms = load(initialDefaultConf, formsList, exportPreferences, appPreferences);
    ConfigurationPanel confPanel = ConfigurationPanel.defaultPanel(initialDefaultConf, true, true);
    new ExportPanel(new TerminationFuture(), forms, ExportPanelForm.from(forms, confPanel), exportPreferences, Runnable::run, new NoOpAnalytics());
    FormStatus form = formsList.get(0);
    String formId = form.getFormDefinition().getFormId();
    assertThat(exportPreferences.nullSafeGet(buildExportDateTimePrefix(formId)), isEmpty());
    forms.appendStatus(form.getFormDefinition(), "some status update", true);
    assertThat(exportPreferences.nullSafeGet(buildExportDateTimePrefix(formId)), isPresent());
}
Also used : ExportForms(org.opendatakit.briefcase.export.ExportForms) NoOpAnalytics(org.opendatakit.briefcase.ui.reused.NoOpAnalytics) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) FormStatus(org.opendatakit.briefcase.model.FormStatus) ConfigurationPanel(org.opendatakit.briefcase.ui.export.components.ConfigurationPanel) TerminationFuture(org.opendatakit.briefcase.model.TerminationFuture) Test(org.junit.Test)

Example 15 with FormStatus

use of org.opendatakit.briefcase.model.FormStatus in project briefcase by opendatakit.

the class ExportPanelUnitTest method saves_to_user_preferences_changes_on_a_custom_configuration.

@Test
public void saves_to_user_preferences_changes_on_a_custom_configuration() throws IOException {
    BriefcasePreferences exportPreferences = new BriefcasePreferences(InMemoryPreferences.empty());
    BriefcasePreferences appPreferences = new BriefcasePreferences(InMemoryPreferences.empty());
    List<FormStatus> formsList = FormStatusBuilder.buildFormStatusList(10);
    initialDefaultConf = ExportConfiguration.empty();
    ExportForms forms = load(initialDefaultConf, formsList, exportPreferences, appPreferences);
    ConfigurationPanel confPanel = ConfigurationPanel.defaultPanel(initialDefaultConf, true, true);
    ExportPanelForm exportPanelForm = ExportPanelForm.from(forms, confPanel);
    new ExportPanel(new TerminationFuture(), forms, exportPanelForm, exportPreferences, Runnable::run, new NoOpAnalytics());
    FormStatus form = formsList.get(0);
    String formId = form.getFormDefinition().getFormId();
    ExportConfiguration conf = ExportConfiguration.empty();
    conf.setExportDir(Paths.get(Files.createTempDirectory("briefcase_test").toUri()));
    assertThat(ExportConfiguration.load(exportPreferences, buildCustomConfPrefix(formId)).getExportDir(), isEmpty());
    forms.putConfiguration(form, conf);
    exportPanelForm.getFormsTable().getViewModel().triggerChange();
    assertThat(ExportConfiguration.load(exportPreferences, buildCustomConfPrefix(formId)).getExportDir(), isPresent());
}
Also used : ExportForms(org.opendatakit.briefcase.export.ExportForms) NoOpAnalytics(org.opendatakit.briefcase.ui.reused.NoOpAnalytics) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) FormStatus(org.opendatakit.briefcase.model.FormStatus) ExportConfiguration(org.opendatakit.briefcase.export.ExportConfiguration) ConfigurationPanel(org.opendatakit.briefcase.ui.export.components.ConfigurationPanel) TerminationFuture(org.opendatakit.briefcase.model.TerminationFuture) Test(org.junit.Test)

Aggregations

FormStatus (org.opendatakit.briefcase.model.FormStatus)22 Test (org.junit.Test)9 FormStatusBuilder.buildFormStatus (org.opendatakit.briefcase.model.FormStatusBuilder.buildFormStatus)7 BriefcaseFormDefinition (org.opendatakit.briefcase.model.BriefcaseFormDefinition)6 File (java.io.File)4 ExportForms (org.opendatakit.briefcase.export.ExportForms)4 BriefcasePreferences (org.opendatakit.briefcase.model.BriefcasePreferences)4 TerminationFuture (org.opendatakit.briefcase.model.TerminationFuture)4 SQLException (java.sql.SQLException)3 LocalDateTime (java.time.LocalDateTime)3 FileSystemException (org.opendatakit.briefcase.model.FileSystemException)3 FormStatusEvent (org.opendatakit.briefcase.model.FormStatusEvent)3 ServerConnectionInfo (org.opendatakit.briefcase.model.ServerConnectionInfo)3 IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)2 Arrays (java.util.Arrays)2 List (java.util.List)2 Optional (java.util.Optional)2 Collectors.toList (java.util.stream.Collectors.toList)2 ExportConfiguration (org.opendatakit.briefcase.export.ExportConfiguration)2