Search in sources :

Example 1 with TransferForms

use of org.opendatakit.briefcase.transfer.TransferForms in project briefcase by opendatakit.

the class PullFormFromAggregate method pullFormFromAggregate.

public static void pullFormFromAggregate(Path storageDir, Optional<String> formId, String username, String password, URL server, boolean resumeLastPull, Optional<LocalDate> startFromDate, boolean includeIncomplete, Optional<Integer> maybeMaxHttpConnections) {
    CliEventsCompanion.attach(log);
    Path briefcaseDir = Common.getOrCreateBriefcaseDir(storageDir);
    FormCache formCache = FormCache.from(briefcaseDir);
    formCache.update();
    BriefcasePreferences appPreferences = BriefcasePreferences.appScoped();
    FormMetadataPort formMetadataPort = FileSystemFormMetadataAdapter.at(briefcaseDir);
    int maxHttpConnections = Optionals.race(maybeMaxHttpConnections, appPreferences.getMaxHttpConnections()).orElse(DEFAULT_HTTP_CONNECTIONS);
    Http http = appPreferences.getHttpProxy().map(host -> CommonsHttp.of(maxHttpConnections, host)).orElseGet(() -> CommonsHttp.of(maxHttpConnections));
    AggregateServer aggregateServer = AggregateServer.authenticated(server, new Credentials(username, password));
    Response<List<RemoteFormDefinition>> response = http.execute(aggregateServer.getFormListRequest());
    if (!response.isSuccess()) {
        System.err.println(response.isRedirection() ? "Error connecting to Aggregate: Redirection detected" : response.isUnauthorized() ? "Error connecting to Aggregate: Wrong credentials" : response.isNotFound() ? "Error connecting to Aggregate: Aggregate not found" : "Error connecting to Aggregate");
        return;
    }
    List<FormStatus> filteredForms = response.orElseThrow(BriefcaseException::new).stream().filter(f -> formId.map(id -> f.getFormId().equals(id)).orElse(true)).map(FormStatus::new).collect(toList());
    if (formId.isPresent() && filteredForms.isEmpty())
        throw new BriefcaseException("Form " + formId.get() + " not found");
    TransferForms forms = TransferForms.empty();
    forms.load(filteredForms);
    forms.selectAll();
    PullFromAggregate pullOp = new PullFromAggregate(http, aggregateServer, briefcaseDir, includeIncomplete, PullFormFromAggregate::onEvent, formMetadataPort);
    JobsRunner.launchAsync(forms.map(form -> pullOp.pull(form, resolveCursor(resumeLastPull, startFromDate, form, formMetadataPort))), PullFormFromAggregate::onError).waitForCompletion();
    System.out.println();
    System.out.println("All operations completed");
    System.out.println();
}
Also used : Path(java.nio.file.Path) Arrays(java.util.Arrays) FileSystemFormMetadataAdapter(org.opendatakit.briefcase.model.form.FileSystemFormMetadataAdapter) Credentials(org.opendatakit.briefcase.reused.http.Credentials) URL(java.net.URL) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) Optionals(org.opendatakit.briefcase.reused.Optionals) Response(org.opendatakit.briefcase.reused.http.response.Response) LoggerFactory(org.slf4j.LoggerFactory) DEFAULT_HTTP_CONNECTIONS(org.opendatakit.briefcase.reused.http.Http.DEFAULT_HTTP_CONNECTIONS) MAX_HTTP_CONNECTIONS(org.opendatakit.briefcase.operations.Common.MAX_HTTP_CONNECTIONS) FORM_ID(org.opendatakit.briefcase.operations.Common.FORM_ID) Operation(org.opendatakit.common.cli.Operation) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) AggregateServer(org.opendatakit.briefcase.reused.transfer.AggregateServer) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) SERVER_URL(org.opendatakit.briefcase.operations.Common.SERVER_URL) CREDENTIALS_PASSWORD(org.opendatakit.briefcase.operations.Common.CREDENTIALS_PASSWORD) FormCache(org.opendatakit.briefcase.util.FormCache) Path(java.nio.file.Path) FormKey(org.opendatakit.briefcase.model.form.FormKey) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) RemoteFormDefinition(org.opendatakit.briefcase.model.RemoteFormDefinition) FormStatus(org.opendatakit.briefcase.model.FormStatus) FormStatusEvent(org.opendatakit.briefcase.model.FormStatusEvent) Logger(org.slf4j.Logger) FormMetadataQueries.lastCursorOf(org.opendatakit.briefcase.model.form.FormMetadataQueries.lastCursorOf) Param(org.opendatakit.common.cli.Param) Cursor(org.opendatakit.briefcase.pull.aggregate.Cursor) STORAGE_DIR(org.opendatakit.briefcase.operations.Common.STORAGE_DIR) JobsRunner(org.opendatakit.briefcase.reused.job.JobsRunner) Http(org.opendatakit.briefcase.reused.http.Http) CREDENTIALS_USERNAME(org.opendatakit.briefcase.operations.Common.CREDENTIALS_USERNAME) Collectors.toList(java.util.stream.Collectors.toList) List(java.util.List) PullFromAggregate(org.opendatakit.briefcase.pull.aggregate.PullFromAggregate) LocalDate(java.time.LocalDate) Optional(java.util.Optional) FormCache(org.opendatakit.briefcase.util.FormCache) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) Http(org.opendatakit.briefcase.reused.http.Http) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) PullFromAggregate(org.opendatakit.briefcase.pull.aggregate.PullFromAggregate) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) FormStatus(org.opendatakit.briefcase.model.FormStatus) Collectors.toList(java.util.stream.Collectors.toList) List(java.util.List) AggregateServer(org.opendatakit.briefcase.reused.transfer.AggregateServer) Credentials(org.opendatakit.briefcase.reused.http.Credentials)

Example 2 with TransferForms

use of org.opendatakit.briefcase.transfer.TransferForms in project briefcase by opendatakit.

the class ImportFromODK method importODK.

public static void importODK(Path storageDir, Path odkDir, Optional<String> formId) {
    CliEventsCompanion.attach(log);
    Path briefcaseDir = Common.getOrCreateBriefcaseDir(storageDir);
    FormCache formCache = FormCache.from(briefcaseDir);
    formCache.update();
    FormMetadataPort formMetadataPort = FileSystemFormMetadataAdapter.at(briefcaseDir);
    TransferForms forms = TransferForms.from(FileSystemUtils.getODKFormList(odkDir.toFile()).stream().map(FormStatus::new).filter(form -> formId.map(id -> form.getFormDefinition().getFormId().equals(id)).orElse(true)).collect(toList()));
    forms.selectAll();
    if (formId.isPresent() && forms.isEmpty())
        throw new BriefcaseException("Form " + formId.get() + " not found");
    PullFromCollect.pullForms(formMetadataPort, forms, briefcaseDir, odkDir, ImportFromODK::onEvent).waitForCompletion();
}
Also used : Path(java.nio.file.Path) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) FormCache(org.opendatakit.briefcase.util.FormCache) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) TransferForms(org.opendatakit.briefcase.transfer.TransferForms)

Example 3 with TransferForms

use of org.opendatakit.briefcase.transfer.TransferForms in project briefcase by opendatakit.

the class PullFormFromCentral method pullFromCentral.

private static void pullFromCentral(Args args) {
    CliEventsCompanion.attach(log);
    Path briefcaseDir = Common.getOrCreateBriefcaseDir(args.get(STORAGE_DIR));
    FormCache formCache = FormCache.from(briefcaseDir);
    formCache.update();
    BriefcasePreferences appPreferences = BriefcasePreferences.appScoped();
    FormMetadataPort formMetadataPort = FileSystemFormMetadataAdapter.at(briefcaseDir);
    int maxHttpConnections = Optionals.race(args.getOptional(MAX_HTTP_CONNECTIONS), appPreferences.getMaxHttpConnections()).orElse(DEFAULT_HTTP_CONNECTIONS);
    Http http = appPreferences.getHttpProxy().map(host -> CommonsHttp.of(maxHttpConnections, host)).orElseGet(() -> CommonsHttp.of(maxHttpConnections));
    CentralServer server = CentralServer.of(args.get(SERVER_URL), args.get(PROJECT_ID), new Credentials(args.get(CREDENTIALS_EMAIL), args.get(CREDENTIALS_PASSWORD)));
    String token = http.execute(server.getSessionTokenRequest()).orElseThrow(() -> new BriefcaseException("Can't authenticate with ODK Central"));
    Response<List<RemoteFormDefinition>> response = http.execute(server.getFormsListRequest(token));
    if (!response.isSuccess()) {
        System.err.println(response.isRedirection() ? "Error connecting to Central: Redirection detected" : response.isUnauthorized() ? "Error connecting to Central: Wrong credentials" : response.isNotFound() ? "Error connecting to Central: Central not found" : "Error connecting to Central");
        return;
    }
    Optional<String> formId = args.getOptional(FORM_ID);
    List<FormStatus> filteredForms = response.orElseThrow(BriefcaseException::new).stream().filter(f -> formId.map(id -> f.getFormId().equals(id)).orElse(true)).map(FormStatus::new).collect(toList());
    if (formId.isPresent() && filteredForms.isEmpty())
        throw new BriefcaseException("Form " + formId.get() + " not found");
    TransferForms forms = TransferForms.empty();
    forms.load(filteredForms);
    forms.selectAll();
    PullFromCentral pullOp = new PullFromCentral(http, server, briefcaseDir, token, PullFormFromCentral::onEvent, formMetadataPort);
    JobsRunner.launchAsync(forms.map(pullOp::pull), PullFormFromCentral::onError).waitForCompletion();
    System.out.println();
    System.out.println("All operations completed");
    System.out.println();
}
Also used : Path(java.nio.file.Path) Arrays(java.util.Arrays) FileSystemFormMetadataAdapter(org.opendatakit.briefcase.model.form.FileSystemFormMetadataAdapter) Credentials(org.opendatakit.briefcase.reused.http.Credentials) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) Optionals(org.opendatakit.briefcase.reused.Optionals) Response(org.opendatakit.briefcase.reused.http.response.Response) LoggerFactory(org.slf4j.LoggerFactory) DEFAULT_HTTP_CONNECTIONS(org.opendatakit.briefcase.reused.http.Http.DEFAULT_HTTP_CONNECTIONS) MAX_HTTP_CONNECTIONS(org.opendatakit.briefcase.operations.Common.MAX_HTTP_CONNECTIONS) FORM_ID(org.opendatakit.briefcase.operations.Common.FORM_ID) Operation(org.opendatakit.common.cli.Operation) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) SERVER_URL(org.opendatakit.briefcase.operations.Common.SERVER_URL) CREDENTIALS_PASSWORD(org.opendatakit.briefcase.operations.Common.CREDENTIALS_PASSWORD) FormCache(org.opendatakit.briefcase.util.FormCache) Path(java.nio.file.Path) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) RemoteFormDefinition(org.opendatakit.briefcase.model.RemoteFormDefinition) FormStatus(org.opendatakit.briefcase.model.FormStatus) FormStatusEvent(org.opendatakit.briefcase.model.FormStatusEvent) CentralServer(org.opendatakit.briefcase.reused.transfer.CentralServer) Logger(org.slf4j.Logger) Param(org.opendatakit.common.cli.Param) STORAGE_DIR(org.opendatakit.briefcase.operations.Common.STORAGE_DIR) JobsRunner(org.opendatakit.briefcase.reused.job.JobsRunner) Http(org.opendatakit.briefcase.reused.http.Http) PullFromCentral(org.opendatakit.briefcase.pull.central.PullFromCentral) Collectors.toList(java.util.stream.Collectors.toList) PROJECT_ID(org.opendatakit.briefcase.operations.Common.PROJECT_ID) List(java.util.List) CREDENTIALS_EMAIL(org.opendatakit.briefcase.operations.Common.CREDENTIALS_EMAIL) Optional(java.util.Optional) Args(org.opendatakit.common.cli.Args) FormCache(org.opendatakit.briefcase.util.FormCache) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) PullFromCentral(org.opendatakit.briefcase.pull.central.PullFromCentral) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) Http(org.opendatakit.briefcase.reused.http.Http) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) FormStatus(org.opendatakit.briefcase.model.FormStatus) CentralServer(org.opendatakit.briefcase.reused.transfer.CentralServer) Collectors.toList(java.util.stream.Collectors.toList) List(java.util.List) Credentials(org.opendatakit.briefcase.reused.http.Credentials)

Example 4 with TransferForms

use of org.opendatakit.briefcase.transfer.TransferForms in project briefcase by opendatakit.

the class Central method push.

@Override
public JobsRunner push(TransferForms forms, Path briefcaseDir) {
    forms.filter(FormStatus::isEncrypted).forEach(form -> form.setStatusString("Skipping. Encrypted forms can't be pushed to ODK Central yet"));
    String token = http.execute(server.getSessionTokenRequest()).orElseThrow(() -> new BriefcaseException("Can't authenticate with ODK Central"));
    PushToCentral pushOp = new PushToCentral(http, server, briefcaseDir, token, EventBus::publish);
    return JobsRunner.launchAsync(forms.filter(f -> !f.isEncrypted()).map(pushOp::push)).onComplete(() -> EventBus.publish(new PushEvent.Complete()));
}
Also used : FileSystemFormMetadataAdapter(org.opendatakit.briefcase.model.form.FileSystemFormMetadataAdapter) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) PushToCentral(org.opendatakit.briefcase.push.central.PushToCentral) HashSet(java.util.HashSet) FormMetadataPort(org.opendatakit.briefcase.model.form.FormMetadataPort) EventBus(org.bushe.swing.event.EventBus) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) CentralServerDialog(org.opendatakit.briefcase.ui.reused.transfer.sourcetarget.CentralServerDialog) Container(java.awt.Container) UI.uncheckedBrowse(org.opendatakit.briefcase.ui.reused.UI.uncheckedBrowse) Path(java.nio.file.Path) FormKey(org.opendatakit.briefcase.model.form.FormKey) FormMetadataQueries.submissionVersionsOf(org.opendatakit.briefcase.model.form.FormMetadataQueries.submissionVersionsOf) Test(org.opendatakit.briefcase.reused.transfer.RemoteServer.Test) FormStatus(org.opendatakit.briefcase.model.FormStatus) CentralServer(org.opendatakit.briefcase.reused.transfer.CentralServer) JobsRunner(org.opendatakit.briefcase.reused.job.JobsRunner) Set(java.util.Set) Http(org.opendatakit.briefcase.reused.http.Http) Consumer(java.util.function.Consumer) PushEvent(org.opendatakit.briefcase.push.PushEvent) JLabel(javax.swing.JLabel) UI.makeClickable(org.opendatakit.briefcase.ui.reused.UI.makeClickable) Optional(java.util.Optional) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) PushToCentral(org.opendatakit.briefcase.push.central.PushToCentral) EventBus(org.bushe.swing.event.EventBus)

Example 5 with TransferForms

use of org.opendatakit.briefcase.transfer.TransferForms in project briefcase by opendatakit.

the class PushFormToAggregate method pushFormToAggregate.

private static void pushFormToAggregate(Path storageDir, Optional<String> formid, String username, String password, URL server, boolean forceSendBlank, Optional<Integer> maybeMaxConnections) {
    CliEventsCompanion.attach(log);
    Path briefcaseDir = Common.getOrCreateBriefcaseDir(storageDir);
    FormCache formCache = FormCache.from(briefcaseDir);
    formCache.update();
    BriefcasePreferences appPreferences = BriefcasePreferences.appScoped();
    int maxHttpConnections = Optionals.race(maybeMaxConnections, appPreferences.getMaxHttpConnections()).orElse(DEFAULT_HTTP_CONNECTIONS);
    Http http = appPreferences.getHttpProxy().map(host -> CommonsHttp.of(maxHttpConnections, host)).orElseGet(() -> CommonsHttp.of(maxHttpConnections));
    AggregateServer aggregateServer = AggregateServer.authenticated(server, new Credentials(username, password));
    Response response = http.execute(aggregateServer.getPushFormPreflightRequest());
    if (!response.isSuccess()) {
        System.err.println(response.isRedirection() ? "Error connecting to Aggregate: Redirection detected" : response.isUnauthorized() ? "Error connecting to Aggregate: Wrong credentials" : response.isNotFound() ? "Error connecting to Aggregate: Aggregate not found" : "Error connecting to Aggregate");
        return;
    }
    List<FormStatus> statuses;
    if (formid.isPresent()) {
        String requestedFormId = formid.get();
        FormStatus status = formCache.getForms().stream().filter(form -> form.getFormId().equals(requestedFormId)).map(FormStatus::new).findFirst().orElseThrow(() -> new BriefcaseException("Form " + requestedFormId + " not found"));
        statuses = Arrays.asList(status);
    } else {
        statuses = formCache.getForms().stream().map(FormStatus::new).collect(Collectors.toList());
    }
    TransferForms forms = TransferForms.of(statuses);
    forms.selectAll();
    PushToAggregate pushOp = new PushToAggregate(http, aggregateServer, briefcaseDir, forceSendBlank, PushFormToAggregate::onEvent);
    JobsRunner.launchAsync(forms.map(pushOp::push), PushFormToAggregate::onError).waitForCompletion();
    System.out.println();
    System.out.println("All operations completed");
    System.out.println();
}
Also used : Path(java.nio.file.Path) Arrays(java.util.Arrays) Credentials(org.opendatakit.briefcase.reused.http.Credentials) URL(java.net.URL) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) Optionals(org.opendatakit.briefcase.reused.Optionals) Response(org.opendatakit.briefcase.reused.http.response.Response) LoggerFactory(org.slf4j.LoggerFactory) DEFAULT_HTTP_CONNECTIONS(org.opendatakit.briefcase.reused.http.Http.DEFAULT_HTTP_CONNECTIONS) MAX_HTTP_CONNECTIONS(org.opendatakit.briefcase.operations.Common.MAX_HTTP_CONNECTIONS) FORM_ID(org.opendatakit.briefcase.operations.Common.FORM_ID) Operation(org.opendatakit.common.cli.Operation) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) AggregateServer(org.opendatakit.briefcase.reused.transfer.AggregateServer) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) SERVER_URL(org.opendatakit.briefcase.operations.Common.SERVER_URL) CREDENTIALS_PASSWORD(org.opendatakit.briefcase.operations.Common.CREDENTIALS_PASSWORD) FormCache(org.opendatakit.briefcase.util.FormCache) Path(java.nio.file.Path) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) PushToAggregate(org.opendatakit.briefcase.push.aggregate.PushToAggregate) FormStatus(org.opendatakit.briefcase.model.FormStatus) FormStatusEvent(org.opendatakit.briefcase.model.FormStatusEvent) Logger(org.slf4j.Logger) Param(org.opendatakit.common.cli.Param) STORAGE_DIR(org.opendatakit.briefcase.operations.Common.STORAGE_DIR) JobsRunner(org.opendatakit.briefcase.reused.job.JobsRunner) Collectors(java.util.stream.Collectors) Http(org.opendatakit.briefcase.reused.http.Http) CREDENTIALS_USERNAME(org.opendatakit.briefcase.operations.Common.CREDENTIALS_USERNAME) List(java.util.List) Optional(java.util.Optional) FormCache(org.opendatakit.briefcase.util.FormCache) BriefcasePreferences(org.opendatakit.briefcase.model.BriefcasePreferences) CommonsHttp(org.opendatakit.briefcase.reused.http.CommonsHttp) Http(org.opendatakit.briefcase.reused.http.Http) Response(org.opendatakit.briefcase.reused.http.response.Response) PushToAggregate(org.opendatakit.briefcase.push.aggregate.PushToAggregate) BriefcaseException(org.opendatakit.briefcase.reused.BriefcaseException) TransferForms(org.opendatakit.briefcase.transfer.TransferForms) FormStatus(org.opendatakit.briefcase.model.FormStatus) AggregateServer(org.opendatakit.briefcase.reused.transfer.AggregateServer) Credentials(org.opendatakit.briefcase.reused.http.Credentials)

Aggregations

Path (java.nio.file.Path)6 BriefcaseException (org.opendatakit.briefcase.reused.BriefcaseException)6 TransferForms (org.opendatakit.briefcase.transfer.TransferForms)6 BriefcasePreferences (org.opendatakit.briefcase.model.BriefcasePreferences)5 FormStatus (org.opendatakit.briefcase.model.FormStatus)5 Http (org.opendatakit.briefcase.reused.http.Http)5 JobsRunner (org.opendatakit.briefcase.reused.job.JobsRunner)5 FormCache (org.opendatakit.briefcase.util.FormCache)5 Arrays (java.util.Arrays)4 List (java.util.List)4 Optional (java.util.Optional)4 FormStatusEvent (org.opendatakit.briefcase.model.FormStatusEvent)4 FormMetadataPort (org.opendatakit.briefcase.model.form.FormMetadataPort)4 CREDENTIALS_PASSWORD (org.opendatakit.briefcase.operations.Common.CREDENTIALS_PASSWORD)4 FORM_ID (org.opendatakit.briefcase.operations.Common.FORM_ID)4 MAX_HTTP_CONNECTIONS (org.opendatakit.briefcase.operations.Common.MAX_HTTP_CONNECTIONS)4 SERVER_URL (org.opendatakit.briefcase.operations.Common.SERVER_URL)4 STORAGE_DIR (org.opendatakit.briefcase.operations.Common.STORAGE_DIR)4 Optionals (org.opendatakit.briefcase.reused.Optionals)4 CommonsHttp (org.opendatakit.briefcase.reused.http.CommonsHttp)4