Curl : Submission to ODKAggregate

Janusma_Caben · November 26, 2015, 6:36am

Greetings.

Currently doing a project using OKDAggregate Server, to use "curl" to
submit form data to Aggregate.
Did read all wiki pages:
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI
https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI
and related wiki pages but no solution.
After some web search and figured out how to use "curl -F" to submit xml
text file to Aggregate.
Just guess it's not so difficult to submit associated file such as jpg
image file to Aggregate.
However, still can't do that after so many tries.

For example image file = picture.jpg
Did try:
curl -F "image=@picture.jpg" "[AggregateServerURL]/submission", not working.
curl -F "media=@picture.jpg" "[AggregateServerURL]/submission", not working.
curl -F "binary=@picture.jpg" "[AggregateServerURL]/submission", not
working.
Also tried:
curl -d --data-binary "file:///[location]/picture.jpg"
"[AggregateServerURL]/submission", not working.

Any help will be appreciated.
Cheers,
Janus

Janusma_Caben · November 26, 2015, 8:21am

Hi Yaw,
Thanks for your quick reply and appreciate you help.
Sorry for repost, since I'm using a low speed RDP remote.

Thanks for your advice. Yes, look at the Collect source code is really a
good idea, although it's a hard job.
If I can't figure that, I will definite do that or even tcpdump to find out.

However, just just before I reply this, I figured out how to do that
luckily.
After I look at
(1) [OKDAggregateURL]/submission webpage with manual upload interface, and
(2) read "javarosa / FormSUbmissionAPI" wiki again, especially below:

Additional Parts

Content Type: arbitrary
Name: matches an appropriate element inside of the XForm Element's
XML.

Then, I believe I found the solution, and it's correct. I'm pretty happy
and relax now.
Thanks, Yaw for your help.
I trust ODK very much because your support.

···

On Thursday, November 26, 2015 at 2:36:53 PM UTC+8, janusm...@gmail.com wrote: > > Greetings. > > Currently doing a project using OKDAggregate Server, to use "curl" to > submit form data to Aggregate. > Did read all wiki pages: > https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI > https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI > and related wiki pages but no solution. > After some web search and figured out how to use "curl -F" to submit xml > text file to Aggregate. > Just guess it's not so difficult to submit associated file such as jpg > image file to Aggregate. > However, still can't do that after so many tries. > > For example image file = picture.jpg > Did try: > curl -F "image=@picture.jpg" "[AggregateServerURL]/submission", not > working. > curl -F "media=@picture.jpg" "[AggregateServerURL]/submission", not > working. > curl -F "binary=@picture.jpg" "[AggregateServerURL]/submission", not > working. > Also tried: > curl -d --data-binary "file:///[location]/picture.jpg" > "[AggregateServerURL]/submission", not working. > > Any help will be appreciated. > Cheers, > Janus >

yanokwa · November 26, 2015, 7:10am

Hi Janus,

Please don't send multiple messages to the list.

Form submissions are sent to servers as a multipart MIME envelope and
you must always send the XML content.

I'm not sure how best to do that in curl, but you should look at at
the Collect source code at

github.com

getodk/collect/blob/master/collect_app/src/main/java/org/odk/collect/android/tasks/InstanceUploaderTask.java#L338


      
          /*
           * Copyright 2016 Nafundi
           *
           * Licensed under the Apache License, Version 2.0 (the "License");
           * you may not use this file except in compliance with the License.
           * You may obtain a copy of the License at
           *
           * http://www.apache.org/licenses/LICENSE-2.0
           *
           * Unless required by applicable law or agreed to in writing, software
           * distributed under the License is distributed on an "AS IS" BASIS,
           * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           * See the License for the specific language governing permissions and
           * limitations under the License.
           */
          
          package org.odk.collect.android.tasks;
          
          import android.net.Uri;
          import android.os.AsyncTask;
          
          import org.odk.collect.android.application.Collect;
          import org.odk.collect.android.listeners.InstanceUploaderListener;
          import org.odk.collect.android.utilities.InstanceAutoDeleteChecker;
          import org.odk.collect.android.utilities.InstancesRepositoryProvider;
          import org.odk.collect.forms.FormsRepository;
          import org.odk.collect.forms.instances.Instance;
          import org.odk.collect.forms.instances.InstancesRepository;
          import org.odk.collect.settings.SettingsProvider;
          import org.odk.collect.settings.keys.ProjectKeys;
          
          import java.util.HashMap;
          import java.util.Set;
          import java.util.stream.Stream;
          
          public abstract class InstanceUploaderTask extends AsyncTask<Long, Integer, InstanceUploaderTask.Outcome> {
          
              private InstancesRepository instancesRepository;
              private FormsRepository formsRepository;
              protected SettingsProvider settingsProvider;
              private InstanceUploaderListener stateListener;
              private Boolean deleteInstanceAfterSubmission;
          
              @Override
              protected void onPostExecute(Outcome outcome) {
                  synchronized (this) {
                      if (outcome != null && stateListener != null) {
                          if (outcome.authRequestingServer != null) {
                              stateListener.authRequest(outcome.authRequestingServer, outcome.messagesByInstanceId);
                          } else {
                              stateListener.uploadingComplete(outcome.messagesByInstanceId);
          
                              // Delete instances that were successfully sent and that need to be deleted
                              // either because app-level auto-delete is enabled or because the form
                              // specifies it.
                              Set<String> instanceIds = outcome.messagesByInstanceId.keySet();
          
                              boolean isFormAutoDeleteOptionEnabled;
          
                              // The custom configuration from the third party app overrides
                              // the app preferences set for delete after submission
                              if (deleteInstanceAfterSubmission != null) {
                                  isFormAutoDeleteOptionEnabled = deleteInstanceAfterSubmission;
                              } else {
                                  isFormAutoDeleteOptionEnabled = settingsProvider.getUnprotectedSettings().getBoolean(ProjectKeys.KEY_DELETE_AFTER_SEND);
                              }
          
                              Stream<Instance> instancesToDelete = instanceIds.stream()
                                      .map(id -> new InstancesRepositoryProvider(Collect.getInstance()).get().get(Long.parseLong(id)))
                                      .filter(instance -> instance.getStatus().equals(Instance.STATUS_SUBMITTED))
                                      .filter(instance -> InstanceAutoDeleteChecker.shouldInstanceBeDeleted(formsRepository, isFormAutoDeleteOptionEnabled, instance));
          
                              DeleteInstancesTask dit = new DeleteInstancesTask(instancesRepository, formsRepository);
                              dit.execute(instancesToDelete.map(Instance::getDbId).toArray(Long[]::new));
                          }
                      }
                  }
              }
          
              @Override
              protected void onProgressUpdate(Integer... values) {
                  synchronized (this) {
                      if (stateListener != null) {
                          stateListener.progressUpdate(values[0], values[1]);
                      }
                  }
              }
          
              public void setUploaderListener(InstanceUploaderListener sl) {
                  synchronized (this) {
                      stateListener = sl;
                  }
              }
          
              public void setDeleteInstanceAfterSubmission(Boolean deleteInstanceAfterSubmission) {
                  this.deleteInstanceAfterSubmission = deleteInstanceAfterSubmission;
              }
          
              public void setRepositories(InstancesRepository instancesRepository, FormsRepository formsRepository, SettingsProvider settingsProvider) {
                  this.instancesRepository = instancesRepository;
                  this.formsRepository = formsRepository;
                  this.settingsProvider = settingsProvider;
              }
          
              /**
               * Represents the results of a submission attempt triggered by explicit user action (as opposed
               * to auto-send). A submission attempt can include finalized forms going to several different
               * servers because the app-level server configuration can be overridden by the blank form.
               * <p>
               * The user-facing message that describes the result of a submission attempt for each specific
               * finalized form is written messages to {@link #messagesByInstanceId}. In the case of an
               * authentication request from the server, {@link #authRequestingServer} is set instead.
               */
              public static class Outcome {
                  /**
                   * The URI for the server that requested authentication when the latest finalized form was
                   * attempted to be sent. This URI may not match the server specified in the app settings or
                   * the blank form because there could have been a redirect. It is included in the Outcome so
                   * that it can be shown to the user so s/he will know where the auth request came from.
                   * <p>
                   * When this field is set, the overall submission attempt is halted so that the user can be
                   * asked for credentials. Once credentials are provided, the submission attempt resumes.
                   */
                  public Uri authRequestingServer;
          
          
                  /**
                   * Map of database IDs for finalized forms to the user-facing status message for the latest
                   * submission attempt. Currently this can be either a localized message in the case of a
                   * common status or an English message in the case of a rare status that is needed for
                   * developer troubleshooting.
                   * <p>
                   * The keys in the map are also used to identify filled forms that were part of the ongoing
                   * submission attempt and don't need to be retried in the case of an authentication request.
                   * See {@link #authRequestingServer}.
                   * <p>
                   * TODO: Consider mapping to something machine-readable like a message ID or status ID
                   * instead of a mix of localized and non-localized user-facing strings.
                   */
                  public HashMap<String, String> messagesByInstanceId = new HashMap<>();
              }
          }

This file has been truncated. show original

to see how it's done in Java and try to go from there. Might also be
easier to build a proof of concept with the lightweight language of
your choice (e.g, Python, Ruby).

If you have more developer questions, please send them to
opendatakit-developers instead of this list.

Yaw

···

-- Need ODK consultants? Nafundi provides form design, server setup, in-field training, and software development for ODK. Go to https://nafundi.com to get started.

On Thu, Nov 26, 2015 at 1:36 PM, janusma.caben@gmail.com wrote:

Greetings.

Currently doing a project using OKDAggregate Server, to use "curl" to submit
form data to Aggregate.
Did read all wiki pages:
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI
https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI
and related wiki pages but no solution.
After some web search and figured out how to use "curl -F" to submit xml
text file to Aggregate.
Just guess it's not so difficult to submit associated file such as jpg image
file to Aggregate.
However, still can't do that after so many tries.

For example image file = picture.jpg
Did try:
curl -F "image=@picture.jpg" "[AggregateServerURL]/submission", not working.
curl -F "media=@picture.jpg" "[AggregateServerURL]/submission", not working.
curl -F "binary=@picture.jpg" "[AggregateServerURL]/submission", not
working.
Also tried:
curl -d --data-binary "file:///[location]/picture.jpg"
"[AggregateServerURL]/submission", not working.

Any help will be appreciated.
Cheers,
Janus

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.