Occasional garbled image upload

Support
1

As you can see in the attached image, sometimes (very occasionally) we get the image uploaded but it appears to have been cut up and stitched back together incorrectly. Understanding how the upload process works when writing to the database might help understand why. Our assumption is a poor internet connectivity during the upload, but i would prefer to hear other opinions as to why this is happening.

we are using our own tomcat/postgres server

image
image480×640 116 KB

Regards
Andrew

2

Hi @Andrew.deklerk!

This corruption could happen anywhere in the pipeline. It could be a bad lens, a bad SD card, a bad device, a bad connection, a bad database, a bug, etc, etc. You'll need to start from the beginning of the pipeline and see where the problem starts.

What will also help you track it down is to explore if this happens on the same device or the same network or the same version of Aggregate, etc. If you can consistently reproduce it, please share the steps on how you repro it.

P.S. For future ODK Aggregate questions, please add that tag so we can keep the forum more organized! :pray:

3

Hi @yanokwa

We have noticed this on two separate projects. In both cases separate ODK instances and separate devices. In the one instance the images were not being saved to an SD card, in the other case we are not sure. We have no knowledge if the finalised forms were sent over mobile network or wifi, and what the connectivity speed was when the data was synchronised.

Is there documentation of how the images are sent to the aggregate server? i.e. are they sent in chunks or as one whole chunk of information?

Andrew

4

The form submission API is described at https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI. I'd encourage you to read the whole thing.

From the content section

OpenRosa submissions are POSTed to servers as a multipart MIME Envelope with at least 1 part (the xml content of the form itself). Each of these parts should adhere to the following requirements

  • Mime Envelope:
  • Content Type: multipart/form-data
  • Contains Exactly 1 XForm Part
  • Contains 0 or More Additional Parts
  • XForm Part:
    • Content Type: text/xml
    • Name: xml_submission_file
  • Additional Parts
  • Content Type: arbitrary
  • Name: matches an appropriate element inside of the XForm Element's XML.

From the Extended Transmission Considerations section

If a full POST of the form's xml submission and its additional parts (e.g., captured image, audio or video clips) would exceed the size specified in the X-OpenRosa-Accept-Content-Length header (the "maxSize"), it is recommended that the client split the POST into multiple individual POST requests, each containing the form's xml submission and one or more additional parts such that each partial POST request is no greater than maxSize; if a single additional part is greater than maxSize, the POST should contain the form's xml submission and that single additional part. Regardless of whether the client observes and honors the X-OpenRosa-Accept-Content-Length header, a compliant server with give its best effort to accept submissions of any length.

The implementation of that API in Collect is at https://github.com/opendatakit/collect/blob/master/collect_app/src/main/java/org/odk/collect/android/tasks/InstanceUploaderTask.java