ODK Briefcase Failing to download - Fetch of Submission Chunk Failed. Error 400

Hi.

Having issues downloading a number of forms from an ODK instance.
First things:
Aggregate 1.4.7
Briefcase 1.4.4

Username and login work for Aggregate Login and I can connect and see the form list in Briefcase. Using an admin account with full rights.

The forms I am trying to download are slightly complex they include repeated groups which themselves contain images.

I can export via Aggregate the main CSVs but obviously intended to use Briefcase to pull the nested/repeated groups and their media files.

When I do this I get the following error message:

Fetching form definition
resolving against briefcase form definitions
preparing to retrieve instance data
retrieving next chunk of instances from server...
NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch of submission download chunk failed. Detailed error: Bad Request (400) while accessing: https://impevaluvira.odk.lshtm.ac.uk/view/submissionList?formId=visite_ctc&cursor=&numEntries=100
Please verify that the submission download chunk that is being uploaded is well-formed.
FAILED.

I have already tried pointing Briefcase at a different storage location without improvement.

Suggestions very gratefully received,

Michael

Did you do anything special w.r.t. configuring your Tomcat server?

i.e., did you enable chunked or gzip compression on the server container?

If so, disable that for everything under the /view/submissionList and
/view/downloadSubmission URLs.

And are you using Tomcat 6?

What do you get if you click on the link (and log in).
You should get a well-formed XML file, with links. What happens if you
click on one of those links?

ยทยทยท On Wed, Jan 6, 2016 at 9:30 AM, wrote:

Hi.

Having issues downloading a number of forms from an ODK instance.
First things:
Aggregate 1.4.7
Briefcase 1.4.4

Username and login work for Aggregate Login and I can connect and see the
form list in Briefcase. Using an admin account with full rights.

The forms I am trying to download are slightly complex they include
repeated groups which themselves contain images.

I can export via Aggregate the main CSVs but obviously intended to use
Briefcase to pull the nested/repeated groups and their media files.

When I do this I get the following error message:

Fetching form definition
resolving against briefcase form definitions
preparing to retrieve instance data
retrieving next chunk of instances from server...
NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch
of submission download chunk failed. Detailed error: Bad Request (400)
while accessing:
https://impevaluvira.odk.lshtm.ac.uk/view/submissionList?formId=visite_ctc&cursor=&numEntries=100
Please verify that the submission download chunk that is being uploaded is
well-formed.
FAILED.

I have already tried pointing Briefcase at a different storage location
without improvement.

Suggestions very gratefully received,

Michael

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Hi Mitch,

Very helpful.
I changed the Briefcase URL as you suggested and downloading now works perfectly- thank you. We have lots of projects running worldwide using ODK - most people have been downloading for their projects directly from aggregate and this is our first user trying briefcase so this information is incredibly helpful.

I'll leave Matthew to follow up on the server/load-balancer setup type things.

I looked at https://impevaluvira.odk.lshtm.ac.uk/impevaluvira/xformsList and this does indeed produce a list of forms:

HH_quest_v1 HH_quest_v1 md5:4b679844606f5ce68b16722ce3e1184b https://impevaluvira.odk.lshtm.ac.uk/impevaluvira/formXml?formId=HH_quest_v1 Etc etc

And the download URLs look correct I think based on your post.

Thank you again - very very helpful and much appreciated.

Hi Mitch,

I didn't setup the Tomcat server (someone else at the University did) but I have enquired.

What do you mean by:
What do you get if you click on the link (and log in).
You should get a well-formed XML file, with links. What happens if you click on one of those links?

Is this something I should be able to do on the aggregate server?

ยทยทยท On Wednesday, January 6, 2016 at 9:36:36 PM UTC, Mitch Sundt wrote: > Did you do anything special w.r.t. configuring your Tomcat server? > > > i.e., did you enable chunked or gzip compression on the server container? > > > If so, disable that for everything under the /view/submissionList and /view/downloadSubmission URLs. > > > And are you using Tomcat 6? > > > What do you get if you click on the link (and log in). > You should get a well-formed XML file, with links. What happens if you click on one of those links? > > > > > On Wed, Jan 6, 2016 at 9:30 AM, wrote: > Hi. > > > > Having issues downloading a number of forms from an ODK instance. > > First things: > > Aggregate 1.4.7 > > Briefcase 1.4.4 > > > > Username and login work for Aggregate Login and I can connect and see the form list in Briefcase. Using an admin account with full rights. > > > > The forms I am trying to download are slightly complex they include repeated groups which themselves contain images. > > > > I can export via Aggregate the main CSVs but obviously intended to use Briefcase to pull the nested/repeated groups and their media files. > > > > When I do this I get the following error message: > > > > Fetching form definition > > resolving against briefcase form definitions > > preparing to retrieve instance data > > retrieving next chunk of instances from server... > > NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch of submission download chunk failed. Detailed error: Bad Request (400) while accessing: https://impevaluvira.odk.lshtm.ac.uk/view/submissionList?formId=visite_ctc&cursor=&numEntries=100 > > Please verify that the submission download chunk that is being uploaded is well-formed. > > FAILED. > > > > > > I have already tried pointing Briefcase at a different storage location without improvement. > > > > Suggestions very gratefully received, > > > > Michael > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google Groups "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com

Hi Mitch

Further to Michael's query (I work with Michael and am the one who set up the server)

  • our Tomcat server setup is pretty much standard
  • there's no compression enabled
  • yes, we're running Tomcat 6

I can confirm the problem with Briefcase 1.4.6 on another instance too.

Perhaps the only unusual part of our config is that we present via a hardware load balancer.

Matthew

ยทยทยท On Wednesday, January 6, 2016 at 9:36:36 PM UTC, Mitch Sundt wrote: > Did you do anything special w.r.t. configuring your Tomcat server? > > > i.e., did you enable chunked or gzip compression on the server container? > > > If so, disable that for everything under the /view/submissionList and /view/downloadSubmission URLs. > > > And are you using Tomcat 6? > > > What do you get if you click on the link (and log in). > You should get a well-formed XML file, with links. What happens if you click on one of those links? > > > > > On Wed, Jan 6, 2016 at 9:30 AM, wrote: > Hi. > > > > Having issues downloading a number of forms from an ODK instance. > > First things: > > Aggregate 1.4.7 > > Briefcase 1.4.4 > > > > Username and login work for Aggregate Login and I can connect and see the form list in Briefcase. Using an admin account with full rights. > > > > The forms I am trying to download are slightly complex they include repeated groups which themselves contain images. > > > > I can export via Aggregate the main CSVs but obviously intended to use Briefcase to pull the nested/repeated groups and their media files. > > > > When I do this I get the following error message: > > > > Fetching form definition > > resolving against briefcase form definitions > > preparing to retrieve instance data > > retrieving next chunk of instances from server... > > NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch of submission download chunk failed. Detailed error: Bad Request (400) while accessing: https://impevaluvira.odk.lshtm.ac.uk/view/submissionList?formId=visite_ctc&cursor=&numEntries=100 > > Please verify that the submission download chunk that is being uploaded is well-formed. > > FAILED. > > > > > > I have already tried pointing Briefcase at a different storage location without improvement. > > > > Suggestions very gratefully received, > > > > Michael > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google Groups "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com

First, upon looking at the links, it appears you have not configured the
correct URL into Briefcase.

The server URL should be

https://impevaluvira.odk.lshtm.ac.uk/*impevaluvira/*

you have specified

https://impevaluvira.odk.lshtm.ac.uk/

This will cause problems in ODK Briefcase, and can cause problems in ODK
Collect.

Next, during the ODK Aggregate installer, you should have specified the IP
address or DNS name of the load balancer. NOT of the server. Similarly, it
should be the port on the load balancer, NOT the port on the server, if
those are not the same.

One way to confirm this is to look at the results of
https://impevaluvira.odk.lshtm.ac.uk/*impevaluvira/xformsList*

This will display a list of forms on the server and their links. E.g.,
https://opendatakit.appspot.com/xformsList produces:

CascadingSelect Cascading Select Form 2012072302 2012072302 md5:d4750440f463f31ba1de0b918d3a5582 https://opendatakit.appspot.com/formXml?formId=CascadingSelect ...

Look at the link. does its URL match (for your server):

https://impevaluvira.odk.lshtm.ac.uk/*impevaluvira/formXml?...*

If not, then you need to re-run the installer and specify the details of
the load balancer.

If your server has a firewall that masks or alters the application path in
any way, you may need to fix that, too (though it sounds like you didn't
try to be clever with configuration).

If none of this resolves the issue, I would first verify that talking
directly to your Tomcat server works (without any load balancer). If you
have Tomcat working through an Apache proxy, a different group has had
difficulty with an Apache proxy stripping out X- headers from the requests
and responses.

Once you get a direct Briefcase v1.4.6 interaction with your server
working, at that point, I think the only way to debug this is going to be
using a network sniffer (e.g., wireshark).

Collect a trace of the Briefcase interaction directly to your server.

Then change the configuration to add in the loadbalancer. And collect a
trace with that.

And then wade through them, looking for significant differences.

ยทยทยท On Thu, Jan 7, 2016 at 9:09 AM, wrote:

Hi Mitch

Further to Michael's query (I work with Michael and am the one who set up
the server)

  • our Tomcat server setup is pretty much standard
  • there's no compression enabled
  • yes, we're running Tomcat 6

I can confirm the problem with Briefcase 1.4.6 on another instance too.

Perhaps the only unusual part of our config is that we present via a
hardware load balancer.

Matthew

On Wednesday, January 6, 2016 at 9:36:36 PM UTC, Mitch Sundt wrote:

Did you do anything special w.r.t. configuring your Tomcat server?

i.e., did you enable chunked or gzip compression on the server container?

If so, disable that for everything under the /view/submissionList and
/view/downloadSubmission URLs.

And are you using Tomcat 6?

What do you get if you click on the link (and log in).
You should get a well-formed XML file, with links. What happens if you
click on one of those links?

On Wed, Jan 6, 2016 at 9:30 AM, michaeled...@gmail.com wrote:
Hi.

Having issues downloading a number of forms from an ODK instance.

First things:

Aggregate 1.4.7

Briefcase 1.4.4

Username and login work for Aggregate Login and I can connect and see
the form list in Briefcase. Using an admin account with full rights.

The forms I am trying to download are slightly complex they include
repeated groups which themselves contain images.

I can export via Aggregate the main CSVs but obviously intended to use
Briefcase to pull the nested/repeated groups and their media files.

When I do this I get the following error message:

Fetching form definition

resolving against briefcase form definitions

preparing to retrieve instance data

retrieving next chunk of instances from server...

NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch
of submission download chunk failed. Detailed error: Bad Request (400)
while accessing:
https://impevaluvira.odk.lshtm.ac.uk/view/submissionList?formId=visite_ctc&cursor=&numEntries=100

Please verify that the submission download chunk that is being uploaded
is well-formed.

FAILED.

I have already tried pointing Briefcase at a different storage location
without improvement.

Suggestions very gratefully received,

Michael

--

--

Post: opend...@googlegroups.com

Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.

To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

1 Like