1. What is the problem? Be very detailed.
When I try to pull data from one of my forms using ODK Briefcase v1.15.0 I get an error saying "failed". The details shows that this occurs after "fetched instance 3143". I also get a failure error when I try to export data to CSV from this form in Aggregate. The data is encrypted and being uploaded by partners in another country. Previous data pulls were successful until yesterday so it may be due to new submissions. I'm able to pull data from my other forms on Aggregate, just not one of them.
2. What app or server are you using and on what device and operating system? Include version numbers.
I'm using AppEngine with Briefcase 1.15.0. I can't tell which version of Aggregate I'm using but it was set up in 2015.
3. What you have you tried to fix the problem?
I've tried changing the storage location in Briefcase but it still fails. I think it might be due to a corrupted submission but the type of error I am getting (see below) is different than the 500 error I have seen mentioned in other posts.
4. What steps can we take to reproduce the problem?
5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.
I'm using Google AppEngine -- in the error log it shows:
0.1.0.2 - - [27/May/2019:10:22:35 -0400] "GET /gae/watchdog HTTP/1.1" 503 - http://background.jimma-survey.appspot.com/gae/csvGeneratorTask?formId=index_woman_01apr2019enc&persistentResult=aggregate.opendatakit.org%3APersistentResults[%40version%3Dnull+and+%40uiVersion%3Dnull]%2F_persistent_results[%40key%3Duuid%3A48621b9d-e966-46e9-aeeb-b00cecac9a91]&attemptCount=2 "AppEngine-Google; (+http://code.google.com/appengine)" "background.jimma-survey.appspot.com" ms=20912 cpu_ms=0 cpm_usd=0 loading_request=0 instance=- app_engine_release=1.9.71 trace_id=a82fe1d38ef18301d661df42210e9d9e
Any advice on what might be causing this issue and how to fix it would be very much appreciated.
thanks for the detailed error report, and welcome to the forum community. (if you have a moment, please consider posting a hello on the introduce yourself thread!)
it looks like the error code here is
503, and you are using app engine, and the process used to work. usually, this combination of clues indicates that you have run into your usage quota on app engine. i'm guessing it works again now, or if it doesn't you just have too much data in and out and the free app engine tier may not be enough for you anymore.
you can find more information about the quota limits here, and in general if you search this forum for "quota" you will likely find other people who have run into this problem.
Thanks very much for your message and suggestion. I actually had a similar problem when we ran our first survey in 2016 and it turned out this was due to exceeding the quota, so I enabled billing at that time. I don't think it's a quota issue this time. I tried exporting the data again a few minutes ago and it still fails...
Do you have any other ideas of what the issue might be?
Hi, I'm still having the same issue and can't pull my data using Briefcase or export data using Aggregate. Both of these fail.
If it is an issue with a corrupted submission how can I find this out?
I'm using Aggregate v1.4.7.
Hi @mkulkarni, sorry you are still having problems. Aggregate v1.4.7 is very old and that makes it hard to troubleshoot.
- Do you need to get data out of this server or are you looking to start a new campaign?
- Briefcase should include a briefcase.log in the folder where you ran it. Can you attach it?
briefcase.zip (278.2 KB) Hi @yanokwa, thanks for your reply. We need to get data out of the server. I've attached the briefcase.log here - hope you can help!
Thanks for attaching the log file! Unfortunately it doesn't shed much light into the situation, as 500 errors in Aggregate/Briefcase have been known for masking real causes of problems.
Before continuing, the logs are suggesting that you're successfully pulling all submissions but one. Can you confirm this? Briefcase should try to download each individual submissions and not stop the whole operation when one of them fails. If this happens, it would be normal to see a final "Failed" status, which doesn't mean that starting from the one that's failed, the rest haven't been pulled.
In any case, this problem is something that we will probably have to solve server-side. If you suspect that the failing submission might be corrupt, a straightforward solution would be to remove that submission using the Google Cloud DataStore. The instance ID to remove is detailed in the error message in the log.
Hi @ggalmazor - thanks! Yes, I'm successfully pulling all submissions up to a certain point, and then it fails. I assume this means that it can't pull the corrupted submission or any of the forms after that. Could you direct me to some step by step instructions on how to remove the submission from the AppEngine server?
Sure! I'll try to be as precise as possible, but my instructions could require some tweaking:
First go to your Google Cloud project's DataStore dashboard at https://console.cloud.google.com/datastore/entities
Kind field, search for
opendatakit.index_woman_01apr2019enc_CORE or similar
Click on the
FILTER ENTITIES button.
(New fields show up)
- On the
Key field, select
- On the
Value (at the right of the field saying
equal to) field, copy & paste this
- Click on
You should see only one submission on the table below. You can click on it and check on the side panel that opens all the related information.
When you're sure, click on the
Delete button on the top side of the page.
After doing this, try to pull the form again.
Hi @ggalmazor - this worked and I've managed to successfully pull the data, thanks so much!
So glad the issue is fixed, @mkulkarni!
When you get to a good stopping point in your campaign, I'd encourage you make a backup of your data and upgrade to Aggregate v1.7.3 (the latest version that supports App Engine). It has improvements for these kinds of problems.