1. What is the problem? Be very detailed.
When I try to pull data from one of my forms using ODK Briefcase v1.15.0 I get an error saying "failed". The details shows that this occurs after "fetched instance 3143". I also get a failure error when I try to export data to CSV from this form in Aggregate. The data is encrypted and being uploaded by partners in another country. Previous data pulls were successful until yesterday so it may be due to new submissions. I'm able to pull data from my other forms on Aggregate, just not one of them.
2. What app or server are you using and on what device and operating system? Include version numbers.
I'm using AppEngine with Briefcase 1.15.0. I can't tell which version of Aggregate I'm using but it was set up in 2015.
3. What you have you tried to fix the problem?
I've tried changing the storage location in Briefcase but it still fails. I think it might be due to a corrupted submission but the type of error I am getting (see below) is different than the 500 error I have seen mentioned in other posts.
4. What steps can we take to reproduce the problem?
Not sure.
5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.
thanks for the detailed error report, and welcome to the forum community. (if you have a moment, please consider posting a hello on the introduce yourself thread!)
it looks like the error code here is 503, and you are using app engine, and the process used to work. usually, this combination of clues indicates that you have run into your usage quota on app engine. i'm guessing it works again now, or if it doesn't you just have too much data in and out and the free app engine tier may not be enough for you anymore.
you can find more information about the quota limits here, and in general if you search this forum for "quota" you will likely find other people who have run into this problem.
Thanks very much for your message and suggestion. I actually had a similar problem when we ran our first survey in 2016 and it turned out this was due to exceeding the quota, so I enabled billing at that time. I don't think it's a quota issue this time. I tried exporting the data again a few minutes ago and it still fails...
Do you have any other ideas of what the issue might be?
briefcase.zip (278.2 KB) Hi @yanokwa, thanks for your reply. We need to get data out of the server. I've attached the briefcase.log here - hope you can help!
Thanks for attaching the log file! Unfortunately it doesn't shed much light into the situation, as 500 errors in Aggregate/Briefcase have been known for masking real causes of problems.
Before continuing, the logs are suggesting that you're successfully pulling all submissions but one. Can you confirm this? Briefcase should try to download each individual submissions and not stop the whole operation when one of them fails. If this happens, it would be normal to see a final "Failed" status, which doesn't mean that starting from the one that's failed, the rest haven't been pulled.
In any case, this problem is something that we will probably have to solve server-side. If you suspect that the failing submission might be corrupt, a straightforward solution would be to remove that submission using the Google Cloud DataStore. The instance ID to remove is detailed in the error message in the log.
Hi @ggalmazor - thanks! Yes, I'm successfully pulling all submissions up to a certain point, and then it fails. I assume this means that it can't pull the corrupted submission or any of the forms after that. Could you direct me to some step by step instructions on how to remove the submission from the AppEngine server?
When you get to a good stopping point in your campaign, I'd encourage you make a backup of your data and upgrade to Aggregate v1.7.3 (the latest version that supports App Engine). It has improvements for these kinds of problems.