I'm technical support for a research project that has decided on using ODK Aggregate on Google App Engine as a backend (with ODK Collect to collect the data). The project has collected data for almost a year, and now it is time to start analyzing it. Annoyingly, about the same time some limit was hit in Aggregate preventing access to the data.
What is the problem? Please be detailed.
Most of the time when trying to list submissions or publish/export data from Aggregate, nothing happens for about a minute, followed by the page reloading, and sometimes followed by an error page questioning if the bill to Google has been paid. That is; the app crashes and reloads.
Looking at the application error logs in App Engine, I see "OutOfMemoryError: Java heap space" errors at the exact same time the problem occurs. The class responsible for the error differs, but the three main contributors are BinaryContentManipulator.java, ImageUtilImpl.java and ProtocolSink.java.
Worth mentioning is that almost every entry collected has 1-3 pictures attached, usually full size pictures taken with the smartphone that is doing the collection.
When the instance class is F2 (max ram 256MB), the error is not much of a surprise when image manipulation is involved for hundreds of pictures. I would like to have the instance class changed to F4, but can't figure out how to do that.
What ODK tool and version are you using? And on what device and operating system version?
ODK Aggregate is version v1.4.15 (according to the log), running on Google App Engine.
What steps can we take to reproduce the problem?
Probably requires hundreds of entries with several full size pictures on each entry to reproduce, since the test system with fewer entries did not catch the issue.
What you have you tried to fix the problem?
Not much - the collection is still ongoing, so any major changes or upgrades are out of the question at the moment, and only the live Aggregate instance has data that triggers the problem - making testing difficult.
What I would like is to find a simple way of changing the instance class to F4, since that would probably "hide" the problem. Unfortunately, that appears to require a modified install of Aggregate since it is no longer possible to change instance type directly in the App Engine.