Hello. I have been having some issues when uploading a large number of instances to a local instances of aggregate (both the beta .93 and the current 0.92 release). I have been running a script to post the instances and it runs smoothly for for first few hundred instances until it starts slowing down and gives me the following error.
Any idea how to get around this memory issue?
Thanks,
a
I haven't looked at the memory allocation in Aggregate. If you have a
system where you can submit requests over time, using a tool like
https://visualvm.dev.java.net/features.html to monitor it over time can help
identify where the runaway memory allocation is occurring.
If you can identify what class has continually increasing numbers of
instances over time, I can help sleuth out where the problem is.
Mitch
Hello. I have been having some issues when uploading a large number of
instances to a local instances of aggregate (both the beta .93 and the
current 0.92 release). I have been running a script to post the instances
and it runs smoothly for for first few hundred instances until it starts
slowing down and gives me the following error http://dpaste.com/232425/.Any idea how to get around this memory issue?
Thanks,
a
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
I think it would be worth using the tool that Mitch provided to see
where you issue is as it could be a bug in our code.
However, as long as the Request is allowed to complete before the next
request to Aggregrate, the part of the code that ODK team is
responsible for is not running after a request is complete. Therefore
I have doubts it is a memory issue in Aggregate code (could be) but
it's more likely a Google App Engine Java Development Server issue.
When running on your local machine Aggregate utilizes the GAE Java
Development Server which stores things in a flat file and does not
scale as well as the amount of data grows. It may be that if you
slowed the request rate it could handle the flat file storage better.
Google keeps talking about releasing a real version of GAE for local
machines however I don't know how large of incentive they have since
it appears they would much rather have you using their cloud services.
Aggregate 1.0 will fix this issue by releasing a version of Aggregate
that we have designed to work on a local system using a local database
and tomcat.