Hi ODK Team,
What are the minimum specifications for a server with ODK Aggregate?
Thanks,
Victor
It largely depends upon how large your forms are (how many columns) and how
many images/audio clips/videos you have per submission, and how large they
are.
The standard AppEngine deployment uses a "600MHz 128MB" virtual machine for
the frontend operations, and a 256MB virtual machine for the backend
instance.
See:
If you are deploying to Tomcat, the frontend and backend instance are the
same ( they aren't separate servers ).
Submissions to the server are chunked into 10MB or smaller blocks of
submission + media attachments. If you're recording very long videos (>
10MB in size), then each of these would be uploaded in its own HTTP request
and you will probably need to increase the JVM if these are very large (>
50MB). Operating with poorer quality data connections and low-bandwidth
situations, you should reconsider what you are doing, as uploading such
large media files is probably going to be very painful... .
The biggest limitation is during the generation of KML or CSV files for
download. The software needs to hold the entire file contents in memory
plus an 100-submission chunk of records that it works on during each pass
of its outer loop. So if you have very large data, generating a KML or CSV
will be your limitation.
I don't have a good data/sense of how you would need to increase your JVM
to accomodate larger forms. The default 256MB AppEngine backend seems to
work fine for most groups.
Mitch
···
On Wed, Jun 27, 2012 at 6:05 AM, Victor Lins wrote:Hi ODK Team,
What are the minimum specifications for a server with ODK Aggregate?
Thanks,
Victor--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Hi Mitch,
I'm able to export KML files, but unable to download it.
size of KML is around 3.1 MB, is there any way to download it?
-Res
···
On Wed, Jun 27, 2012 at 10:35 PM, Mitch S wrote:It largely depends upon how large your forms are (how many columns) and
how many images/audio clips/videos you have per submission, and how large
they are.The standard AppEngine deployment uses a "600MHz 128MB" virtual machine
for the frontend operations, and a 256MB virtual machine for the backend
instance.If you are deploying to Tomcat, the frontend and backend instance are the
same ( they aren't separate servers ).Submissions to the server are chunked into 10MB or smaller blocks of
submission + media attachments. If you're recording very long videos (>
10MB in size), then each of these would be uploaded in its own HTTP request
and you will probably need to increase the JVM if these are very large (>
50MB). Operating with poorer quality data connections and low-bandwidth
situations, you should reconsider what you are doing, as uploading such
large media files is probably going to be very painful... .The biggest limitation is during the generation of KML or CSV files for
download. The software needs to hold the entire file contents in memory
plus an 100-submission chunk of records that it works on during each pass
of its outer loop. So if you have very large data, generating a KML or CSV
will be your limitation.I don't have a good data/sense of how you would need to increase your JVM
to accomodate larger forms. The default 256MB AppEngine backend seems to
work fine for most groups.Mitch
On Wed, Jun 27, 2012 at 6:05 AM, Victor Lins vhlins@gmail.com wrote:
Hi ODK Team,
What are the minimum specifications for a server with ODK Aggregate?
Thanks,
Victor--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
What is the error in the Logs when you click download? The logs are
reachable from your application's Dashboard (via
https://appengine.google.com/ ) by clicking on "Logs" on the left sidebar.
The error would appear under a log entry for "/view/binaryData..."
If you see an out-of-memory error (I'd be surprised if this was happening
for that file size), you can increase the memory size of your server by
clicking on the "Application Settings" link on the left sidebar. On that
settings page, there is a setting under "Performance" for the Frontend
instance class. You can increase this (e.g., to 256MB) and see if the error
goes away (this will increase your charges or decrease the 'free usage'
time for your server).
Mitch
···
On Mon, Sep 10, 2012 at 9:09 PM, res tester wrote:Hi Mitch,
I'm able to export KML files, but unable to download it.
size of KML is around 3.1 MB, is there any way to download it?-Res
On Wed, Jun 27, 2012 at 10:35 PM, Mitch S mitchellsundt@gmail.com wrote:
It largely depends upon how large your forms are (how many columns) and
how many images/audio clips/videos you have per submission, and how large
they are.The standard AppEngine deployment uses a "600MHz 128MB" virtual machine
for the frontend operations, and a 256MB virtual machine for the backend
instance.If you are deploying to Tomcat, the frontend and backend instance are the
same ( they aren't separate servers ).Submissions to the server are chunked into 10MB or smaller blocks of
submission + media attachments. If you're recording very long videos (>
10MB in size), then each of these would be uploaded in its own HTTP request
and you will probably need to increase the JVM if these are very large (>
50MB). Operating with poorer quality data connections and low-bandwidth
situations, you should reconsider what you are doing, as uploading such
large media files is probably going to be very painful... .The biggest limitation is during the generation of KML or CSV files for
download. The software needs to hold the entire file contents in memory
plus an 100-submission chunk of records that it works on during each pass
of its outer loop. So if you have very large data, generating a KML or CSV
will be your limitation.I don't have a good data/sense of how you would need to increase your JVM
to accomodate larger forms. The default 256MB AppEngine backend seems to
work fine for most groups.Mitch
On Wed, Jun 27, 2012 at 6:05 AM, Victor Lins vhlins@gmail.com wrote:
Hi ODK Team,
What are the minimum specifications for a server with ODK Aggregate?
Thanks,
Victor--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Looking for recommended server specifications for a local installation of Aggregate that will handle data exports in CSV smoothly. Expecting the form to have ~20 columns, average of 3 repeats of the form and up to 5,000 entries(unique records), no media or GIS information. Will be setting up Aggregate on a local server using standard configuration- Tomcat + PostgreSQL.
Hey as per your requirement you need a system which has atleast:
- 4 GB RAM
- 500 GB Hard Drive/128 GB is enough for your work.
- Java 9 or above version
- Tomcat 8.0 and above
However, I will recommand you to visit https://docs.opendatakit.org/aggregate-deployment-planning/#computer-skills for more information.
Best,
@iamnarendrasingh
Thanks @iamnarendrasingh for the link- it doesn't specifically talk about hardware specifications which is what I am after. What are the optimum specs for processor and memory; how much memory should be allocated to Tomcat (to deal with memory leaks).