JVM size for Tomcat. Can't upload forms with images

Hi all,

I'm running ODK Aggregate 1.4.12 with MariaDB (MySQL) on a Synology NAS
(DS219+). Form submission is working fine unless I have images in the
answers, then I get an error 500 in ODK Collect when sending the form. The
error message is: "Error: Internal Server Error (500) at http://....."

I have tweaked the JVM size in Tomcat by creating a setenv.sh file with the
following content:

export CATALINA_OPTS="$CATALINA_OPTS -Xms512m"

export CATALINA_OPTS="$CATALINA_OPTS -Xmx8192m"

export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m"

I've checked the settings in Tomcat and the available memory has increased
after tweaking.

Despite the change in JVM size it's not possible to upload forms with
images to ODK Aggregate. Should I increase memory allocation further?
Anyone had this problem? And how did you resolve it?

Regards,

Sid

Is there anything in your Tomcat logs to indicate what is failing?

Download the VM image, confirm that you can upload your form definition to
it and submit filled-in form submissions to it having multiple images.

This confirms that it is not a problem with ODK Collect or your form.

Then compare the configuration of the VM with your configuration.

··· On Thu, Oct 6, 2016 at 11:45 PM, Sid Patel <4sidpatel@gmail.com> wrote:

Hi all,

I'm running ODK Aggregate 1.4.12 with MariaDB (MySQL) on a Synology NAS
(DS219+). Form submission is working fine unless I have images in the
answers, then I get an error 500 in ODK Collect when sending the form. The
error message is: "Error: Internal Server Error (500) at http://....."

I have tweaked the JVM size in Tomcat by creating a setenv.sh file with
the following content:

export CATALINA_OPTS="$CATALINA_OPTS -Xms512m"

export CATALINA_OPTS="$CATALINA_OPTS -Xmx8192m"

export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m"

I've checked the settings in Tomcat and the available memory has increased
after tweaking.

Despite the change in JVM size it's not possible to upload forms with
images to ODK Aggregate. Should I increase memory allocation further?
Anyone had this problem? And how did you resolve it?

Regards,

Sid

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

I've tried my form on VM and that's working. But checking my log
catalina.out on Synology NAS I see the following lines:

Oct 08, 2016 9:39:49 AM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Persist failure -
org.springframework.dao.TransientDataAccessResourceException:
PreparedStatementCallback; SQL [INSERT INTO odk.GEOTEST161006_IMG_BLB (
_URI, _CREATOR_URI_USER, _CREATION_DATE, _LAST_UPDATE_URI_USER,
_LAST_UPDATE_DATE, _TOP_LEVEL_AURI, VALUE ) VALUES ( ?, ?, ?, ?, ?,
?, ? ) ]; Packet for query is too large (4726657 > 1048576). You can change
this value on the server by setting the max_allowed_packet' variable.;
nested exception is com.mysql.jdbc.PacketTooBigException: Packet for query
is too large (4726657 > 1048576). You can change this value on the server
by setting the max_allowed_packet' variable.
org.opendatakit.common.persistence.exception.ODKEntityPersistException:
org.springframework.dao.TransientDataAccessResourceException:
PreparedStatementCallback; SQL [INSERT INTO odk.GEOTEST161006_IMG_BLB (
_URI, _CREATOR_URI_USER, _CREATION_DATE, _LAST_UPDATE_URI_USER,
_LAST_UPDATE_DATE, _TOP_LEVEL_AURI, VALUE ) VALUES ( ?, ?, ?, ?, ?,
?, ? ) ]; Packet for query is too large (4726657 > 1048576). You can change
this value on the server by setting the max_allowed_packet' variable.;
nested exception is com.mysql.jdbc.PacketTooBigException: Packet for query
is too large (4726657 > 1048576). You can change this value on the server
by setting the max_allowed_packet' variable.

I suppose this has to do with my database (MariaDB). I'll try and figure
out how to execute the recommended solution from the log and report back.

Greatful for advise if possible.

Regards,
Sid

Den fredag 7 oktober 2016 kl. 08:45:25 UTC+2 skrev Sid Patel:

··· > > Hi all, > > I'm running ODK Aggregate 1.4.12 with MariaDB (MySQL) on a Synology NAS > (DS219+). Form submission is working fine unless I have images in the > answers, then I get an error 500 in ODK Collect when sending the form. The > error message is: "Error: Internal Server Error (500) at http://....." > > I have tweaked the JVM size in Tomcat by creating a setenv.sh file with > the following content: > > export CATALINA_OPTS="$CATALINA_OPTS -Xms512m" > > export CATALINA_OPTS="$CATALINA_OPTS -Xmx8192m" > > export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m" > > > I've checked the settings in Tomcat and the available memory has increased > after tweaking. > > > Despite the change in JVM size it's not possible to upload forms with > images to ODK Aggregate. Should I increase memory allocation further? > Anyone had this problem? And how did you resolve it? > > > Regards, > > Sid > >

Mitch is right! Check the logs first... After applying solution below to my
MariaDB on Synology NAS uploading images is now working fine.

Solution from
https://www.synology.com/en-uk/knowledgebase/DSM/tutorial/Application/Can_MariaDB_settings_be_customized_on_Synology_NAS:

When managing a database in DSM 5.0 or above, the default settings of the
MariaDB package should satisfy the requirements of most users. However, if
necessary, you can manually change the settings of MariaDB in order to suit
your specific needs.
To do so, follow the instructions below:
If it does not exist, create a configuration file named my.cnf. This file
should include the settings you wish to change in MariaDB (see example
below).
Place the my.cnf file at the following path on your Synology NAS:
/var/packages/MariaDB/etc
Reboot the Synology NAS or execute /usr/syno/bin/synopkg restart MariaDB to
apply the new settings. The new settings shall be preserved when updating
the MariaDB package or rebooting DSM.
For example, to increase the maximum size of incoming files in MariaDB, you
could add the following lines in the my.cnf configuration file:
[mysqld]
max_allowed_packet = 1073741824

//Sid

Den fredag 7 oktober 2016 kl. 08:45:25 UTC+2 skrev Sid Patel:

··· > > Hi all, > > I'm running ODK Aggregate 1.4.12 with MariaDB (MySQL) on a Synology NAS > (DS219+). Form submission is working fine unless I have images in the > answers, then I get an error 500 in ODK Collect when sending the form. The > error message is: "Error: Internal Server Error (500) at http://....." > > I have tweaked the JVM size in Tomcat by creating a setenv.sh file with > the following content: > > export CATALINA_OPTS="$CATALINA_OPTS -Xms512m" > > export CATALINA_OPTS="$CATALINA_OPTS -Xmx8192m" > > export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m" > > > I've checked the settings in Tomcat and the available memory has increased > after tweaking. > > > Despite the change in JVM size it's not possible to upload forms with > images to ODK Aggregate. Should I increase memory allocation further? > Anyone had this problem? And how did you resolve it? > > > Regards, > > Sid > >

Glad you finally got this working.

You should also check the character set and collation order. For MySQL, we
expect these to be utf-8. See our documentation
https://opendatakit.org/use/aggregate/tomcat-install/

If you didn't have the packet setting in the right place before, then you
will also not have had these settings correctly applied.

··· On Sat, Oct 8, 2016 at 1:33 AM, Sid Patel <4sidpatel@gmail.com> wrote:

Mitch is right! Check the logs first... After applying solution below to
my MariaDB on Synology NAS uploading images is now working fine.

Solution from https://www.synology.com/en-uk/knowledgebase/DSM/tutorial/
Application/Can_MariaDB_settings_be_customized_on_Synology_NAS:

When managing a database in DSM 5.0 or above, the default settings of the
MariaDB package should satisfy the requirements of most users. However, if
necessary, you can manually change the settings of MariaDB in order to suit
your specific needs.
To do so, follow the instructions below:
If it does not exist, create a configuration file named my.cnf. This file
should include the settings you wish to change in MariaDB (see example
below).
Place the my.cnf file at the following path on your Synology NAS:
/var/packages/MariaDB/etc
Reboot the Synology NAS or execute /usr/syno/bin/synopkg restart MariaDB
to apply the new settings. The new settings shall be preserved when
updating the MariaDB package or rebooting DSM.
For example, to increase the maximum size of incoming files in MariaDB,
you could add the following lines in the my.cnf configuration file:
[mysqld]
max_allowed_packet = 1073741824

//Sid

Den fredag 7 oktober 2016 kl. 08:45:25 UTC+2 skrev Sid Patel:

Hi all,

I'm running ODK Aggregate 1.4.12 with MariaDB (MySQL) on a Synology NAS
(DS219+). Form submission is working fine unless I have images in the
answers, then I get an error 500 in ODK Collect when sending the form. The
error message is: "Error: Internal Server Error (500) at http://....."

I have tweaked the JVM size in Tomcat by creating a setenv.sh file with
the following content:

export CATALINA_OPTS="$CATALINA_OPTS -Xms512m"

export CATALINA_OPTS="$CATALINA_OPTS -Xmx8192m"

export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m"

I've checked the settings in Tomcat and the available memory has
increased after tweaking.

Despite the change in JVM size it's not possible to upload forms with
images to ODK Aggregate. Should I increase memory allocation further?
Anyone had this problem? And how did you resolve it?

Regards,

Sid

--
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com