Failure in submission of a form if image file is present

Hello team,

We are in a middle of data collection process and we came across a problem
that when we upload the form without images, it is getting successfully
uploaded into the aggregate server but when we are trying to send the form
with images however it fails.

We already referred the previous threads in the ODK community and tried out
almost every possible solution provided there. We tried to resolve by
modifying at web server level like maxPostSize in tomcat,
connectionTimeout, maxHttpHeaderSize, Java heap memory size, Compression
algorithm. At database level max_allowed_packet, key_bufer_size,
buffer_pool_size.

We are using the below configurations of ODKAggregate, tomcat and mySql.

ODK version: 1.4.7
MySql version 5.5.45 community for linux
tomcat webserver: version 6

Please find attached log file for your reference.

Many thanks
Rohit Patnaik
SDRC (P) Ltd.

tomcat1.txt (5.07 KB)

tomcat2.txt (4.18 KB)

Looks like the connection breaks before the upload is complete. I sometimes
experience this on slow connections. Have you tried a faster connection?

Regards,
Ronald

··· On Thu, Sep 29, 2016 at 2:29 PM, Rohit Patnaik wrote:

Hello team,

We are in a middle of data collection process and we came across a problem
that when we upload the form without images, it is getting successfully
uploaded into the aggregate server but when we are trying to send the form
with images however it fails.

We already referred the previous threads in the ODK community and tried
out almost every possible solution provided there. We tried to resolve by
modifying at web server level like maxPostSize in tomcat,
connectionTimeout, maxHttpHeaderSize, Java heap memory size, Compression
algorithm. At database level max_allowed_packet, key_bufer_size,
buffer_pool_size.

We are using the below configurations of ODKAggregate, tomcat and mySql.

ODK version: 1.4.7
MySql version 5.5.45 community for linux
tomcat webserver: version 6

Please find attached log file for your reference.

Many thanks
Rohit Patnaik
SDRC (P) Ltd.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Ronald. K

Yes we have already tested for faster connection but same issue is arising when the form is uploaded with multiple images.

Both of those errors indicate problems receiving the submission in a timely
manner.

I.e., slow transmission over the network to the server.

Can you open a browser on your device and access your ODK Aggregate
server? Can you view the Submissions screen?

What happens if you disable cellular data transmission and use only WiFi
for submitting data? Some cellular networks may restrict the quantity of
data you upload (favoring download speeds).

··· On Thu, Sep 29, 2016 at 8:07 AM, wrote:

Yes we have already tested for faster connection but same issue is arising
when the form is uploaded with multiple images.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Thank you Mitch.

We have tried with better internet connectivity(4G network) and submission
through WiFi but same issue is arising when the form is uploaded with
multiple(preferably 2 images).

In the device we can access our ODK Aggregate server also the submission
screen. But still we are not able to upload our data. Even the image
quality has been reduced below 400KB/image.

Hope your support will help us to resolve the issue soon.

regards,
Rohit Patnaik

··· On Fri, Sep 30, 2016 at 10:22 PM, Mitch Sundt wrote:

Both of those errors indicate problems receiving the submission in a
timely manner.

I.e., slow transmission over the network to the server.

Can you open a browser on your device and access your ODK Aggregate
server? Can you view the Submissions screen?

What happens if you disable cellular data transmission and use only WiFi
for submitting data? Some cellular networks may restrict the quantity of
data you upload (favoring download speeds).

On Thu, Sep 29, 2016 at 8:07 AM, anirudha@sdrc.co.in wrote:

Yes we have already tested for faster connection but same issue is
arising when the form is uploaded with multiple images.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/opendatakit-developers/MN79ns46GCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--


Rohit Patnaik
Senior Technical Analyst, SDRC (P) Limited
m: +91.7682.8831.05
+91.8895.2916.19
skype: rohitpatnaik27
www.sdrc.co.in
SDRC enables social change initiatives.

I recommend finding a networking or systems admin contractor to assist in
diagnosing and configuring your server and network. Something is slowing
data transmission down.

I don't think there's anything more anyone off-site can do. You need
someone to come and see your networking and computing hardware, and help
with that.

··· On Tue, Oct 4, 2016 at 1:52 AM, Rohit Patnaik wrote:

Thank you Mitch.

We have tried with better internet connectivity(4G network) and submission
through WiFi but same issue is arising when the form is uploaded with
multiple(preferably 2 images).

In the device we can access our ODK Aggregate server also the submission
screen. But still we are not able to upload our data. Even the image
quality has been reduced below 400KB/image.

Hope your support will help us to resolve the issue soon.

regards,
Rohit Patnaik

On Fri, Sep 30, 2016 at 10:22 PM, Mitch Sundt mitchellsundt@gmail.com wrote:

Both of those errors indicate problems receiving the submission in a
timely manner.

I.e., slow transmission over the network to the server.

Can you open a browser on your device and access your ODK Aggregate
server? Can you view the Submissions screen?

What happens if you disable cellular data transmission and use only WiFi
for submitting data? Some cellular networks may restrict the quantity of
data you upload (favoring download speeds).

On Thu, Sep 29, 2016 at 8:07 AM, anirudha@sdrc.co.in wrote:

Yes we have already tested for faster connection but same issue is
arising when the form is uploaded with multiple images.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/opendatakit-developers/MN79ns46GCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--


Rohit Patnaik
Senior Technical Analyst, SDRC (P) Limited
m: +91.7682.8831.05
+91.8895.2916.19
skype: rohitpatnaik27
www.sdrc.co.in
SDRC enables social change initiatives.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

You should also look at the performance of Tomcat and MySQL (htop
works great for this). Do you have enough RAM and CPU? I've seen
timeouts when machines are under heavy load.

Yaw

··· On Tue, Oct 4, 2016 at 6:56 PM, Mitch Sundt wrote: > I recommend finding a networking or systems admin contractor to assist in > diagnosing and configuring your server and network. Something is slowing > data transmission down. > > I don't think there's anything more anyone off-site can do. You need someone > to come and see your networking and computing hardware, and help with that. > > > On Tue, Oct 4, 2016 at 1:52 AM, Rohit Patnaik wrote: >> >> Thank you Mitch. >> >> We have tried with better internet connectivity(4G network) and submission >> through WiFi but same issue is arising when the form is uploaded with >> multiple(preferably 2 images). >> >> In the device we can access our ODK Aggregate server also the submission >> screen. But still we are not able to upload our data. Even the image quality >> has been reduced below 400KB/image. >> >> Hope your support will help us to resolve the issue soon. >> >> regards, >> Rohit Patnaik >> >> On Fri, Sep 30, 2016 at 10:22 PM, Mitch Sundt wrote: >>> >>> Both of those errors indicate problems receiving the submission in a >>> timely manner. >>> >>> I.e., slow transmission over the network to the server. >>> >>> Can you open a browser on your device and access your ODK Aggregate >>> server? Can you view the Submissions screen? >>> >>> What happens if you disable cellular data transmission and use only WiFi >>> for submitting data? Some cellular networks may restrict the quantity of >>> data you upload (favoring download speeds). >>> >>> >>> >>> On Thu, Sep 29, 2016 at 8:07 AM, wrote: >>>> >>>> Yes we have already tested for faster connection but same issue is >>>> arising when the form is uploaded with multiple images. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "ODK Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to opendatakit-developers+unsubscribe@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> -- >>> Mitch Sundt >>> Software Engineer >>> University of Washington >>> mitchellsundt@gmail.com >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "ODK Developers" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/opendatakit-developers/MN79ns46GCg/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> opendatakit-developers+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> >> >> -- >> >> >> ------------------------------------------------------ >> Rohit Patnaik >> Senior Technical Analyst, SDRC (P) Limited >> m: +91.7682.8831.05 >> +91.8895.2916.19 >> skype: rohitpatnaik27 >> www.sdrc.co.in >> SDRC enables social change initiatives. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "ODK Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit-developers+unsubscribe@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. > > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitchellsundt@gmail.com > > -- > You received this message because you are subscribed to the Google Groups > "ODK Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to opendatakit-developers+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout.

[image: Inline image 1]

[image: Inline image 2]

Dear Anokwa,

Please find the above image of server ram and cpu performance and also
processes for mysql and tomcat. Hope this may help.

Regards,
Anirudha

··· -------------------------------------------- Anirudha Rath Infra Manager, SDRC (P) Ltd. M:-+91:7682883110

On Wed, Oct 5, 2016 at 3:29 PM, Yaw Anokwa yanokwa@nafundi.com wrote:

You should also look at the performance of Tomcat and MySQL (htop
works great for this). Do you have enough RAM and CPU? I've seen
timeouts when machines are under heavy load.

Yaw

On Tue, Oct 4, 2016 at 6:56 PM, Mitch Sundt mitchellsundt@gmail.com wrote:

I recommend finding a networking or systems admin contractor to assist in
diagnosing and configuring your server and network. Something is slowing
data transmission down.

I don't think there's anything more anyone off-site can do. You need
someone
to come and see your networking and computing hardware, and help with
that.

On Tue, Oct 4, 2016 at 1:52 AM, Rohit Patnaik rohit@sdrc.co.in wrote:

Thank you Mitch.

We have tried with better internet connectivity(4G network) and
submission
through WiFi but same issue is arising when the form is uploaded with
multiple(preferably 2 images).

In the device we can access our ODK Aggregate server also the submission
screen. But still we are not able to upload our data. Even the image
quality
has been reduced below 400KB/image.

Hope your support will help us to resolve the issue soon.

regards,
Rohit Patnaik

On Fri, Sep 30, 2016 at 10:22 PM, Mitch Sundt mitchellsundt@gmail.com wrote:

Both of those errors indicate problems receiving the submission in a
timely manner.

I.e., slow transmission over the network to the server.

Can you open a browser on your device and access your ODK Aggregate
server? Can you view the Submissions screen?

What happens if you disable cellular data transmission and use only
WiFi
for submitting data? Some cellular networks may restrict the quantity
of
data you upload (favoring download speeds).

On Thu, Sep 29, 2016 at 8:07 AM, anirudha@sdrc.co.in wrote:

Yes we have already tested for faster connection but same issue is
arising when the form is uploaded with multiple images.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit-developers/
MN79ns46GCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--


Rohit Patnaik
Senior Technical Analyst, SDRC (P) Limited
m: +91.7682.8831.05
+91.8895.2916.19
skype: rohitpatnaik27
www.sdrc.co.in
SDRC enables social change initiatives.

--
You received this message because you are subscribed to the Google
Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Anirudha,

The screenshots you send look fine, but as Mitch suggested you'll likely
need someone who has access to your network and machines to dig into the
problem more.

Yaw

··· On Wed, Oct 5, 2016 at 1:10 PM, Anirudha Rath wrote:

[image: Inline image 1]

[image: Inline image 2]

Dear Anokwa,

Please find the above image of server ram and cpu performance and also
processes for mysql and tomcat. Hope this may help.

Regards,
Anirudha

Anirudha Rath
Infra Manager, SDRC (P) Ltd.
M:-+91:7682883110

On Wed, Oct 5, 2016 at 3:29 PM, Yaw Anokwa yanokwa@nafundi.com wrote:

You should also look at the performance of Tomcat and MySQL (htop
works great for this). Do you have enough RAM and CPU? I've seen
timeouts when machines are under heavy load.

Yaw

On Tue, Oct 4, 2016 at 6:56 PM, Mitch Sundt mitchellsundt@gmail.com wrote:

I recommend finding a networking or systems admin contractor to assist
in
diagnosing and configuring your server and network. Something is slowing
data transmission down.

I don't think there's anything more anyone off-site can do. You need
someone
to come and see your networking and computing hardware, and help with
that.

On Tue, Oct 4, 2016 at 1:52 AM, Rohit Patnaik rohit@sdrc.co.in wrote:

Thank you Mitch.

We have tried with better internet connectivity(4G network) and
submission
through WiFi but same issue is arising when the form is uploaded with
multiple(preferably 2 images).

In the device we can access our ODK Aggregate server also the
submission
screen. But still we are not able to upload our data. Even the image
quality
has been reduced below 400KB/image.

Hope your support will help us to resolve the issue soon.

regards,
Rohit Patnaik

On Fri, Sep 30, 2016 at 10:22 PM, Mitch Sundt <mitchellsundt@gmail.com wrote:

Both of those errors indicate problems receiving the submission in a
timely manner.

I.e., slow transmission over the network to the server.

Can you open a browser on your device and access your ODK Aggregate
server? Can you view the Submissions screen?

What happens if you disable cellular data transmission and use only
WiFi
for submitting data? Some cellular networks may restrict the quantity
of
data you upload (favoring download speeds).

On Thu, Sep 29, 2016 at 8:07 AM, anirudha@sdrc.co.in wrote:

Yes we have already tested for faster connection but same issue is
arising when the form is uploaded with multiple images.

--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit-developers/MN7
9ns46GCg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--


Rohit Patnaik
Senior Technical Analyst, SDRC (P) Limited
m: +91.7682.8831.05
+91.8895.2916.19
skype: rohitpatnaik27
www.sdrc.co.in
SDRC enables social change initiatives.

--
You received this message because you are subscribed to the Google
Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to the Google
Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.