ODKAggregate Socket Timeout issue - Tomcat Stops Working

PLD3 · April 25, 2014, 7:35am

Hello,

We are facing an error with ODKAggregate. Tomcat server keeps stop working and when we checked the logs of tomcat server then we see the below error:

Apr 23, 2014 11:12:16 AM org.opendatakit.aggregate.servlet.SubmissionServlet doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751)
at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781)
at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:100)
at org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at

We searched above error on Google Group and found this thread: https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8

According to Mitch reply we changed the tomcat connection timeout from tomcat settings and make sure that picture size in ODK From is small. But still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly help us to solve this problem.

Thanks,
PLD3

catalina.out (1.18 MB)

Mitch_S · April 25, 2014, 5:12pm

Do you have this issue only when submitting data from the device ( ODK
Collect ), or do you also see this if you upload a submission using the
webpage interface (log in as Site Admin, go to Form Management / Submission
Admin, and click the "Manually Add Submission Data" button.

If it occurs only when using the device / ODK Collect, are you using 3G, 4G
or WiFi for the transmission?

If it does happen only when submitting data from the device, please collect
a logtrace off the device during the time you are submitting the data (and
when the submission fails).

See http://code.google.com/p/opendatakit/wiki/CollectTroubleshooting

···

On Fri, Apr 25, 2014 at 12:35 AM, PLD3 wrote:

Hello,

We are facing an error with ODKAggregate. Tomcat server keeps stop working
and when we checked the logs of tomcat server then we see the below error:

Apr 23, 2014 11:12:16 AM
org.opendatakit.aggregate.servlet.SubmissionServlet doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:146)
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751)
at
org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781)
at
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
at
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710)
at org.apache.coyote.Request.doRead(Request.java:427)
at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)
at
org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)
at
org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)
at
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at
org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at
org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:100)
at
org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at

We searched above error on Google Group and found this thread:
https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8

According to Mitch reply we changed the tomcat connection timeout from
tomcat settings and make sure that picture size in ODK From is small. But
still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly
help us to solve this problem.

Thanks,
PLD3

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

PLD3 · April 29, 2014, 10:25am

Hello Mitch,

Yes we have only this Issue while submitting from ODK Collect application. No we are not using 3G, 4G or WiFi for the transmission. Its submitting through GPRS/Edge.

I will collect the log trace of device and will submit on this thread soon. What can be the possible issue for this problem?

PLD3 · May 3, 2014, 12:35pm

Hi Mitch,

We are unable to generate the logs of device. I have used aLogcat application to catch the logs but nothing shows in the logs while this issue comes. According to my understanding aLogcat show logs for crashes but our ODK Application is not crashing and it not showing any logs.

When this issue comes On ODK Application we saw this message "Downtime Error (Service Temporarily Down 503 Error)". Tomcat server getting shutdown again and it seems that there is some issue with processing of ODKAggregate. Kindly Guide.

Thanks,
PLD3 Team

···

On Friday, April 25, 2014 10:12:57 PM UTC+5, Mitch Sundt wrote: > Do you have this issue only when submitting data from the device ( ODK Collect ), or do you also see this if you upload a submission using the webpage interface (log in as Site Admin, go to Form Management / Submission Admin, and click the "Manually Add Submission Data" button. > > > If it occurs only when using the device / ODK Collect, are you using 3G, 4G or WiFi for the transmission? > > If it does happen only when submitting data from the device, please collect a logtrace off the device during the time you are submitting the data (and when the submission fails). > > > See http://code.google.com/p/opendatakit/wiki/CollectTroubleshooting > > > > > > > > > On Fri, Apr 25, 2014 at 12:35 AM, PLD3 wrote: > > Hello, > > > > We are facing an error with ODKAggregate. Tomcat server keeps stop working and when we checked the logs of tomcat server then we see the below error: > > > > Apr 23, 2014 11:12:16 AM org.opendatakit.aggregate.servlet.SubmissionServlet doPost > > SEVERE: Unexpected exception: Read timed out > > java.net.SocketTimeoutException: Read timed out > > at java.net.SocketInputStream.socketRead0(Native Method) > > at java.net.SocketInputStream.read(SocketInputStream.java:146) > > at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751) > > at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781) > > at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116) > > at org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710) > > at org.apache.coyote.Request.doRead(Request.java:427) > > at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304) > > at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419) > > at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327) > > at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193) > > at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976) > > at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886) > > at java.io.FilterInputStream.read(FilterInputStream.java:133) > > at org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125) > > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) > > at org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:100) > > at org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > > at > > > > We searched above error on Google Group and found this thread: https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8 > > > > > According to Mitch reply we changed the tomcat connection timeout from tomcat settings and make sure that picture size in ODK From is small. But still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly help us to solve this problem. > > > > > Thanks, > > PLD3 > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google Groups "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com

yanokwa · May 3, 2014, 8:14pm

The point of capturing the logs was not necessarily to catch a crash,
but to get a general sense of what was going on in the app at the time
of submission. If nothing is showing up, then don't use aLogcat.
Install the Android SDK and use adb.
https://code.google.com/p/opendatakit/wiki/CollectTroubleshooting has
instructions.

Your server logs shows six types of severe exceptions.

70x: SEVERE: Unexpected exception: Read timed out
6x: SEVERE: Unexpected exception: Stream ended unexpectedly
6x: SEVERE: Servlet.service() for servlet binaryData threw exception
1x: SEVERE: BAD packet signature 5635
1x: SEVERE: The web application [/ODKAggregate] appears to have
started a thread named [Abandoned connection cleanup thread] but has
failed to stop it. This is very likely to create a memory leak.
1x: SEVERE: The web application [/ODKAggregate] appears to have
started a thread named [com.google.common.base.internal.Finalizer] but
has failed to stop it. This is very likely to create a memory leak.

Sorted by frequency, it definitely suggests that you are having some
connectivity issues (lots of timeouts and stream ending). But, I don't
think that would result in crashes. My guess is that you are running
out of memory in Tomcat.

If you look at Servlet.service() for servlet binaryData threw
exception, you'll see that it reports java.lang.OutOfMemoryError: Java
heap space immediately after.

Give Tomcat as much RAM as you can give and see if the problem goes away.

Yaw

···

-- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Sat, May 3, 2014 at 5:35 AM, ameerh90@gmail.com wrote:

Hi Mitch,

We are unable to generate the logs of device. I have used aLogcat application to catch the logs but nothing shows in the logs while this issue comes. According to my understanding aLogcat show logs for crashes but our ODK Application is not crashing and it not showing any logs.

When this issue comes On ODK Application we saw this message "Downtime Error (Service Temporarily Down 503 Error)". Tomcat server getting shutdown again and it seems that there is some issue with processing of ODKAggregate. Kindly Guide.

Thanks,
PLD3 Team

On Friday, April 25, 2014 10:12:57 PM UTC+5, Mitch Sundt wrote:
Do you have this issue only when submitting data from the device ( ODK Collect ), or do you also see this if you upload a submission using the webpage interface (log in as Site Admin, go to Form Management / Submission Admin, and click the "Manually Add Submission Data" button.

If it occurs only when using the device / ODK Collect, are you using 3G, 4G or WiFi for the transmission?

If it does happen only when submitting data from the device, please collect a logtrace off the device during the time you are submitting the data (and when the submission fails).

See http://code.google.com/p/opendatakit/wiki/CollectTroubleshooting

On Fri, Apr 25, 2014 at 12:35 AM, PLD3 amee...@gmail.com wrote:

Hello,

We are facing an error with ODKAggregate. Tomcat server keeps stop working and when we checked the logs of tomcat server then we see the below error:

Apr 23, 2014 11:12:16 AM org.opendatakit.aggregate.servlet.SubmissionServlet doPost

SEVERE: Unexpected exception: Read timed out

java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)

    at java.net.SocketInputStream.read(SocketInputStream.java:146)

    at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751)

    at org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781)

    at org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)

    at org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710)

    at org.apache.coyote.Request.doRead(Request.java:427)

    at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)

    at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)

    at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)

    at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)

    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)

    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)

    at java.io.FilterInputStream.read(FilterInputStream.java:133)

    at org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125)

    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)

    at java.io.BufferedInputStream.read(BufferedInputStream.java:254)

    at org.opendatakit.aggregate.parser.MultiPartFormData.<init>(MultiPartFormData.java:100)

    at org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

    at
We searched above error on Google Group and found this thread: https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8

According to Mitch reply we changed the tomcat connection timeout from tomcat settings and make sure that picture size in ODK From is small. But still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly help us to solve this problem.

Thanks,

PLD3

--

--

Post: opend...@googlegroups.com

Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups "ODK Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com
--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

PLD3 · May 15, 2014, 11:32am

Hi Yaw,

I have followed your suggestion and increase the allocated RAM to Tomcat6
but issue is still there and server stop working again. Our server has
total 1GB RAM and earlier tomcat was having only 512MB RAM allocated, I
upgraded to 1GB RAM after your suggestion but after a week same "Read timed
out" happened again.

We can upgrade our server with bigger one with 4GB RAM but needs to hear
from you guys about this. Will 4GB RAM solve this issue? What should be the
server specification at least to run ODKAggregate?

Kindly Advise.

Thanks,
PLD3 Team

···

On Sunday, May 4, 2014 1:14:20 AM UTC+5, Yaw Anokwa wrote: > > The point of capturing the logs was not necessarily to catch a crash, > but to get a general sense of what was going on in the app at the time > of submission. If nothing is showing up, then don't use aLogcat. > Install the Android SDK and use adb. > https://code.google.com/p/opendatakit/wiki/CollectTroubleshooting has > instructions. > > Your server logs shows six types of severe exceptions. > > 70x: SEVERE: Unexpected exception: Read timed out > 6x: SEVERE: Unexpected exception: Stream ended unexpectedly > 6x: SEVERE: Servlet.service() for servlet binaryData threw exception > 1x: SEVERE: BAD packet signature 5635 > 1x: SEVERE: The web application [/ODKAggregate] appears to have > started a thread named [Abandoned connection cleanup thread] but has > failed to stop it. This is very likely to create a memory leak. > 1x: SEVERE: The web application [/ODKAggregate] appears to have > started a thread named [com.google.common.base.internal.Finalizer] but > has failed to stop it. This is very likely to create a memory leak. > > Sorted by frequency, it definitely suggests that you are having some > connectivity issues (lots of timeouts and stream ending). But, I don't > think that would result in crashes. My guess is that you are running > out of memory in Tomcat. > > If you look at Servlet.service() for servlet binaryData threw > exception, you'll see that it reports java.lang.OutOfMemoryError: Java > heap space immediately after. > > Give Tomcat as much RAM as you can give and see if the problem goes away. > > Yaw > -- > Need ODK services? http://nafundi.com provides form design, server > setup, professional support, and software development for ODK. > > > > On Sat, May 3, 2014 at 5:35 AM, <amee...@gmail.com > wrote: > > Hi Mitch, > > > > We are unable to generate the logs of device. I have used aLogcat > application to catch the logs but nothing shows in the logs while this > issue comes. According to my understanding aLogcat show logs for crashes > but our ODK Application is not crashing and it not showing any logs. > > > > When this issue comes On ODK Application we saw this message "Downtime > Error (Service Temporarily Down 503 Error)". Tomcat server getting shutdown > again and it seems that there is some issue with processing of > ODKAggregate. Kindly Guide. > > > > Thanks, > > PLD3 Team > > > > > > > > > > On Friday, April 25, 2014 10:12:57 PM UTC+5, Mitch Sundt wrote: > >> Do you have this issue only when submitting data from the device ( ODK > Collect ), or do you also see this if you upload a submission using the > webpage interface (log in as Site Admin, go to Form Management / Submission > Admin, and click the "Manually Add Submission Data" button. > >> > >> > >> If it occurs only when using the device / ODK Collect, are you using > 3G, 4G or WiFi for the transmission? > >> > >> If it does happen only when submitting data from the device, please > collect a logtrace off the device during the time you are submitting the > data (and when the submission fails). > >> > >> > >> See http://code.google.com/p/opendatakit/wiki/CollectTroubleshooting > >> > >> > >> > >> > >> > >> > >> > >> > >> On Fri, Apr 25, 2014 at 12:35 AM, PLD3 wrote: > >> > >> Hello, > >> > >> > >> > >> We are facing an error with ODKAggregate. Tomcat server keeps stop > working and when we checked the logs of tomcat server then we see the below > error: > >> > >> > >> > >> Apr 23, 2014 11:12:16 AM > org.opendatakit.aggregate.servlet.SubmissionServlet doPost > >> > >> SEVERE: Unexpected exception: Read timed out > >> > >> java.net.SocketTimeoutException: Read timed out > >> > >> at java.net.SocketInputStream.socketRead0(Native Method) > >> > >> at java.net.SocketInputStream.read(SocketInputStream.java:146) > >> > >> at > org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751) > > >> > >> at > org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781) > > >> > >> at > org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116) > > >> > >> at > org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710) > > >> > >> at org.apache.coyote.Request.doRead(Request.java:427) > >> > >> at > org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304) > > >> > >> at > org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419) > >> > >> at > org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327) > >> > >> at > org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193) > > >> > >> at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976) > > >> > >> at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886) > > >> > >> at java.io.FilterInputStream.read(FilterInputStream.java:133) > >> > >> at > org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125) > > >> > >> at > java.io.BufferedInputStream.fill(BufferedInputStream.java:235) > >> > >> at > java.io.BufferedInputStream.read(BufferedInputStream.java:254) > >> > >> at > org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:100) > > >> > >> at > org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223) > > >> > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:637) > >> > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) > >> > >> at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > > >> > >> at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > > >> > >> at > >> > >> > >> > >> We searched above error on Google Group and found this thread: > https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8 > >> > >> > >> > >> > >> According to Mitch reply we changed the tomcat connection timeout from > tomcat settings and make sure that picture size in ODK From is small. But > still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly > help us to solve this problem. > >> > >> > >> > >> > >> Thanks, > >> > >> PLD3 > >> > >> > >> > >> -- > >> > >> -- > >> > >> Post: opend...@googlegroups.com > >> > >> Unsubscribe: opendatakit...@googlegroups.com > >> > >> Options: http://groups.google.com/group/opendatakit?hl=en > >> > >> > >> > >> --- > >> > >> You received this message because you are subscribed to the Google > Groups "ODK Community" group. > >> > >> To unsubscribe from this group and stop receiving emails from it, send > an email to opendatakit...@googlegroups.com. > >> > >> For more options, visit https://groups.google.com/d/optout. > >> > >> > >> > >> > >> -- > >> Mitch Sundt > >> Software Engineer > >> University of Washington > >> mitche...@gmail.com > > > > -- > > -- > > Post: opend...@googlegroups.com > > Unsubscribe: opendatakit...@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google > Groups "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to opendatakit...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. >

yanokwa · May 16, 2014, 5:32pm

No idea if 4GB RAM will solve the issue. Depends on if you are
actually running out of RAM or not and how easy/cheap it is to get
more RAM. Do your logs still report heap issues?

As far as the timeout, you just need to keep testing until it's
reproducible. Does it happen with all forms? What about a small form?
What about submitting over WiFi? What about 3G? Does it happen if you
have no pictures? Does it happen on Google App Engine?

Yaw

···

-- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Thu, May 15, 2014 at 4:32 AM, PLD3 ameerh90@gmail.com wrote:

Hi Yaw,

I have followed your suggestion and increase the allocated RAM to Tomcat6
but issue is still there and server stop working again. Our server has total
1GB RAM and earlier tomcat was having only 512MB RAM allocated, I upgraded
to 1GB RAM after your suggestion but after a week same "Read timed out"
happened again.

We can upgrade our server with bigger one with 4GB RAM but needs to hear
from you guys about this. Will 4GB RAM solve this issue? What should be the
server specification at least to run ODKAggregate?

Kindly Advise.

Thanks,
PLD3 Team

On Sunday, May 4, 2014 1:14:20 AM UTC+5, Yaw Anokwa wrote:
The point of capturing the logs was not necessarily to catch a crash,
but to get a general sense of what was going on in the app at the time
of submission. If nothing is showing up, then don't use aLogcat.
Install the Android SDK and use adb.
https://code.google.com/p/opendatakit/wiki/CollectTroubleshooting has
instructions.

Your server logs shows six types of severe exceptions.

70x: SEVERE: Unexpected exception: Read timed out
6x: SEVERE: Unexpected exception: Stream ended unexpectedly
6x: SEVERE: Servlet.service() for servlet binaryData threw exception
1x: SEVERE: BAD packet signature 5635
1x: SEVERE: The web application [/ODKAggregate] appears to have
started a thread named [Abandoned connection cleanup thread] but has
failed to stop it. This is very likely to create a memory leak.
1x: SEVERE: The web application [/ODKAggregate] appears to have
started a thread named [com.google.common.base.internal.Finalizer] but
has failed to stop it. This is very likely to create a memory leak.

Sorted by frequency, it definitely suggests that you are having some
connectivity issues (lots of timeouts and stream ending). But, I don't
think that would result in crashes. My guess is that you are running
out of memory in Tomcat.

If you look at Servlet.service() for servlet binaryData threw
exception, you'll see that it reports java.lang.OutOfMemoryError: Java
heap space immediately after.

Give Tomcat as much RAM as you can give and see if the problem goes away.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Sat, May 3, 2014 at 5:35 AM, amee...@gmail.com wrote:
Hi Mitch,

We are unable to generate the logs of device. I have used aLogcat
application to catch the logs but nothing shows in the logs while this issue
comes. According to my understanding aLogcat show logs for crashes but our
ODK Application is not crashing and it not showing any logs.

When this issue comes On ODK Application we saw this message "Downtime
Error (Service Temporarily Down 503 Error)". Tomcat server getting shutdown
again and it seems that there is some issue with processing of ODKAggregate.
Kindly Guide.

Thanks,
PLD3 Team

On Friday, April 25, 2014 10:12:57 PM UTC+5, Mitch Sundt wrote:
Do you have this issue only when submitting data from the device ( ODK
Collect ), or do you also see this if you upload a submission using the
webpage interface (log in as Site Admin, go to Form Management / Submission
Admin, and click the "Manually Add Submission Data" button.

If it occurs only when using the device / ODK Collect, are you using
3G, 4G or WiFi for the transmission?

If it does happen only when submitting data from the device, please
collect a logtrace off the device during the time you are submitting the
data (and when the submission fails).

See http://code.google.com/p/opendatakit/wiki/CollectTroubleshooting

On Fri, Apr 25, 2014 at 12:35 AM, PLD3 amee...@gmail.com wrote:

Hello,

We are facing an error with ODKAggregate. Tomcat server keeps stop
working and when we checked the logs of tomcat server then we see the below
error:

Apr 23, 2014 11:12:16 AM
org.opendatakit.aggregate.servlet.SubmissionServlet doPost

SEVERE: Unexpected exception: Read timed out

java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)

    at java.net.SocketInputStream.read(SocketInputStream.java:146)

    at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:751)
    at
org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:781)
    at
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
    at
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:710)
    at org.apache.coyote.Request.doRead(Request.java:427)

    at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)
    at
org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)
    at
org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)
    at
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
    at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)
    at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
    at java.io.FilterInputStream.read(FilterInputStream.java:133)

    at
org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125)
    at
java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
    at
java.io.BufferedInputStream.read(BufferedInputStream.java:254)
    at
org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:100)
    at
org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)

    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

    at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at
We searched above error on Google Group and found this thread:
https://groups.google.com/forum/#!topic/opendatakit/XrRz5L1Mor8

According to Mitch reply we changed the tomcat connection timeout from
tomcat settings and make sure that picture size in ODK From is small. But
still issue is coming. We are using ODKAggregate V1.4 (latest one). Kindly
help us to solve this problem.

Thanks,

PLD3

--

--

Post: opend...@googlegroups.com

Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.

To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com
--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

everst0ne · October 18, 2022, 9:10am

I've had the same problem.

Tried increasing the memory of JVM and tried setting the connectionTimeout to 100 secs but still the error occurs.

If I were to send 10 data, only 2 - 3 will be successful. Also tried on different devices. Situation is exactly the same, using wifi, no firewall. But still we have not solved the problem.

Please help
catalina2.txt (1.6 MB)

everst0ne · November 15, 2022, 8:48am

@yanokwa Please help

yanokwa · November 15, 2022, 5:22pm

Aggregate has reached end-of-life and is not supported by the core team. Please use Central instead.

If this is an urgent issue, considering posting in the marketplace and paying someone to look into why your connections are timing out.