Everything was fine last week, now I can't send finalized forms

Patrick_McKay · February 17, 2014, 2:22pm

Like the title says, last week everything was fine, this week I get the
following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:756)
at
org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116)
at
org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327)
at
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976)
at
org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at
org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at
org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:97)
at
org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.opendatakit.common.security.spring.SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
at
org.opendatakit.common.security.spring.OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.java:40)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.authentication.www.DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:115)
at
org.opendatakit.common.security.spring.DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:40)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.opendatakit.common.security.spring.OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.opendatakit.common.security.spring.Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server has been
restarted. Any ideas? I'm tearing my hair out!

Patrick_McKay · February 17, 2014, 3:09pm

It appears to be related to images. Files with images fail, those without
go through. I'm using tomcat with Postrgre. Any ideas where to look?

···

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote: > > Like the title says, last week everything was fine, this week I get the > following error when trying to send finalised forms: > > Feb 17, 2014 4:17:49 PM > org.opendatakit.aggregate.servlet.SubmissionServlet doHead > INFO: Inside doHead > Feb 17, 2014 4:18:10 PM > org.opendatakit.aggregate.servlet.SubmissionServlet doPost > SEVERE: Unexpected exception: Read timed out > java.net.SocketTimeoutException: Read timed out > at java.net.SocketInputStream.socketRead0(Native Method) > at java.net.SocketInputStream.read(Unknown Source) > at java.net.SocketInputStream.read(Unknown Source) > at > org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:756) > at > org.apache.coyote.http11.InternalInputBuffer$InputStreamInputBuffer.doRead(InternalInputBuffer.java:786) > at > org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:116) > at > org.apache.coyote.http11.InternalInputBuffer.doRead(InternalInputBuffer.java:715) > at org.apache.coyote.Request.doRead(Request.java:427) > at > org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:304) > at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419) > at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:327) > at > org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:193) > at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:976) > at > org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886) > at java.io.FilterInputStream.read(Unknown Source) > at > org.apache.commons.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:125) > at java.io.BufferedInputStream.fill(Unknown Source) > at java.io.BufferedInputStream.read(Unknown Source) > at > org.opendatakit.aggregate.parser.MultiPartFormData.(MultiPartFormData.java:97) > at > org.opendatakit.aggregate.servlet.SubmissionServlet.doPost(SubmissionServlet.java:223) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:643) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:723) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.opendatakit.common.security.spring.SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(SecurityContextHolderAwareAuthPreservingRequestFilter.java:66) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183) > at > org.opendatakit.common.security.spring.OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.java:40) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.www.DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:115) > at > org.opendatakit.common.security.spring.DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:40) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.opendatakit.common.security.spring.OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.opendatakit.common.security.spring.Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) > at > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Unknown Source) > > > I have updated to the latest ODK. Same error for all forms. Server has > been restarted. Any ideas? I'm tearing my hair out! >

Mitch_S · February 17, 2014, 4:44pm

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the directories
underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G, WiFi,
Satellite), it may take a very long time to transmit your images to your
ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to increase how
long it waits for data sent to it. If everything is as you expect, change
the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid 3G and
satellite connections due to their reduced bandwidth. They can work, but
are much slower than WiFi or 4G.

Mitch

···

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay wrote:

It appears to be related to images. Files with images fail, those without
go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get the
following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.
doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.OpenIDAuthenticationFilter.
doFilter(OpenIDAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:115)
at org.opendatakit.common.security.spring.DigestAuthenticationFilter.
doFilter(DigestAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilt
er.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(
DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server has
been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Patrick_McKay · February 17, 2014, 5:55pm

I've set image resolution to a minimum, even taken a completely black image
which was just 10kb but it also fails. I'm on Wi-Fi with a 2mb connection
in the background. I did try adjusting my way time from 20000 to 60000 but
there was no change. The same error just take longer. I'm really confused
here. I saw this thread about a similar problem but with mysql:
https://groups.google.com/forum/?nomobile=true#!searchin/opendatakit/images/opendatakit/9iYBEvvJCd8

Does postgre have a similar setting?

···

On Feb 17, 2014 6:44 PM, "Mitch Sundt" wrote:

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the directories
underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G,
WiFi, Satellite), it may take a very long time to transmit your images to
your ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to increase
how long it waits for data sent to it. If everything is as you expect,
change the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid 3G
and satellite connections due to their reduced bandwidth. They can work,
but are much slower than WiFi or 4G.

Mitch

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay mckayp@gmail.com wrote:

It appears to be related to images. Files with images fail, those without
go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get the
following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.
java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.
doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.
OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:115)
at org.opendatakit.common.security.spring.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilt
er.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(
DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server has
been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Mitch_S · February 17, 2014, 7:08pm

If you have the same problem with the 10kb image, then it rules out any
issue similar to the MySQL communications buffer problem. I am not aware of
any similar limitation in PostgreSQL.

Can you submit the form and image via the Submission Upload button on the
ODK Aggregate website from your own computer (the one hosting ODK
Aggregate)?

That would isolate the problem to your ODK Aggregate server setup.

If it works on your computer, then test it from another computer on your
network using its browser to submit data to ODK Aggregate.

If that works, then try the same from your device, using the Chrome browser
on the device.

If one of these won't work, then it would indicate some sort of network /
router / firewall problem.

If all of those work, but ODK Collect does not,

Open a browser and change the 'Aggregate.html' to 'xformsList' and look at
the downloadUrl field in the returned set.

Is the http/https, server, port and application prefix portion (everything
before /formXml ) of that the URL what you are using to access your server?

If it does not match, then you need to re-run the ODK Aggregate installer
to specify the correct values for these.

ODK Collect, in its negotiation with ODK Aggregate, will redirect to that
URL prefix (followed by /submission) before it sends any data.

This allows for misconfiguration of the device (e.g., http instead of
https) while still enforcing secure transmission of data to the server.

Mitch

···

On Mon, Feb 17, 2014 at 9:55 AM, Patrick McKay wrote:

I've set image resolution to a minimum, even taken a completely black
image which was just 10kb but it also fails. I'm on Wi-Fi with a 2mb
connection in the background. I did try adjusting my way time from 20000 to
60000 but there was no change. The same error just take longer. I'm really
confused here. I saw this thread about a similar problem but with mysql:
https://groups.google.com/forum/?nomobile=true#!searchin/opendatakit/images/opendatakit/9iYBEvvJCd8

Does postgre have a similar setting?
On Feb 17, 2014 6:44 PM, "Mitch Sundt" mitchellsundt@gmail.com wrote:

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the directories
underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G,
WiFi, Satellite), it may take a very long time to transmit your images to
your ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to increase
how long it waits for data sent to it. If everything is as you expect,
change the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid 3G
and satellite connections due to their reduced bandwidth. They can work,
but are much slower than WiFi or 4G.

Mitch

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay mckayp@gmail.com wrote:

It appears to be related to images. Files with images fail, those
without go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get the
following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.
java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.
SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.
OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:115)
at org.opendatakit.common.security.spring.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(
SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(
DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.
process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server has
been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Patrick_McKay · February 19, 2014, 8:19am

It appears to be the internet connection at the office. When I went past a
restaurant with wifi, it transmitted the forms with pictures. Now I'd like
to figure out what is causing the problem at the office. Are the images
sent through a different port or protocol that our firewall may be blocking?

···

On Mon, Feb 17, 2014 at 9:08 PM, Mitch Sundt wrote:

If you have the same problem with the 10kb image, then it rules out any
issue similar to the MySQL communications buffer problem. I am not aware of
any similar limitation in PostgreSQL.

Can you submit the form and image via the Submission Upload button on the
ODK Aggregate website from your own computer (the one hosting ODK
Aggregate)?

That would isolate the problem to your ODK Aggregate server setup.

If it works on your computer, then test it from another computer on your
network using its browser to submit data to ODK Aggregate.

If that works, then try the same from your device, using the Chrome
browser on the device.

If one of these won't work, then it would indicate some sort of network /
router / firewall problem.

If all of those work, but ODK Collect does not,

Open a browser and change the 'Aggregate.html' to 'xformsList' and look at
the downloadUrl field in the returned set.

Is the http/https, server, port and application prefix portion (everything
before /formXml ) of that the URL what you are using to access your server?

If it does not match, then you need to re-run the ODK Aggregate installer
to specify the correct values for these.

ODK Collect, in its negotiation with ODK Aggregate, will redirect to that
URL prefix (followed by /submission) before it sends any data.

This allows for misconfiguration of the device (e.g., http instead of
https) while still enforcing secure transmission of data to the server.

Mitch

On Mon, Feb 17, 2014 at 9:55 AM, Patrick McKay mckayp@gmail.com wrote:

I've set image resolution to a minimum, even taken a completely black
image which was just 10kb but it also fails. I'm on Wi-Fi with a 2mb
connection in the background. I did try adjusting my way time from 20000 to
60000 but there was no change. The same error just take longer. I'm really
confused here. I saw this thread about a similar problem but with mysql:
https://groups.google.com/forum/?nomobile=true#!searchin/opendatakit/images/opendatakit/9iYBEvvJCd8

Does postgre have a similar setting?
On Feb 17, 2014 6:44 PM, "Mitch Sundt" mitchellsundt@gmail.com wrote:

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the directories
underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G,
WiFi, Satellite), it may take a very long time to transmit your images to
your ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to increase
how long it waits for data sent to it. If everything is as you expect,
change the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid 3G
and satellite connections due to their reduced bandwidth. They can work,
but are much slower than WiFi or 4G.

Mitch

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay mckayp@gmail.com wrote:

It appears to be related to images. Files with images fail, those
without go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get
the following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.
java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.
SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.
OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:115)
at org.opendatakit.common.security.spring.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(
SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(
DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$
Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server has
been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Mitch_S · February 19, 2014, 6:20pm

Nope. Everything comes through the same scheme, hostname and port.

Also, the images are sent as part of a form-data-post body, never as naked
image files during the submission process. The spec is here:
https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI and
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaRequest and
https://bitbucket.org/javarosa/javarosa/wiki/AuthenticationAPI

So if these are getting blocked, it would indicate that something is
looking at form-data POST bodies? Perhaps doing intelligent filtering of
your network activity?

Please let me know what you discover. I've never heard of this happening.

Mitch

···

On Wed, Feb 19, 2014 at 12:19 AM, Patrick McKay wrote:

It appears to be the internet connection at the office. When I went past a
restaurant with wifi, it transmitted the forms with pictures. Now I'd like
to figure out what is causing the problem at the office. Are the images
sent through a different port or protocol that our firewall may be blocking?

On Mon, Feb 17, 2014 at 9:08 PM, Mitch Sundt mitchellsundt@gmail.comwrote:

If you have the same problem with the 10kb image, then it rules out any
issue similar to the MySQL communications buffer problem. I am not aware of
any similar limitation in PostgreSQL.

Can you submit the form and image via the Submission Upload button on the
ODK Aggregate website from your own computer (the one hosting ODK
Aggregate)?

That would isolate the problem to your ODK Aggregate server setup.

If it works on your computer, then test it from another computer on your
network using its browser to submit data to ODK Aggregate.

If that works, then try the same from your device, using the Chrome
browser on the device.

If one of these won't work, then it would indicate some sort of network /
router / firewall problem.

If all of those work, but ODK Collect does not,

Open a browser and change the 'Aggregate.html' to 'xformsList' and look
at the downloadUrl field in the returned set.

Is the http/https, server, port and application prefix portion
(everything before /formXml ) of that the URL what you are using to access
your server?

If it does not match, then you need to re-run the ODK Aggregate installer
to specify the correct values for these.

ODK Collect, in its negotiation with ODK Aggregate, will redirect to that
URL prefix (followed by /submission) before it sends any data.

This allows for misconfiguration of the device (e.g., http instead of
https) while still enforcing secure transmission of data to the server.

Mitch

On Mon, Feb 17, 2014 at 9:55 AM, Patrick McKay mckayp@gmail.com wrote:

I've set image resolution to a minimum, even taken a completely black
image which was just 10kb but it also fails. I'm on Wi-Fi with a 2mb
connection in the background. I did try adjusting my way time from 20000 to
60000 but there was no change. The same error just take longer. I'm really
confused here. I saw this thread about a similar problem but with mysql:
https://groups.google.com/forum/?nomobile=true#!searchin/opendatakit/images/opendatakit/9iYBEvvJCd8

Does postgre have a similar setting?
On Feb 17, 2014 6:44 PM, "Mitch Sundt" mitchellsundt@gmail.com wrote:

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the directories
underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G,
WiFi, Satellite), it may take a very long time to transmit your images to
your ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to increase
how long it waits for data sent to it. If everything is as you expect,
change the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid 3G
and satellite connections due to their reduced bandwidth. They can work,
but are much slower than WiFi or 4G.

Mitch

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay mckayp@gmail.comwrote:

It appears to be related to images. Files with images fail, those
without go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get
the following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.
java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.
SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.
OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:115)
at org.opendatakit.common.security.spring.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(
SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(
DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$
Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server
has been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Mitch_S · February 19, 2014, 6:21pm

If you have secure logins, you might try allowing anonymous submissions to
see if it is related to the Digest Auth protocol.

···

On Wed, Feb 19, 2014 at 10:20 AM, Mitch Sundt wrote:

Nope. Everything comes through the same scheme, hostname and port.

Also, the images are sent as part of a form-data-post body, never as naked
image files during the submission process. The spec is here:
https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI and
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaRequest and
https://bitbucket.org/javarosa/javarosa/wiki/AuthenticationAPI

So if these are getting blocked, it would indicate that something is
looking at form-data POST bodies? Perhaps doing intelligent filtering of
your network activity?

Please let me know what you discover. I've never heard of this happening.

Mitch

On Wed, Feb 19, 2014 at 12:19 AM, Patrick McKay mckayp@gmail.com wrote:

It appears to be the internet connection at the office. When I went past
a restaurant with wifi, it transmitted the forms with pictures. Now I'd
like to figure out what is causing the problem at the office. Are the
images sent through a different port or protocol that our firewall may be
blocking?

On Mon, Feb 17, 2014 at 9:08 PM, Mitch Sundt mitchellsundt@gmail.comwrote:

If you have the same problem with the 10kb image, then it rules out any
issue similar to the MySQL communications buffer problem. I am not aware of
any similar limitation in PostgreSQL.

Can you submit the form and image via the Submission Upload button on
the ODK Aggregate website from your own computer (the one hosting ODK
Aggregate)?

That would isolate the problem to your ODK Aggregate server setup.

If it works on your computer, then test it from another computer on your
network using its browser to submit data to ODK Aggregate.

If that works, then try the same from your device, using the Chrome
browser on the device.

If one of these won't work, then it would indicate some sort of network
/ router / firewall problem.

If all of those work, but ODK Collect does not,

Open a browser and change the 'Aggregate.html' to 'xformsList' and look
at the downloadUrl field in the returned set.

Is the http/https, server, port and application prefix portion
(everything before /formXml ) of that the URL what you are using to access
your server?

If it does not match, then you need to re-run the ODK Aggregate
installer to specify the correct values for these.

ODK Collect, in its negotiation with ODK Aggregate, will redirect to
that URL prefix (followed by /submission) before it sends any data.

This allows for misconfiguration of the device (e.g., http instead of
https) while still enforcing secure transmission of data to the server.

Mitch

On Mon, Feb 17, 2014 at 9:55 AM, Patrick McKay mckayp@gmail.com wrote:

I've set image resolution to a minimum, even taken a completely black
image which was just 10kb but it also fails. I'm on Wi-Fi with a 2mb
connection in the background. I did try adjusting my way time from 20000 to
60000 but there was no change. The same error just take longer. I'm really
confused here. I saw this thread about a similar problem but with mysql:
https://groups.google.com/forum/?nomobile=true#!searchin/opendatakit/images/opendatakit/9iYBEvvJCd8

Does postgre have a similar setting?
On Feb 17, 2014 6:44 PM, "Mitch Sundt" mitchellsundt@gmail.com wrote:

Are you posting from ODK Collect?
Over what type (speed) connection?
And how big (what resolution) are the image files?

Images and other media attachments will be stored under the
directories underneath /sdcard/odk/instances/

Depending upon your image resolution, and your network speed (3G, 4G,
WiFi, Satellite), it may take a very long time to transmit your images to
your ODK Aggregate instance.

The SocketTimeoutException indicates that this is the case.

Tomcat can be configured (
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html ) to
increase how long it waits for data sent to it. If everything is as you
expect, change the defaults until the transmissions succeed.

Broadly speaking, if you are trying to send images, you should avoid
3G and satellite connections due to their reduced bandwidth. They can work,
but are much slower than WiFi or 4G.

Mitch

On Mon, Feb 17, 2014 at 7:09 AM, Patrick McKay mckayp@gmail.comwrote:

It appears to be related to images. Files with images fail, those
without go through. I'm using tomcat with Postrgre. Any ideas where to look?

On Monday, February 17, 2014 4:22:24 PM UTC+2, Patrick McKay wrote:

Like the title says, last week everything was fine, this week I get
the following error when trying to send finalised forms:

Feb 17, 2014 4:17:49 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doHead
INFO: Inside doHead
Feb 17, 2014 4:18:10 PM org.opendatakit.aggregate.servlet.SubmissionServlet
doPost
SEVERE: Unexpected exception: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.apache.coyote.http11.InternalInputBuffer.fill(
InternalInputBuffer.java:756)
at org.apache.coyote.http11.InternalInputBuffer$
InputStreamInputBuffer.doRead(InternalInputBuffer.java:786)
at org.apache.coyote.http11.filters.IdentityInputFilter.
doRead(IdentityInputFilter.java:116)
at org.apache.coyote.http11.InternalInputBuffer.doRead(
InternalInputBuffer.java:715)
at org.apache.coyote.Request.doRead(Request.java:427)
at org.apache.catalina.connector.InputBuffer.realReadBytes(
InputBuffer.java:304)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.
java:419)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.
java:327)
at org.apache.catalina.connector.CoyoteInputStream.read(
CoyoteInputStream.java:193)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.makeAvailable(MultipartStream.java:976)
at org.apache.commons.fileupload.MultipartStream$
ItemInputStream.read(MultipartStream.java:886)
at java.io.FilterInputStream.read(Unknown Source)
at org.apache.commons.fileupload.util.LimitedInputStream.read(
LimitedInputStream.java:125)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.opendatakit.aggregate.parser.MultiPartFormData.<
init>(MultiPartFormData.java:97)
at org.opendatakit.aggregate.servlet.SubmissionServlet.
doPost(SubmissionServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:330)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
at org.springframework.security.web.access.intercept.
FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.
java:84)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.
ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.
java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.
SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
SecurityContextHolderAwareAuthPreservingRequestFilter.doFilter(
SecurityContextHolderAwareAuthPreservingRequestFilter.java:66)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AnonymousAuthenticationFilter.doFilter(
AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.
RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.
AbstractAuthenticationProcessingFilter.doFilter(
AbstractAuthenticationProcessingFilter.java:183)
at org.opendatakit.common.security.spring.
OpenIDAuthenticationFilter.doFilter(OpenIDAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:115)
at org.opendatakit.common.security.spring.
DigestAuthenticationFilter.doFilter(DigestAuthenticationFilter.
java:40)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
OutOfBandUserFilter.doFilter(OutOfBandUserFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.opendatakit.common.security.spring.
Oauth2ResourceFilter.doFilter(Oauth2ResourceFilter.java:351)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.logout.
LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.
SecurityContextPersistenceFilter.doFilter(
SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.channel.
ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144)
at org.springframework.security.web.FilterChainProxy$
VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.
doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(
FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.
invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.
doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.
internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:103)
at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$
Http11ConnectionHandler.process(Http11Protocol.java:606)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(
JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)

I have updated to the latest ODK. Same error for all forms. Server
has been restarted. Any ideas? I'm tearing my hair out!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/XrRz5L1Mor8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com