We found some strange behavior on select multiple questions. Sometimes,
instances will hit the ODK Aggregate server with all choices duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7" will
appear in the data set.
This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
Luke,
Where do the duplicates start? Is it in the form on the device? In the
database on the server? In the export to CSV?
Yaw
On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions. Sometimes,
instances will hit the ODK Aggregate server with all choices duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7" will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
It's on the server. Not sure if its on the device - we have not been able to verify this yet.
Luke
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions. Sometimes,
instances will hit the ODK Aggregate server with all choices duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7" will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the Google Groups "ODK Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.
Yaw
On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been able to verify this yet.
Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions. Sometimes,
instances will hit the ODK Aggregate server with all choices duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7" will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the Google Groups "ODK Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
When the problem occurs, the XML file on the phone does not contain
duplicates but the database on the Aggregate server contains duplicates
values for all multi-selection. We never see more replicates (e.g.
triplicates), and we never see this problem on another question type (e.g.
single select or integers)
Interestingly, the pattern is like so:
a a b b c c d d
Not
a b c d a b c d
The problem occurs very rarely. Perhaps 1 in 5,000 forms.
Luke
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been
able to verify this yet.Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions.
Sometimes,
instances will hit the ODK Aggregate server with all choices
duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7"
will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
What version of Aggregate are you running? And on GAE, MySQL or PostgreSQL?
Yaw
On Mon, Sep 21, 2015 at 3:32 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
When the problem occurs, the XML file on the phone does not contain
duplicates but the database on the Aggregate server contains duplicates
values for all multi-selection. We never see more replicates (e.g.
triplicates), and we never see this problem on another question type (e.g.
single select or integers)Interestingly, the pattern is like so:
a a b b c c d d
Not
a b c d a b c d
The problem occurs very rarely. Perhaps 1 in 5,000 forms.
Luke
On Sat, Sep 19, 2015 at 11:38 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been
able to verify this yet.Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions.
Sometimes,
instances will hit the ODK Aggregate server with all choices
duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7"
will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent it?
Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
We have a bunch of servers all launched on google app engine in 2013-2014,
but let's assume most are "Aggregate v1.4.2 Production."
What version of Aggregate are you running? And on GAE, MySQL or PostgreSQL?
Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Mon, Sep 21, 2015 at 3:32 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
When the problem occurs, the XML file on the phone does not contain
duplicates but the database on the Aggregate server contains duplicates
values for all multi-selection. We never see more replicates (e.g.
triplicates), and we never see this problem on another question type
(e.g.
single select or integers)Interestingly, the pattern is like so:
a a b b c c d d
Not
a b c d a b c d
The problem occurs very rarely. Perhaps 1 in 5,000 forms.
Luke
On Sat, Sep 19, 2015 at 11:38 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been
able to verify this yet.Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In
the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions.
Sometimes,
instances will hit the ODK Aggregate server with all choices
duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7 choice7"
will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent
it?Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in
the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visithttps://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I checked the appengine logs. There are no unusual submissions coming from
the device that submitted the form that was corrupted in the data set.
Here is an log entry:
"2015-06-24 01:07:43.059 /submission?deviceID=imei%3A869845002622XXX 204
66ms 0kb module=default version=1
105.167.222.XXX - - [23/Jun/2015:22:07:43 -0700] "HEAD
/submission?deviceID=imei%3A869845002622XXX HTTP/1.1" 204 158 - - "
pma2020ke.appspot.com" ms=67 cpu_ms=22 cpm_usd=0.000018
app_engine_release=1.9.22 instance=00c61b117XXXXXXXXXXXX
I 2015-06-24 01:07:43.058
org.opendatakit.aggregate.servlet.SubmissionServlet doHead: Inside doHead
We have a bunch of servers all launched on google app engine in 2013-2014,
but let's assume most are "Aggregate v1.4.2 Production."On Mon, Sep 21, 2015 at 3:34 PM, Yaw Anokwa yanokwa@nafundi.com wrote:
What version of Aggregate are you running? And on GAE, MySQL or
PostgreSQL?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Mon, Sep 21, 2015 at 3:32 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
When the problem occurs, the XML file on the phone does not contain
duplicates but the database on the Aggregate server contains duplicates
values for all multi-selection. We never see more replicates (e.g.
triplicates), and we never see this problem on another question type
(e.g.
single select or integers)Interestingly, the pattern is like so:
a a b b c c d d
Not
a b c d a b c d
The problem occurs very rarely. Perhaps 1 in 5,000 forms.
Luke
On Sat, Sep 19, 2015 at 11:38 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been
able to verify this yet.Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In
the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions.
Sometimes,
instances will hit the ODK Aggregate server with all choices
duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7
choice7"
will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent
it?Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in
the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visithttps://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I added additional sanity checks in later versions of ODK Aggregate that
could detect double-inserts like this.
Double-inserts (corrupted submissions) can occur if the quota is reached in
the middle of a form submission.
If you do upgrade to ODK Aggregate 1.45 or later, you should see errors in
the logs for these problematic submissions (you would only see them on a
new publisher or a Briefcase pull to a new, empty, Briefcase storage
location).
The steps to resolve these issues are outlined here:
I checked the appengine logs. There are no unusual submissions coming from
the device that submitted the form that was corrupted in the data set.Here is an log entry:
"2015-06-24 01:07:43.059 /submission?deviceID=imei%3A869845002622XXX 204
66ms 0kb module=default version=1
105.167.222.XXX - - [23/Jun/2015:22:07:43 -0700] "HEAD
/submission?deviceID=imei%3A869845002622XXX HTTP/1.1" 204 158 - - "
pma2020ke.appspot.com" ms=67 cpu_ms=22 cpm_usd=0.000018
app_engine_release=1.9.22 instance=00c61b117XXXXXXXXXXXX
I 2015-06-24 01:07:43.058
org.opendatakit.aggregate.servlet.SubmissionServlet doHead: Inside doHeadOn Mon, Sep 21, 2015 at 3:41 PM, Luke MacDonald < luke.h.macdonald@gmail.com> wrote:
We have a bunch of servers all launched on google app engine in
2013-2014, but let's assume most are "Aggregate v1.4.2 Production."On Mon, Sep 21, 2015 at 3:34 PM, Yaw Anokwa yanokwa@nafundi.com wrote:
What version of Aggregate are you running? And on GAE, MySQL or
PostgreSQL?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Mon, Sep 21, 2015 at 3:32 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
When the problem occurs, the XML file on the phone does not contain
duplicates but the database on the Aggregate server contains duplicates
values for all multi-selection. We never see more replicates (e.g.
triplicates), and we never see this problem on another question type
(e.g.
single select or integers)Interestingly, the pattern is like so:
a a b b c c d d
Not
a b c d a b c d
The problem occurs very rarely. Perhaps 1 in 5,000 forms.
Luke
On Sat, Sep 19, 2015 at 11:38 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Hi Luke,
Until we know where the duplicates are happening, it's hard to track
down. Check every part of the pipeline and let us know when it starts.Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Sat, Sep 19, 2015 at 11:35 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
It's on the server. Not sure if its on the device - we have not been
able to verify this yet.Luke
Sent from my iPhone
On Sep 19, 2015, at 8:09 AM, Yaw Anokwa yanokwa@nafundi.com wrote:
Luke,
Where do the duplicates start? Is it in the form on the device? In
the
database on the server? In the export to CSV?Yaw
Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.On Fri, Sep 18, 2015 at 12:40 PM, Luke MacDonald luke.h.macdonald@gmail.com wrote:
We found some strange behavior on select multiple questions.
Sometimes,
instances will hit the ODK Aggregate server with all choices
duplicated.
Things like "choice1 choice1" and "choice3 choice 3 choice7
choice7"
will
appear in the data set.This happens very rarely, well under 1% of all form submissions.
Has anyone else experienced this issue? Is there a way to prevent
it?Thanks,
Luke
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in
the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visithttps://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/QGba6WqHGA4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com