NaN value for integers and decimals

Aggregate submissions fail (with a 500 Unexpected Exception) when a
submission for a calculated item multiplying 2 integers has the value NaN.
See this
thread: https://groups.google.com/forum/#!topic/opendatakit/UBIyAJNL8Z4

NaN is a valid and useful value for Number types in XPath 1.0, so I wonder
if this should be accepted for any integer or decimal datatype.

What do you think?

I realize now that this issue is probably related to the database, and in
order to support a variety of databases without manipulating data, it would
makes sense to set NaN to empty in the client (like I presume ODK Collect
does). Unless others object, I'll do this in Enketo. (track:
https://github.com/MartijnR/enketo-core/issues/40)

··· On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote: > > Aggregate submissions fail (with a 500 Unexpected Exception) when a > submission for a calculated item multiplying 2 integers has the value NaN. > See this thread: > https://groups.google.com/forum/#!topic/opendatakit/UBIyAJNL8Z4 > > NaN is a valid and useful value for Number types in XPath 1.0, so I wonder > if this should be accepted for any integer or decimal datatype. > > What do you think? > >

sorry forgot attachment

DecimalInfinity.xml (1.34 KB)

··· On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote: > > Aggregate submissions fail (with a 500 Unexpected Exception) when a > submission for a calculated item multiplying 2 integers has the value NaN. > See this thread: > https://groups.google.com/forum/#!topic/opendatakit/UBIyAJNL8Z4 > > NaN is a valid and useful value for Number types in XPath 1.0, so I wonder > if this should be accepted for any integer or decimal datatype. > > What do you think? > >

On the Aggregate side, it is an issue with Java BigDecimal not accepting
"NaN" or "+/-Infinity"

I'm rather surprised that BigDecimal doesn't natively handle NaN or
Infinity.

BigDecimal is used for decimal values within ODK Aggregate.
Long is used for integer values within ODK Aggregate.

PostgreSQL: decimal values stored as DECIMAL(,) ; integer values are
stored as BIGINT() or INTEGER()
MySQL: decimal values stored as DECIMAL(,) ; integer values stored as
BIGINT() or INTEGER()
GAE: decimal values stored as numeric fields (Doubles); integer values
stored as long fields.

MySQL and PostgreSQL DBAs can tweak the precisions of what are stored in
the DB, and, after a reboot, ODK Aggregate will honor those changes.

··· On Mon, Jul 21, 2014 at 11:15 AM, Martijn van de Rijdt wrote:

I realize now that this issue is probably related to the database, and in
order to support a variety of databases without manipulating data, it would
makes sense to set NaN to empty in the client (like I presume ODK Collect
does). Unless others object, I'll do this in Enketo. (track:
https://github.com/MartijnR/enketo-core/issues/40)

On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote:

Aggregate submissions fail (with a 500 Unexpected Exception) when a
submission for a calculated item multiplying 2 integers has the value NaN.
See this thread: https://groups.google.com/forum/#!topic/opendatakit/
UBIyAJNL8Z4

NaN is a valid and useful value for Number types in XPath 1.0, so I
wonder if this should be accepted for any integer or decimal datatype.

What do you think?

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

This is a bug and/or non-conformance of ODK Aggregate.

··· On Tue, Jul 22, 2014 at 10:33 AM, Martijn van de Rijdt wrote:

sorry forgot attachment

On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote:

Aggregate submissions fail (with a 500 Unexpected Exception) when a
submission for a calculated item multiplying 2 integers has the value NaN.
See this thread: https://groups.google.com/forum/#!topic/opendatakit/
UBIyAJNL8Z4

NaN is a valid and useful value for Number types in XPath 1.0, so I
wonder if this should be accepted for any integer or decimal datatype.

What do you think?

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Interesting. Thanks.

So ODK Collect also quietly sets Infinity and -Infinity to empty?

··· On Monday, July 21, 2014 1:25:38 PM UTC-6, Mitch wrote: > > On the Aggregate side, it is an issue with Java BigDecimal not accepting > "NaN" or "+/-Infinity" > > > http://stackoverflow.com/questions/10080084/how-to-convert-double-positive-infinity-to-bigdecimal > > I'm rather surprised that BigDecimal doesn't natively handle NaN or > Infinity. > > BigDecimal is used for decimal values within ODK Aggregate. > Long is used for integer values within ODK Aggregate. > > PostgreSQL: decimal values stored as DECIMAL(,) ; integer values are > stored as BIGINT() or INTEGER() > MySQL: decimal values stored as DECIMAL(,) ; integer values stored as > BIGINT() or INTEGER() > GAE: decimal values stored as numeric fields (Doubles); integer values > stored as long fields. > > MySQL and PostgreSQL DBAs can tweak the precisions of what are stored in > the DB, and, after a reboot, ODK Aggregate will honor those changes. > > > > On Mon, Jul 21, 2014 at 11:15 AM, Martijn van de Rijdt <mar...@enketo.org > wrote: > >> I realize now that this issue is probably related to the database, and in >> order to support a variety of databases without manipulating data, it would >> makes sense to set NaN to empty in the client (like I presume ODK Collect >> does). Unless others object, I'll do this in Enketo. (track: >> https://github.com/MartijnR/enketo-core/issues/40) >> >> On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote: >>> >>> Aggregate submissions fail (with a 500 Unexpected Exception) when a >>> submission for a calculated item multiplying 2 integers has the value NaN. >>> See this thread: https://groups.google.com/forum/#!topic/opendatakit/ >>> UBIyAJNL8Z4 >>> >>> NaN is a valid and useful value for Number types in XPath 1.0, so I >>> wonder if this should be accepted for any integer or decimal datatype. >>> >>> What do you think? >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "ODK Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit-developers+unsubscribe@googlegroups.com >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >

Thanks. I posted the issue for Infinity and -Infinity here:
https://code.google.com/p/opendatakit/issues/detail?id=1035

If not accepting NaN is also considered a bug (in addition to Infinity and
-Infinity), I'll reverse a workaround I made in Enketo for NaN in decimal
fields.

··· On Wed, Jul 23, 2014 at 5:38 PM, Mitch Sundt wrote:

This is a bug and/or non-conformance of ODK Aggregate.

On Tue, Jul 22, 2014 at 10:33 AM, Martijn van de Rijdt <martijn@enketo.org wrote:

sorry forgot attachment

On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote:

Aggregate submissions fail (with a 500 Unexpected Exception) when a
submission for a calculated item multiplying 2 integers has the value NaN.
See this thread: https://groups.google.com/forum/#!topic/opendatakit/
UBIyAJNL8Z4

NaN is a valid and useful value for Number types in XPath 1.0, so I
wonder if this should be accepted for any integer or decimal datatype.

What do you think?

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit-developers/fqJhZqPEI5Q/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Did you know that Enketo Smart Paper has now become the #1 tool for data
collection? Don't fall behind. Use it!

Enketo https://enketo.org | LinkedIn
http://www.linkedin.com/company/enketo-llc | GitHub
https://github.com/MartijnR | Twitter https://twitter.com/enketo

Just tested. Infinite and -Infinite values for a node with the decimal type
cause the 500 error when submitting to Aggregate from ODK Collect. Not sure
where to file this bug (Collect, Aggregate or JavaRosa). See attached test
form.

Would be curious how you'd want to resolve this. I'm a bit reluctant to
overwrite NaN, Infinity and -Infinity completely in the XPath library. I'm
afraid it will have unintended consequences.

To reproduce:

  1. upload form to latest Aggregate
  2. load the form in ODK Collect
  3. submit

Expected: Successful submission

Seen in ODK Collect: Error: Internal Server Error (500)

Cheers,
Martijn

··· On Tuesday, July 22, 2014 11:02:30 AM UTC-6, Martijn van de Rijdt wrote: > > Interesting. Thanks. > > So ODK Collect also quietly sets Infinity and -Infinity to empty? > > On Monday, July 21, 2014 1:25:38 PM UTC-6, Mitch wrote: >> >> On the Aggregate side, it is an issue with Java BigDecimal not accepting >> "NaN" or "+/-Infinity" >> >> >> http://stackoverflow.com/questions/10080084/how-to-convert-double-positive-infinity-to-bigdecimal >> >> I'm rather surprised that BigDecimal doesn't natively handle NaN or >> Infinity. >> >> BigDecimal is used for decimal values within ODK Aggregate. >> Long is used for integer values within ODK Aggregate. >> >> PostgreSQL: decimal values stored as DECIMAL(,) ; integer values are >> stored as BIGINT() or INTEGER() >> MySQL: decimal values stored as DECIMAL(,) ; integer values stored as >> BIGINT() or INTEGER() >> GAE: decimal values stored as numeric fields (Doubles); integer values >> stored as long fields. >> >> MySQL and PostgreSQL DBAs can tweak the precisions of what are stored in >> the DB, and, after a reboot, ODK Aggregate will honor those changes. >> >> >> >> On Mon, Jul 21, 2014 at 11:15 AM, Martijn van de Rijdt > >>> I realize now that this issue is probably related to the database, and >>> in order to support a variety of databases without manipulating data, it >>> would makes sense to set NaN to empty in the client (like I presume ODK >>> Collect does). Unless others object, I'll do this in Enketo. (track: >>> https://github.com/MartijnR/enketo-core/issues/40) >>> >>> On Monday, July 21, 2014 11:29:41 AM UTC-6, Martijn van de Rijdt wrote: >>>> >>>> Aggregate submissions fail (with a 500 Unexpected Exception) when a >>>> submission for a calculated item multiplying 2 integers has the value NaN. >>>> See this thread: https://groups.google.com/forum/#!topic/opendatakit/ >>>> UBIyAJNL8Z4 >>>> >>>> NaN is a valid and useful value for Number types in XPath 1.0, so I >>>> wonder if this should be accepted for any integer or decimal datatype. >>>> >>>> What do you think? >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "ODK Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to opendatakit-developers+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Mitch Sundt >> Software Engineer >> University of Washington >> mitche...@gmail.com >> >