Size limit on xlsform input

Hi,

We're creating a form that due to circumstances beyond our control contains
some ridiculous logic... some of the statements in the 'relevant' column
are getting large; so large, we suspect that the parser is breaking. One
of them looks like this:

((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and
((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or
(selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or
(selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and
((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or
(selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5')))))

We've been over this extensively and we're convinced it should work but the
parser breaks and we guess it's because we exceed some limitation. Is it
possible to build a custom version of xlsform that has a much bigger input
buffer? Or is there a better way to do this?

Thanks!

-Chris Estes-

Chris,

What do you mean the parser is breaking? What exact message are you
getting? And from where? xlsform.opendatakit.org?

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Tue, Apr 22, 2014 at 12:26 PM, Chris Estes cestes@clsamerica.com wrote:

Hi,

We're creating a form that due to circumstances beyond our control contains
some ridiculous logic... some of the statements in the 'relevant' column are
getting large; so large, we suspect that the parser is breaking. One of
them looks like this:

((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and
((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or
(selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or
(selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and
((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or
(selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5')))))

We've been over this extensively and we're convinced it should work but the
parser breaks and we guess it's because we exceed some limitation. Is it
possible to build a custom version of xlsform that has a much bigger input
buffer? Or is there a better way to do this?

Thanks!

-Chris Estes-

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

We're using http://opendatakit.org/use/xlsform/ to convert the spreadsheet
to a form. The error message looks like this:

Error: ODK Validate Errors:
Invalid XPath expression [((selected( ${Q03a} , 'OP') and not(selected(
${Q03a} , 'HX')) and ((selected( ${Q03a} , 'U1')) or (selected( ${Q03a} ,
'U2')) or (selected( ${Q03a} , 'U4')) or (selected( ${Q03a} , 'U5')))) or
(selected( ${Q03b} , 'OP') and not(selected( ${Q03b} , 'HX')) and
((selected]!

Something broke the parser. See above for a hint.

Result: Invalid

Thanks!

··· On Tuesday, April 22, 2014 3:26:57 PM UTC-4, Chris Estes wrote: > > Hi, > > We're creating a form that due to circumstances beyond our control > contains some ridiculous logic... some of the statements in the 'relevant' > column are getting large; so large, we suspect that the parser is breaking. > One of them looks like this: > > ((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and > ((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or > (selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or > (selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and > ((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or > (selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5'))))) > > We've been over this extensively and we're convinced it should work but > the parser breaks and we guess it's because we exceed some limitation. Is > it possible to build a custom version of xlsform that has a much bigger > input buffer? Or is there a better way to do this? > > Thanks! > > -Chris Estes- >

Hi Chris,

The XPath expression that is referenced is broken in some way. You are probably
missing or have an extra parenthesis.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Wed, Apr 23, 2014 at 8:29 AM, Chris Estes cestes@clsamerica.com wrote:

We're using http://opendatakit.org/use/xlsform/ to convert the spreadsheet
to a form. The error message looks like this:

Error: ODK Validate Errors:
Invalid XPath expression [((selected( ${Q03a} , 'OP') and not(selected(
${Q03a} , 'HX')) and ((selected( ${Q03a} , 'U1')) or (selected( ${Q03a} ,
'U2')) or (selected( ${Q03a} , 'U4')) or (selected( ${Q03a} , 'U5')))) or
(selected( ${Q03b} , 'OP') and not(selected( ${Q03b} , 'HX')) and
((selected]!

Something broke the parser. See above for a hint.

Result: Invalid

Thanks!

On Tuesday, April 22, 2014 3:26:57 PM UTC-4, Chris Estes wrote:

Hi,

We're creating a form that due to circumstances beyond our control
contains some ridiculous logic... some of the statements in the 'relevant'
column are getting large; so large, we suspect that the parser is breaking.
One of them looks like this:

((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and
((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or
(selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or
(selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and
((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or
(selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5')))))

We've been over this extensively and we're convinced it should work but
the parser breaks and we guess it's because we exceed some limitation. Is
it possible to build a custom version of xlsform that has a much bigger
input buffer? Or is there a better way to do this?

Thanks!

-Chris Estes-

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I agree with Yaw -- I'm pretty sure the parentheses are mis-matched.
XLSForm runs on a server, and can easily handle forms of over 1MB in size.

Using a programmer's text editor that highlights matching parentheses can
help.

BUT - You may still run into issues on the device with these really long
expressions.

I would recommend splitting the expression into several parts that have
functional meaning to you.

e.g., if it is about medications and symptoms, perhaps have:

  • a calculate field that is true if a patient is taking a penicillin
    derivative
  • a calculate field that is true if a patient is taking a sulfa drug
  • a calculate field if the patient is reporting head/nasal/lung symptoms
  • a calculate field if the patient is reporting head or neck aches
    etc.

Then combine these more basic predicates into more complex logic like e.g,.

if the patient reports headaches without nasal/lung symptoms and is taking
penicillin.

Mitch

··· On Wed, Apr 23, 2014 at 9:25 AM, Yaw Anokwa wrote:

Hi Chris,

The XPath expression that is referenced is broken in some way. You are
probably
missing or have an extra parenthesis.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Wed, Apr 23, 2014 at 8:29 AM, Chris Estes cestes@clsamerica.com wrote:

We're using http://opendatakit.org/use/xlsform/ to convert the
spreadsheet
to a form. The error message looks like this:

Error: ODK Validate Errors:
Invalid XPath expression [((selected( ${Q03a} , 'OP') and not(selected(
${Q03a} , 'HX')) and ((selected( ${Q03a} , 'U1')) or (selected( ${Q03a} ,
'U2')) or (selected( ${Q03a} , 'U4')) or (selected( ${Q03a} , 'U5')))) or
(selected( ${Q03b} , 'OP') and not(selected( ${Q03b} , 'HX')) and
((selected]!

Something broke the parser. See above for a hint.

Result: Invalid

Thanks!

On Tuesday, April 22, 2014 3:26:57 PM UTC-4, Chris Estes wrote:

Hi,

We're creating a form that due to circumstances beyond our control
contains some ridiculous logic... some of the statements in the
'relevant'
column are getting large; so large, we suspect that the parser is
breaking.
One of them looks like this:

((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and
((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or
(selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or
(selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and
((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or
(selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5')))))

We've been over this extensively and we're convinced it should work but
the parser breaks and we guess it's because we exceed some limitation.
Is
it possible to build a custom version of xlsform that has a much bigger
input buffer? Or is there a better way to do this?

Thanks!

-Chris Estes-

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Thanks for the input.

It turns out the parser does have a size limitation -- it breaks at around
234 bytes... we tested it with much simpler statement without all the
parens just to see when it chokes.

Agreed about breaking it down into more questions... it make this part
easier, but we do a lot of downstream processing of the output which will
have some increased complexity as a result of decomposing the structure of
the form.

Thanks again!

··· On Wednesday, April 23, 2014 12:53:35 PM UTC-4, Mitch Sundt wrote: > > I agree with Yaw -- I'm pretty sure the parentheses are mis-matched. > XLSForm runs on a server, and can easily handle forms of over 1MB in size. > > Using a programmer's text editor that highlights matching parentheses can > help. > > BUT - You may still run into issues on the device with these really long > expressions. > > I would recommend splitting the expression into several parts that have > functional meaning to you. > > e.g., if it is about medications and symptoms, perhaps have: > - a calculate field that is true if a patient is taking a penicillin > derivative > - a calculate field that is true if a patient is taking a sulfa drug > - a calculate field if the patient is reporting head/nasal/lung symptoms > - a calculate field if the patient is reporting head or neck aches > etc. > > Then combine these more basic predicates into more complex logic like e.g,. > > if the patient reports headaches without nasal/lung symptoms and is taking > penicillin. > > Mitch > > > > On Wed, Apr 23, 2014 at 9:25 AM, Yaw Anokwa <yan...@nafundi.com wrote: > >> Hi Chris, >> >> The XPath expression that is referenced is broken in some way. You are >> probably >> missing or have an extra parenthesis. >> >> Yaw >> -- >> Need ODK services? http://nafundi.com provides form design, server >> setup, professional support, and software development for ODK. >> >> On Wed, Apr 23, 2014 at 8:29 AM, Chris Estes <ces...@clsamerica.com> wrote: >> > We're using http://opendatakit.org/use/xlsform/ to convert the >> spreadsheet >> > to a form. The error message looks like this: >> > >> > >> > Error: ODK Validate Errors: >> > Invalid XPath expression [((selected( ${Q03a} , 'OP') and not(selected( >> > ${Q03a} , 'HX')) and ((selected( ${Q03a} , 'U1')) or (selected( ${Q03a} >> , >> > 'U2')) or (selected( ${Q03a} , 'U4')) or (selected( ${Q03a} , 'U5')))) >> or >> > (selected( ${Q03b} , 'OP') and not(selected( ${Q03b} , 'HX')) and >> > ((selected]! >> > >> >>> Something broke the parser. See above for a hint. >> > >> > Result: Invalid >> > >> > >> > Thanks! >> > >> > On Tuesday, April 22, 2014 3:26:57 PM UTC-4, Chris Estes wrote: >> >> >> >> Hi, >> >> >> >> We're creating a form that due to circumstances beyond our control >> >> contains some ridiculous logic... some of the statements in the >> 'relevant' >> >> column are getting large; so large, we suspect that the parser is >> breaking. >> >> One of them looks like this: >> >> >> >> ((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and >> >> ((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or >> >> (selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or >> >> (selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and >> >> ((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or >> >> (selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5'))))) >> >> >> >> We've been over this extensively and we're convinced it should work but >> >> the parser breaks and we guess it's because we exceed some limitation. >> Is >> >> it possible to build a custom version of xlsform that has a much bigger >> >> input buffer? Or is there a better way to do this? >> >> >> >> Thanks! >> >> >> >> -Chris Estes- >> > >> > -- >> > -- >> > Post: opend...@googlegroups.com >> > Unsubscribe: opendatakit...@googlegroups.com >> > Options: http://groups.google.com/group/opendatakit?hl=en >> > >> > --- >> > You received this message because you are subscribed to the Google >> Groups >> > "ODK Community" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to opendatakit...@googlegroups.com . >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ODK Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >

Very odd. I've forwarded this to Matt Berg for investigation.

··· On Wed, Apr 23, 2014 at 1:54 PM, Chris Estes wrote:

Thanks for the input.

It turns out the parser does have a size limitation -- it breaks at around
234 bytes... we tested it with much simpler statement without all the
parens just to see when it chokes.

Agreed about breaking it down into more questions... it make this part
easier, but we do a lot of downstream processing of the output which will
have some increased complexity as a result of decomposing the structure of
the form.

Thanks again!

On Wednesday, April 23, 2014 12:53:35 PM UTC-4, Mitch Sundt wrote:

I agree with Yaw -- I'm pretty sure the parentheses are mis-matched.
XLSForm runs on a server, and can easily handle forms of over 1MB in size.

Using a programmer's text editor that highlights matching parentheses can
help.

BUT - You may still run into issues on the device with these really long
expressions.

I would recommend splitting the expression into several parts that have
functional meaning to you.

e.g., if it is about medications and symptoms, perhaps have:

  • a calculate field that is true if a patient is taking a penicillin
    derivative
  • a calculate field that is true if a patient is taking a sulfa drug
  • a calculate field if the patient is reporting head/nasal/lung symptoms
  • a calculate field if the patient is reporting head or neck aches
    etc.

Then combine these more basic predicates into more complex logic like
e.g,.

if the patient reports headaches without nasal/lung symptoms and is
taking penicillin.

Mitch

On Wed, Apr 23, 2014 at 9:25 AM, Yaw Anokwa yan...@nafundi.com wrote:

Hi Chris,

The XPath expression that is referenced is broken in some way. You are
probably
missing or have an extra parenthesis.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Wed, Apr 23, 2014 at 8:29 AM, Chris Estes ces...@clsamerica.com wrote:

We're using http://opendatakit.org/use/xlsform/ to convert the
spreadsheet
to a form. The error message looks like this:

Error: ODK Validate Errors:
Invalid XPath expression [((selected( ${Q03a} , 'OP') and not(selected(
${Q03a} , 'HX')) and ((selected( ${Q03a} , 'U1')) or (selected(
${Q03a} ,
'U2')) or (selected( ${Q03a} , 'U4')) or (selected( ${Q03a} , 'U5'))))
or
(selected( ${Q03b} , 'OP') and not(selected( ${Q03b} , 'HX')) and
((selected]!

Something broke the parser. See above for a hint.

Result: Invalid

Thanks!

On Tuesday, April 22, 2014 3:26:57 PM UTC-4, Chris Estes wrote:

Hi,

We're creating a form that due to circumstances beyond our control
contains some ridiculous logic... some of the statements in the
'relevant'
column are getting large; so large, we suspect that the parser is
breaking.
One of them looks like this:

((selected(${Q03a}, 'OP') and not(selected(${Q03a}, 'HX')) and
((selected(${Q03a}, 'U1')) or (selected(${Q03a}, 'U2')) or
(selected(${Q03a}, 'U4')) or (selected(${Q03a}, 'U5')))) or
(selected(${Q03b}, 'OP') and not(selected(${Q03b}, 'HX')) and
((selected(${Q03b}, 'U1')) or (selected(${Q03b}, 'U2')) or
(selected(${Q03b}, 'U4')) or (selected(${Q03b}, 'U5')))))

We've been over this extensively and we're convinced it should work
but
the parser breaks and we guess it's because we exceed some
limitation. Is
it possible to build a custom version of xlsform that has a much
bigger
input buffer? Or is there a better way to do this?

Thanks!

-Chris Estes-

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com