Combining lookups on responses to single & multiple selects to determine global and filtered join/max

Support
, , ,
1

Part III - Putting it all together!
Previously: I - Dynamic repeat labels, II - Filtering repeats into a repeat

tl;dr - A calculation as join(' ', instance1, instance2, instance3etc) or max(instance1, instance2, instance3etc) works in Enketo/Webforms but errors in Collect. But join(' ', join(' ', instance1), join(' ', instance2), join(' ', instance3etc)) or max(max(instance1), max(instance2), max(instance3etc)) works everywhere.

1. What is the issue? Please be detailed.
Part II resulted in logic to look at a repeat, filter the responses to those matching a group, join a string, count up how many there were / find the maximum value of a single question within that subset.

Now, I would like to be able to do the above for

  • more than one question in a repeat instance, and
  • both single and multiple selects

I had a little trouble initially, as operators like max() work on nodesets, and while you can write an instance lookup that results in a nodeset, the value returned from it is a string so you can't generate a space separated string in one calculation and then use that inside the operator, it has to be one operation.

I managed to get a form working in Enketo and also Webforms and it appears to work correctly across all combinations of single/multiple select with multiple repeat instances, but will error in Collect (This field is repeated, you may need to use indexed-repeat). After a bunch of different calculation logic trials I managed to get it working in Collect as well, but am not sure why the calc logic breaks Collect but is a-ok in Enketo & Webforms but I bet @JenniferQ knows!

  • This is ok everywhere, but a bit messy (field rep_max)
    • max(max(instance('options')/root/item[name = ${rep_singleselect}]/severity) , max(instance('options')/root/item[selected(${rep_multipleselect},name)]/severity), max(instance('options')/root/item[name = ${rep_singleselect2}]/severity) , max(instance('options')/root/item[selected(${rep_multipleselect2},name)]/severity))
    • join(' ', join(' ', instance('options')/root/item[name = ${rep_singleselect}]/severity) , join(' ', instance('options')/root/item[selected(${rep_multipleselect},name)]/severity), join(' ', instance('options')/root/item[name = ${rep_singleselect2}]/severity) , join(' ', instance('options')/root/item[selected(${rep_multipleselect2},name)]/severity))
  • This is ok in Enketo/Webforms but breaks Collect, and is a little cleaner (field local_join)
    • max(instance('options')/root/item[name = ${rep_singleselect}]/severity , instance('options')/root/item[selected(${rep_multipleselect},name)]/severity, instance('options')/root/item[name = ${rep_singleselect2}]/severity , instance('options')/root/item[selected(${rep_multipleselect2},name)]/severity)
    • join(' ' , instance('options')/root/item[name = ${rep_singleselect}]/severity , instance('options')/root/item[selected(${rep_multipleselect},name)]/severity, instance('options')/root/item[name = ${rep_singleselect2}]/severity , instance('options')/root/item[selected(${rep_multipleselect2},name)]/severity)

2. What steps can we take to reproduce this issue?
See the attached form, swap out the max & join calculations as above (note the attached form has a different max calc now that is tolerant of blank questions so it's slightly different)

  • Both versions of the join and max functions seem to work in Enketo and Webforms with no problems, but with the second set Collect won't evaulate if a multiple select has >1 chosen (notes showing value stop updating and error message appears when navigating away from repeat element)
    • For Collect, the calcs inside the finding repeat only use the first selected value from the select-multiples, and >1 will throw an error
    • For Collect, the calcs outside the finding repeat work if there is only one repeat element and only single selections. Adding a second repeat then throws another field is repeated error

3. What have you tried to fix the issue?
As above, lots of different trials of writing varying calc logic

4. Upload any forms or screenshots you can share publicly below.

This form should work in Enketo/Webforms/Collect (the max calculations don't work if any question is unanswered in Enketo/Webforms fixed, Collect tolerates this) and has 3x types of calculation after the finding repeat;

  1. grp_global_max: Join of every single response & max of these
  2. grp_filtered_global_max: Join of all responses that match selected category & max of these
  3. rep_summary: Dynamic repeat with an element for each category that has join of all matching responses and max of these

find max severity from select multiple.xlsx (581.4 KB)

2

Thanks for the detailed breakdown and the attached form! You're right; there is a difference in how XPath engines operate between Enketo/Webforms (which are JavaScript-based) and ODK Collect (powered by JavaRosa).

Enketo/Webforms uses the browser's Math.max() function (see MDN docs), which returns NaN if any argument is NaN, such as from blanks. In contrast, Collect's JavaRosa ignores NaN and calculates only from valid values (see JavaRosa code).

When using Enketo/Webforms, the form must address invalid numbers by either filtering them out or replacing them with a default value that fits the form's logic (like zero).

3

Ah yeah, the NaN was a separate issue i fixed by putting if statements around the max operator.

What i really don't understand is why Collect doesn't allow max(instance, instance, instance.,...) but does allow max(max(instance), max(instance), max(instance)...), and both are fine in enketo/WF. And same for join etc.

4

I had a quick chat with the team; it seems to be a bug in Collect. The behaviors you’re observing in Web Forms are correct. I’ve opened issues in GitHub to fix these:

5

Thanks! I found it errored on max and join, so I assume it affects other nodeset operators like count/sum...

I wonder how many forms there are in use that will NaN after this fix that previously took advantage of Collect's behaviour!