Incorrect (setvalue) trigger behavior in Enketo when used inside repeats

1. What is the issue? Please be detailed.
XForm setvalue action (aka XLSForm trigger) behaves differently - and incorrectly? - when performed within repeat groups by Enketo than the behavior of same under Collect. Specifically, in Collect, a control within a repeat iteration whose value triggered by another question in the same repeat group, will be correctly updated when the source triggering question in that iteration is modified. However, under Enketo, whenever the source triggering question changes - irregardless of what repeat iteration you are currently in - only the triggered question in the first repeat iteration is updated (!)

2. What steps can we take to reproduce this issue?

Run the following form under Enketo vs Collect:

RepeatWithTrigger.xlsx (14.4 KB)

If you run this under Enketo, whenever you answer or change the (triggering) age question, only the (triggered) age_ts in the first iteration is updated. You can see when you add more iterations and enter another age, the timestamp in the first iteration keeps changing; wheas all subsequent timestamps remain null.

However, if you run the same form under Collect, whenever you answer an age only the age_ts of the same iteration is updated, which is the expected and desired behavior.

I have further verified this by looking at the actual submitted data from each client. After running the form once under both Enketo first and then Collect, and entering the same data for 3 repeat iterations, the resulting dataset submitted to Central is [ignore the last 2 entries which are from an earlier test]:

Shown in yellow is the data from Collect, showing the 3 iterations with each new age with its corresponding age_ts timestamp. Showin in red is the submission data from Enketo, showing only the first repeat iteration has its age_ts timestamp set; for the remaining 2 iterations age_ts remains null.

3. What have you tried to fix the issue?

There appears to be no workaround; setvalue within a repeat group does not behave as expected under Enketo. I suspect the problem may lie with the fact the setvalue action ref for questions within a repeat is not interpreted to be relative to the current iteration under Enketo, whereas this setvalue ref is correctly handled under Collect.

@LN I did look thru https://github.com/enketo/enketo/issues/298, but as best I can tell (and I may be wrong!) this was mostly about misbehavior of specifically the asynchronous odk:setgeopoint functionality in repeats under Enketo, particularly in relation to odk-new-repeat events. Whereas what's described here is more the basic (and synchronous!) xform-value-changed event/setvalue action Enketo-repeat behavior.

Although, that said, could it be possible that the root cause here is the same? That is, in both cases, it is actually because the ref - either for setvalue or odk:setgeopoint - is not being handled correctly under Enketo within repeat groups?

Please note, I observe the same incorrect behavior - under Enketo - when I tried an xform-value-changed event with a setgeopoint action [ala Spec proposal: expose xforms-value-changed event with odk:setgeopoint action in XLSForm, which is how I ended up down this rabbit hole...]. Once again, this works as expected under Collect, but the same under Enketo updates the geopoint of the first iteration only:

As a consequence, neither regular setvalue triggers nor the existing xform-value-changed background GPS capture appear to work as expected under Enketo inside repeat groups. I suspect this is an Enketo bug; if you agree I can open a github issue tracker for it (or 2, if you want a separate one for each action type?). I have confirmed both cases do appear to behave as expected under Collect.

Thanks for digging into it. If you change /data/people/age_ts to ../age_ts, does that work as expected? If so, that would be something to address in pyxform.

I'll give that a try and report back.

Boy, @Lindsay_Stevens_Au is going to love me after this week... :grin:

1 Like

Nope, worse in fact. None of the triggered repeat control were populated, not even the first one anymore! :upside_down_face:

RepeatWithTriggerWorkaround.xml (1.9 KB)

(I also confirmed in the actual submitted data to Central that all the age_ts were indeed null)

FYI I've created new Enketo github tickets to track these issues here:

As noted, these may/not have the same underlying root cause.

1 Like

Does the ../age_ts change work in Collect?

I believe there's no way that /data/people/age_ts would work in Enketo because Enketo will always use the first people instance, as you observed. That is consistent with XPath evaluation rules. Collect has an additional extension that applies repeat instance context to an unqualified reference like /data/people/age_ts but that's outside of standard XPath. For that reason, I do think that pyxform should be generating a relative reference. I'm pretty sure that would work in Collect but it would be good to check.

Beyond that, it looks like Enketo has an issue with that relative reference that needs to be figured out. Maybe it's using the wrong evaluation context?

Nope. In fact it looks like Collect barfs during parsing and wont even load the form (!?)

Here's a form that does work correctly (under Collect, but not Enketo...) with the full unqualified path:

RepeatWithTrigger.xml (1.2 KB)

<setvalue ref="/data/people/age_ts" event="xforms-value-changed" value="now()"/>

Here's the slightly modified form of above where the trigger action ref in the repeat loop is changed to a relative path:

RepeatWithTriggerFix2.xml (1.2 KB)

<setvalue ref="../age_ts" event="xforms-value-changed" value="now()"/>

Enketo will load this OK, but as described above, it doesnt work properly and only updates the first iteration. Whereas Collect wont even load this version of the form:

Ah, ok. Sorry, now that I think about it more deeply, ref in the body is always an absolute reference and I don't think relative references are expected to be supported.

That would be true if this reference were used in the value attribute expression. I now think it's the correct path for ref the same way that the ref for an input named q1 inside of /data/people would have ref /data/people/q1.

So I think you're right that it's just an Enketo issue.

And gives a very unhelpful error message. That's something we will be fixing soon.

So basically this is an Enketo 'bug' (or new feature, depending on your taste...) for which there is no present workaround via manually fiddling with the XForm (aka tweaking pyxform translation). Correct?

1 Like

Collect has an additional extension that applies repeat instance context to an unqualified reference like /data/people/age_ts but that's outside of standard XPath.

So to address this Enketo issue in a manner that will be consistent with (working) current Collect behavior - as well as current pyxform XForm 'trigger' output - it would seem we'd need to replicate whatever functionality this custom (javarosa?) XPath extension is performing into Enketo, right?

Curious, what is the new GetODK Web Forms' xforms-engine doing about this (or haven't you got there yet)?

No, I'm sorry, I really confused things! The goal is for pyxform to always generate relative references within repeats in places where relative references are allowed/expected.

I briefly thought that that absolute reference might have been part of the issue but it's not. The ref for actions is the same as the ref for body controls and is expected to always be an absolute path. I believe Enketo should be contextualizing the action's ref the same way it does for body controls but it looks like there's an issue with how that happens.

ODK Web Forms currently does not support the out of spec behavior and it's likely it won't ever. Pyxform has been generating relative references for some time and we don't expect many forms coming from other sources. It's something we could add in if it turns out to be needed, though.