XLSForm proposal to calculate a value when another value changes

Development
,
1

We have already agreed upon and specified a mechanism to calculate a value when another value changes. Both Enketo and ODK Collect support this feature already. See the xforms-value-changed event in the ODK XForms spec for technical spec details if you're interested in such things.

Now, I think it is time to agree upon a syntax in XLSForm, after which it can be implemented in pyxform and then actually be used. I'd like to get this discussion started with a proposal to introduce a new trigger column that works in conjunction with type=calculate (only).

type name label calculation trigger
text a Enter text
calculate b now() ${a}

Strengths of trigger:

  • it is one word
  • it is flexible enough to in the future expand with other (2) events in our XForms specification that do not refer to another node, e.g. the column could in the future also allow the values first-load or new-repeat or something similar (with other question types).
  • it would work well with another action (subject of a future proposal) to record the location (which could be another question type)

Weaknesses of trigger:

  • the future flexibility means the column name is not as descriptive as something like upon_change or when_changed would be
  • we already use the word "trigger" as an alias for acknowledge (though - thankfully - this is not documented in xlsform.org)

I looking forward to hearing what you think!

3 Likes
2

Given trigger is already a well-defined XForm element term, I'd be reluctant to re-use it in a quasi-similar (wrt both being 'event' related') but quite different context, even though the former, original, XForm usage isnt readily exposed via XLSForm today...

Although I do agree trigger is certainly a good fit here otherwise! But perhaps instead (but less descriptive, alas) a synonym like activate, actuate, initiate, instigate.... (incite? :slight_smile: )

2 Likes
3

Thanks for this proposal, @martijnr! It would indeed be great to add to XLSForm.

What are you imagining that the resulting XML would look like? Would the calculate become a field with a string bind, no calculate and no corresponding body control element? It seems that it can't have a calculation or that would override the result of any setvalue action, right?

That was also my gut reaction and it's still making me uneasy.

It's not really the case that a field is a trigger for a setvalue, the trigger is the change in that field. If you're comfortable with a field being in the proposed trigger column, how about using the label when? I think that works very nicely for other events

type name label calculation when
text a Enter text
calculate b now() ${a}

I think the two major use cases for the value changed event will be to pull some value from outside the form as you've demonstrated (time, location) and setting dynamic defaults based on values supplied by the user (typically pulled from external datasets).

Do you agree that the latter case is worth supporting? In that case, trigger would need to apply to types other than calculate.

When I started thinking about this, I was imagining it flipped so that the expression would be defined in the row of the value that changes. For example, to set the value of b when a changes, I was thinking of something like:

type name label on_change
text a Enter text ${b}=instance('foo')/root/item[name=${a}]
select_one foo b Select one

(Note that this would probably be more realistic with pulldata given current documentation)

The downsides of this is that it introduces some new syntax with the = and that it's not as clear what the hidden value case like your example should look like. One concept would be to introduce a column for suppressing a body element:

type name label on_change user_facing
text a Enter text ${b}=now()
dateTime b no

This would also address the request for typed calculate fields that comes up periodically. I think that pyxform could validate that a field without a body element is either the target of an action or has a calculation but not both. But this adds to the cross product of possible question properties so certainly adds complexity.

One of the things you mention thinking ahead to is how to surface the setgeopoint action triggered by events other than odk-instance-first-load which I agree is important. Is this what you'd have in mind?

type name label trigger
text a Enter text
auto_geopoint b ${a}

In what I described as an alternative, this might be done by introducing a special keyword that could be used in the default or on_change columns. That wouldn't feel so clean.

I'm still thinking through all this and am not advocating for one solution over the other yet. I hope that sharing my thinking may spark some ideas.

1 Like
4

WIP: discussion with @ln and @Xiphware:

type name label calculation when
text a Enter text
dateTime b now() ${a}
dateTime c Date of diagnostic now() ${a}
dateTime d Date of something ${a} empties value!
dateTime e ${a} empties value!
Change rules:
  • If there is no calculation a label is required unless it has when.
  • If it has a label, a body element is included in the XForms output.
  • An empty calculation with when, will reset the value to empty.
5

It looks like pyxform currently allows the absence of a label if the question has a hint, so I want to propose a tiny tweak to our plan:

  • If there is no calculation a label or a hint is required unless it has a value for 'when'.
  • If it has a label or a hint, a body element is included in the XForms output.

This would avoid issues with editing existing forms.

Does that seem alright?

1 Like
6

Hmm... I'm not sure this behavior is strictly XForm spec conformant. Specifically, from label:

8.2.1 The label Element
This element provides a descriptive label for the containing form control. ...

whereas, for hint (emphasis added):

8.2.3 The hint Element
The author-optional element hint provides a convenient way to attach hint information to a form control. ...

Although not explicit, my interpretation might be that label is not optional, whereas hint is (do you agree?)

Not that this probably directly affects the issue at hand, other than perhaps we might be perpetuating a pre-existing pyxform non-conformance... :wink:

2 Likes
7

Good point @Xiphware. The W3C site seems inaccessible atm, but I think (maybe wishfully) that the <label> element is indeed required, but that it may be empty.

Pyxform doesn't actually output an empty <label/> though.

1 Like
8

Thanks for a good discussion, @martijnr and @Xiphware. I'm on board for the extensions described at XLSForm proposal to calculate a value when another value changes - #4 by martijnr with the small tweaks to maintain current hint/label rules.

I think we can move forward with that and separately take on the question of whether we should consider changing the label and hint requirement rules or output. I agree with @Xiphware's reading of the W3C spec that a label element should always exist. It does seem like an empty element should be ok so that might be an easy way to make pyxform compliant without affecting edits to existing form definitions.

@martijnr do you want to pull all this into a pyxform issue that we can do one last sanity check on?

9

Yes, I will do that shortly, and I've heard somebody is going to prepare a PR for this.

1 Like
10

I agree. I could forsee situations where you might want a widget without a (text) label

11

I drafted something here: https://github.com/XLSForm/pyxform/issues/438 with two related issues that we discussed in this thread linked at the top.

1 Like
12

Thanks, @martijnr! As I wrote on the issue, this seems ready to build to me.

When we had a call about this, I said I'd confirm a couple of things about the JavaRosa/Collect implementation:

  • There can be multiple setvalue actions nested in a form control (test)
  • A setvalue action can set the value of a readonly field (test)

Not exactly related to this spec but brought up by the second point, we also wanted to confirm that calculates on readonly fields with form controls were executed. They are indeed so I believe this behavior is consistent across the ecosystem. Should it be explicitly called out in the spec?

I also had in my notes that I would expand the xforms-value-changed documentation to make clear that actions triggered by the event should be nested within the form control of the value they observe. However, I'm wondering whether our current link out to the W3C XForms spec may be sufficient. It makes it clear that core form controls are the event targets. Would adding an example help avoid confusion? Do you think there also needs to be some additional text describing the event?

1 Like
13

Yes, I think that would be very good to do this. I just added https://github.com/opendatakit/xforms-spec/issues/267

I think we should elaborate a bit. At least an example showing multiple setvalue/xforms-value-change
elements inside one input, and one of these having its own form control would probably be helpful.

1 Like
14

Sorry i'm late to the discussion. I don't hold strong feelings against naming the column "when", but I'm trying to imagine seeing the column in the wild and what people will expect the behavior to be.

Most rows in an XLSForm describe questions with UI elements. If I didn't know better, and I saw a "when" value next to one, I might expect it to have meaning akin to "relevant" . Ie, next to a question or group, the "when" value might describe when the cell should show up in the form.

I suggested in one of my questions on the PR #442 - "Add "when" column to use value changes as triggers for calculations" that maybe "calculate_after" (or "calculate_when") might work. I think this better describes the values in the cells, especially because (from what I gather) the value of this column is only applicable to rows with a calculation value.

# an example with "calculate_when"
| type     | name      | label | calculation | calculate_when |
| text     | a         | A?    |             |                |
| dateTime | time_of_a |       | now()       | ${a}           |

(And I realize it's already been coded, so again i'm sorry for being late here. But if we want to change the name I could update the PR)

15

One thing that I forgot to mention in Slack is that we want to eventually use this column for actions other than setvalue in addition to with events other than xforms-value-changed.

For example, it could be used to surface the setgeopoint action on the xforms-value-changed event. This would likely look like

type name label when
text a Enter text
auto_geopoint b ${a}

https://www.w3.org/TR/2002/WD-xforms-20020118/slice10.html has W3C XForms actions that we might consider adding to the ODK XForms spec in the future.

It feels like calculate_when would not make as much sense in this context. I see your point about the meaning not being entirely obvious from reading a form but is this a common scenario? I guess a related downside is that it's not practical to search documentation for when. Perhaps trigger_when gets at a similar idea and could work with other actions.

16

Oh, ok. I like the idea of a trigger for a hidden geopoint / auto_geopoint.

I am guessing that the default behavior would be --in the absence of a value in that column-- that it would be collected after the previous question in the form?

When naming variables in code, I generally opt towards making variables and stuff more verbose, with the opportunity to simplify with a grep-and-replace at a later date if it feels like simplification is worthwhile. However, this isn't always a good idea in pyxform stuff because it can lead to alias soup.

With XLSForm representations of surveys, i'd say my opinions are usually influenced by the JSON representation of the question. And, in this case, I think "when" works fine.

# In YAML because it's easier to read than JSON

- type: text
  name: yourname
  label: Your name?

- type: dateTime
  calculation: now()
  name: timestamp_yourname
  when: ${yourname}

although, I am nagged by the fact that all the example "when" values are simply ${} wrappers around an individual question name. And now that I've typed it out... in this scenario I might prefer to describe it like:

- type: text
  name: yourname
  label: Your name?
  # with a "timestamp" column? for example
  timestamp: name_of_timestamp
  # pyxform could add a calculation:now() afterwards
  # add it to the instance as <name_of_timestamp>

Ok... so. What's a hypothetical future example of a more complex value of a "when" on a calculation that can't be met by some sort of timestamp column?

Like... maybe there's a scavenger hunt where you want to get the timestamp and GPS when the person has answered N questions correctly? Or perhaps there's an example more relevant to field work.

Anyways, I really like the functionality that this will add!

17

Also, I'm a little confused by the example that @martijnr posted with "Date of diagnostic" when= "${a}" with an empty calculation, mostly because the 4 questions in a row and I don't understand when one would want to save an empty value, and when this level of complexity couldn't be achieved with some "relevant"s

18

There's no formal proposal for auto_geopoint yet but I would advocate for a required explicit triggering event.

We no longer introduce aliases to pyxform because it's very confusing when looking at a form definition that has undocumented column names. So we do want to make the name useful immediately.

None of the current maintainers use the JSON representation. It's not user-facing so it's not clear to me why it's a priority. I think the most important thing is that the name make sense to users and be relatively easy to recall and look up in documentation.

The primary use case is dynamic defaults when a value changes. It would overcome the limitations described in the tip at https://docs.getodk.org/form-logic/#dynamic-defaults. So timestamp is not adequately descriptive.

Think of the dynamic default case. Let's say you have required name, age and school questions. You may want to force the enumerator to reconsider age and school if name is changed or just clear those questions so they don't have to do it manually.

1 Like
19

We no longer introduce aliases to pyxform because it's very confusing when looking at a form definition that has undocumented column names. So we do want to make the name useful immediately.

Agreed :100: . I just meant this to say "I opt for explicit" when choosing how to name things.

Re: the JSON representation-- I brought it up as a different way to conceptualize the values. The example I typed up was a flat structure so it is equivalent to the XLSForm rows. But I see how this wasn't pertinent.

Yes the dynamic defaults example is relevant. Thanks for the tip. I'll try to expand an example so I understand it before I write more

2 Likes
20

Nice to see you here again @Alex_Dorey! It's a good exercise (for me) to revisit what makes the xforms-value-changed event in conjunction with a setvalue action so special. It will allow us to do the following things that we currently cannot:

  • trigger a "calculation" (though not a calculation in XForms speak) for B when A changes without a dependency on ${A} inside that calculation
  • have control over exactly "when" that calculation is re-evaluated (which is useful for timestamps but also removes some of the burden on the client to not re-evaluate an expression too often and negatively affect performance. Even a hyper-smart client such as :star2: Enketo :star2: still uses a better-safe-than-sorry approach when it's not sure whether there is a dependency)

A relevant could be used to clear a value, but this relevant-pruning will only be done at the time the form is submitted (or saved as final). So in @lognaturel's example the value Name is changed from Joe to Alex but Joe's details for Age and School are still there (regardless of whether the Name changes to empty in between).

2 Likes