XForms spec proposal: add event fired when new repeat instance is created

Development
,
1

The odk-instance-first-load event combined with the setvalue action make it possible to define dynamic defaults that will be evaluated once when a new form instance is created. However, this mechanism doesn't allow for dynamic defaults set when new repeat instances are added because those instances don't exist when the form instance is created. To support this case in a consistent manner, @martijnr proposed introducing an event that is fired when a new repeat instance is created.

The W3C XForms spec does describe an xforms-insert event. That event is fired in response to an insert action which is generalized (not just for repeats). The event has various properties and the spec says that "An XForms Model Processor [...] should support the notification events xforms-insert and xforms-delete" (not must). I don't yet have a great understanding of how the properties of the event are used but my sense is that this is introducing a lot more complexity than we need at this time. As @martijnr pointed out, JavaRosa/CommCare defines a custom jr-insert event and this is probably why.

I propose that our spec take a similar approach and add a narrow event that just fires when a new repeat instance is added. We could name it jr-insert since that already is in JavaRosa but I'd like to suggest something more descriptive like odk-new-repeat, odk-repeat-add or something like that. If ever we wanted to support the full insert action and corresponding xforms-insert event, the door would still be open for that in the future.

For example:

...
  <model>
    <instance>
      <data>
        <my_age />
        <friends jr:template="">
          <first_name />
          <last_name />
          <age />
        </friends>
      </data>
    </instance>
    ...
    <setvalue event="odk-new-repeat" ref="/data/friends/age" value="/data/my_age + 2" />
    ...
  </model>
<h:body>
  ...
  <repeat nodeset="/data/friends">
    ...
  </repeat>
</h:body>
...

When a new friends repeat is added, the default value will be set to 2 more than whatever the user set as their age. jr-insert-example.xml.zip (2.2 KB) is a form with the jr-insert event name that works in Collect today.

The new repeat would be used to qualify the references used in ref and value so nested repeats arbitrarily deep should be supported.

For XLSForm spec proposal: add syntax to make it easy to use a value from the last saved instance, any expression used in the default column in a row that is nested in a repeat would result in a setvalue expression with the event proposed here added to the XForm model.

One odd case I'm not sure about is when repeat instances are part of the original form definition (e.g. when jr:template isn't specified. The repeat insertion event would not be fired for those so I'm not sure what if anything the XLSForm spec should say about that.

I know that's a lot to digest. Questions to consider:

  • Are we ok inventing a new, narrow event just for repeat creation?
  • Is having a generic insert event that fires on any repeat insertion and using the action ref prefix to filter when the action should occur an acceptable simplification? See explanation.
  • If so, what should the event be named?
  • If so, are the semantics described above what you would expect?
  • Either way, what should XLSForm do when repeat instances are part of the original form definition?
2 Likes
2

One more thing I realized I should have called out explicitly. The proposal above (based on the jr-insert implementation) includes a big simplification on the properties of the xforms-insert event: actions are registered on a generic insert event rather than on insert of a specific repeat type and it's the absolute prefix of the action's ref that determines which actions run.

For example, let's say there were an enemies repeat in the form above. The generic insertion event would fire when an instance of enemies was added and this would trigger the setvalue action. But since the action's ref starts with /data/friends and not /data/enemies, the action would have no effect.

This means that ref always needs to be an absolute path which in turn means inserting a repeat can only change values in that new repeat instance. You couldn't, for example, set up a counter of repeat instances that gets updated on insertion (the count function can be used for that particular requirement).

3

Not to derail anything, but... is there any reason we couldnt simply use the existing once() for this purpose? Its actually probably exactly the assumed behavior of once() [hence the name...], despite the fact that in reality what it is doing is the equivalent of coalesce(., ${whatever})

... and instead fix javaRosa DAG check to permit legitimate XPAth expression like coalesce(., ${whatever}) (which I rather wanted to propose anyway...)

It just seems like we may be introducing a degree of redundancy here between a (new) odk-instance-first-load event and once() XPath function; at least on the surface to a user, semantically they seem to be intended to serve the same/similar purpose (although again, I recognize the actual implementation behaves quite differently)

Which is to say, I'd probably prefer if once() did just that - ie fire once (presumably on form load) and never thereafter - and then enable the existing coalesce(., ${whatever}) to actually do what once() is currently doing today [which I think other clients like Enketo permit]. Or deprecate once() in favor of this new instance-first-load mechanism? Or ... Thoughts anyone?

4

To be explicit, you are suggesting an alternative to both this and XLSForm spec proposal: add syntax to make it easy to use a value from the last saved instance which is to change the semantics of once() from running when the context node's value is empty to once on instance first load and to use that to define dynamic defaults, did I get that right?

I see in the last message in the last saved instance XLSForm syntax thread that the spec was approved during the TSC call. Did y'all discuss your alternative suggestion by any chance?

I agree that once() is not a very good name for what it does. I suspect it was added as a quick hack for functions that rely on state outside the form such as random() and now() and that the ramifications were not thought through at the time. I put it in the same "not-my-favorite-but-users-rely-on-it" bucket as jr:choice-name and indexed-repeat.

I see what you are getting at and it does have a certain simplicity. That said, I'm even less comfortable with this new version of once() which knows when it's being run. I think it's also unfortunate to have both a calculation and a user interface control for the same instance node.

XForms is fundamentally event-driven and provides a nice mechanism for triggering behavior based on events so using events and actions in this case certainly seems more XForms-y. I also can't think of a function-driven way to support things like the xforms-value-changed event so it seems we'll want to continue making use of the event/action paradigm.

I would not be opposed to putting once() as it exists today on a deprecation path.

1 Like
5

Yeah, bit last minute... sorry. :disappointed_relieved: It only just occurred to me when re-reading stuff before the meeting. once() has been nagging at me for a while, and I think it just 'clicked'...

I wanted to run the idea by you before discussing anything further. I dont see an issue with the current proposal, indeed it seems more general purpose (which is a plus). But it does rather leave an open question in my mind - and potentially users - when do I use this new feature vs when do I use once(), what's the difference between the two, etc.

I think proceeding with instance-first-load makes sense, as being the new recommended (and more clearly defined) mechanism to populate elements with a default value on form load, deprecate once() as the hacky old way of trying to accomplish something mostly equivalent via XPath, and do something with javaRosa DAG checking to enable the legitimate use of coalesce(.,${foo}) when needed [which currently you have to use once() to 'fake out' validation]. The last issue of which I was going to raise via a new roadmap entry for TSC consideration...

Agreed, which is why once() always felt wrong. XPath functions are just functions, which ideally should have no side-effects, and certainly no memory (as is implied by 'once'). And whatever actual functionality once() is currently accomplishing is already handled by coalesce().

2 Likes
6

Your proposed plan of action sounds good.

I'd like to wait for a plan on how repeats will be handled before moving forward on implementation, though. I'll be very interested in hearing your thoughts on the narrow odk-new-repeat (or other name) that I've described above.

I aggressively agree. :smile: :beers:

7

Can you wait till the 11th hour, again? :grin:

8

2 posts were merged into an existing topic: Spec proposal: add first-load event to replace xforms-ready

9

Thanks a lot for this proposal!

Wrt the event name odk-new-repeat sounds good to me.

I do not think the presence of an explicit template makes a difference. If a template is not there the implicit template is the emptied copy of a (the first) repeat instance. That brings up an interesting revelation, in addition to the one discussed above wrt once(). I think this new event would allow us to deprecate jr:template (since it's only raison d'être is defaults)!

No, I don't think that filtering from the ref path is the way to target the setvalue action for this event. The ref's only purpose should probably be to point to the node in the model to set the value of.

I'm wondering we should provide context in a similar manner as we describe in the example in our spec for value change events. So in your example, the setvalue directive could perhaps move to the <body> like this:

...
  <model>
    <instance>
      <data>
        <my_age />
        <friends jr:template="">
          <first_name />
          <last_name />
          <age />
        </friends>
      </data>
    </instance>
    ...
  </model>
<h:body>
  ...
  <repeat nodeset="/data/friends">
      <setvalue event="odk-new-repeat" ref="/data/friends/age" value="/data/my_age + 2" />
      <input ref="/data/friends/age">
           <label>Update age</label>
      </input>
      ...
  </repeat>
</h:body>
...

I added the <input> as well in case the question has a form control. The <setvalue> could be probably be inside the <input> as well, or maybe not?

If done like this, we should specify in the spec that the odk-new-repeat event fires on all nodes inside a newly created repeat instance.

10

Thanks as always for your thoughtful comments, @martijnr. :heart_eyes_cat:

Agreed!

I think this would be good to clarify in the ODK XForms spec. My understanding from the W3C spec is that it's the repeat form control in the body that defines the template. I believe that's equivalent to what you're saying.

My question isn't so much about templates, it's about repeat instances already in the body. Maybe easier to see if you think about the case where the form definition defines more than one repeat instance such as in example B in the default value spec. Probably the repeat creation event would get fired for all of those at the time of primary instance load? If so, that would mean that to define some existing repeat instances and also provide a template for new ones, the form would include a setvalue action with an expression that sets the value only if it's not blank.

Or maybe the event would never be fired for those? Then I think that to have dynamic defaults on all instances of the repeat including the one(s) defined in the instance you would need a setvalue action that responds both to form instance and repeat instance creation.

:sob:Ok, fine.

Upon more reflection, here's how I believe the W3C XForms xforms-insert event works with setvalue:

  • the event is dispatched to the instance (see this table)
  • any actions that want to listen for it are directly in the model (as siblings of instance, just like in my example above; that means they are triggered when the event bubbles up)
  • an action that responds to xforms-insert can't be nested in a form control because the event is not dispatched to control elements (or descendants)
  • the event provides an evaluation context for the setvalue action's ref (the newly created repeat instance)
  • if the ref is a relative expression that can be contextualized given the provided context, then the value expression is evaluated using the contextualized ref as the context node. If the ref can't be contextualized given the event context, nothing happens. This is one way for the action to only run when instances of a particular repeat are added.
  • if the ref has an absolute path, I believe the action will run any time any repeat instance is inserted.
  • the action can also set the observer attribute to only respond to inserts of instances of a particular repeat

I think it's the evaluation context piece that's tricky. Here's what W3C XForms says for the evaluation context used to evaluate the XPath expression in setvalue's value: "The evaluation context for this XPath expression is the result from the Single Node Binding". My understanding from the section on Single Node Binding is that this means the contextualized ref is used as the context for value.

And this about a change from XForms 1.0 to 1.1:

"the setvalue action has been improved due to the addition of the context() function. Now it is possible to express the value attribute in terms of the same context node used to evaluate the single node binding. This improves the ability to use setvalue inside of a repeat to set values of instance nodes that are outside of the repeat nodeset based on values that are within the repeat nodeset."

Here's how I understand your counter-proposal; please correct me if I got any of it wrong:

  • actions that are triggered by the odk-new-repeat event must be nested in a repeat form control
  • when a new instance of a repeat is added, the odk-new-repeat event is dispatched to the repeat, triggering actions that are directly nested in that repeat's form control (that is, if an action is defined in the form control for /data/repeat1/repeat2/, it will be triggered when an instance of repeat2 is created, but not when an instance of repeat1 is created)
  • values of any attributes defined on an action triggered by odk-new-repeat are evaluated using the contextualized ref node as context

The nesting makes it clear which new repeats should be responded to. It also provides a clear evaluation context node for the ref (the new instance of the repeat it is immediately nested in). If the evaluation context node for value is indeed the contextualized ref, then if ref is a node outside the repeat, value can't be a node in the repeat. I think that's a fine limitation.

I don't think that would make sense -- the nesting identifies the repeat whose odk-new-repeat event should be responded to, right? I don't see what additional information nesting it in a form control provides.

I'm tentatively onboard with your suggestion. Let's see whether I misunderstood anything or if the additional references I pointed to might provide more ideas.

1 Like
11

Do we need to consider any broader ramifications of this; that is, firing various (other) events as a consequence of what happens to already be in the initial primary instance. eg I could potentially see that you should also fire an initial xforms-insert on basis of the primary instance existing itself (which I dont think we were proposing, or were we?). Specifically:

4.4.1 The xforms-insert Event

Dispatched in response to: Successful insertion of one or more nodes by an XForms insert action.

Target: instance

Bubbles: Yes

Cancelable: No

Context Info:
Property | Type| | Value
inserted-nodes | node-set | The instance data node or nodes inserted.
...

And if we do - that is, fire off various events on existing contents - then we'll have to be careful these are fired off in an appropriate order; eg a repeat's xforms-insert must occur after primary instance xforms-insert... [aside: in my current implementation I dont do anything here - I simply copy the form definition's entire instance XML blob over and it becomes the starting point for the new submission. So its not clear to me what level of 'processing' of the initial primary instance - which is essentially what is implied in order to generate all these events - is required/mandated by the Spec. Obviously (ie selfishly) in my case I'd prefer 'none' :slight_smile: Other than perhaps odk-instance-first-load/xforms-model-construct-done or equivalent to trigger instantiating defaults].

My feeling is that these prescribed W3C XForms events are largely around when things happen to the instance XML - ie the (interin) result - as opposed to when the client itself happens to perform certain operations (although in most cases there's usually a direct consequence to something changing in the XML). That being the case, pre-existing contents in the instance XML would not cause events per se. But I'd be interested to get your two views on this somewhat philosophical question...

12

Agreed that firing an event for repeat instances that are already in the instance is not ideal. I do not think that this event we're describing should be fired based on creation of the primary instance and in fact I think that whatever event we define for now should be limited to repeats (at least for now).

That is what I'm leaning towards as well. So that would mean that in order to define defaults for all instances of a repeat including the one(s) defined in the form definition, a setvalue action would have to be triggered by both instance first load AND repeat instance creation, right?

In the mean time, I've made two interesting discoveries following @Xiphware's lead of looking through the documentation for Orbeon forms (probably the most complete W3C XForms engine implementation):

One thing I get out of that is that even Orbeon works some amount outside of the strict W3C XForms standard.

The Obeon spec says for xxf:default:

For dynamic values, for example coming from request parameters or session values, there is no declarative notation and you must use xforms-submit-done, xforms-model-construct-done, or xforms-submit-ready"

That could be taken to imply that xforms-model-construct-done is equivalent to our newly-introduced odk-instance-first-load as @Xiphware hypothesized here but I don't find it particularly conclusive.

To me, the fact that Orbeon has introduced a separate construct for dynamic defaults suggests that we're not crazy to introduce custom features for ODK XForms.

I don't think we've considered using an attribute. We could follow their lead and do that. odk-instance-first-load would still be supported but the preferred approach would be to use the new attribute.

Or we could have the setvalue used for defaults in repeats respond to both odk-new-repeat and odk-instance-first-load:

   <repeat nodeset="/data/friends">
      <setvalue event="odk-new-repeat odk-instance-first-load" ref="age" value="../../my_age + 2" />
      <input ref="/data/friends/age">
           <label>Update age</label>
      </input>
      ...
  </repeat>

This is also a good opportunity to make sure we're on the same page regarding evaluation context. In the example above with relative references, the evaluation context node for age would be a repeat instance e.g. /data/friends[3] and so would be resolved to e.g. /data/friends[3]/age. Then that would be used to contextualize the value so e.g. /data/friends[3]/age/../../my_age -> /data/my_age.

1 Like
13

<setvalue event="odk-new-repeat odk-instance-first-load" ref="age" value="../../my_age + 2" />

Can you point me to an (W3C?) example where multiple events in a single handler are listed like this? I've not found one yet, and just want to confirm this is legit [not that it probably matters, ala "...we're not crazy to introduce custom features for ODK XForms" :wink: ]

1 Like
14

Well, that is certainly a LOT simpler for user's (ie ODK form writers) than having to mess around with event handlers. Basically an (arbitrary XPath) calculate that's really evaluated only once! [unlike our beloved ODK once()... :roll_eyes: ]

It'd also probably be a simple addition to XLSForm: under the default column, if its a literal pre-populate the XML element as today, or if not stick the expression (or node reference) in a new default binding [in fact you could put literals in the default binding too if you wanted and forgo pre-populating the instance XML entirely]

Implementation might be a bit easier too: soon as a new instance XML is instantiated (but not on reload), run thru all the bindings and directly (and immediately) update any elements having a default in their binding. Ditto run thru any associated bindings when instantiating a new repeat group and do likewise.

But I'm sure I'm missing some gotchas... :slight_smile:

15

Thanks for pushing to get this right, @Xiphware! :heart_eyes_cat:

If we follow XML Events 2, it's unambiguously supported. See in the introduction and in the description for action that the type for event is QNames. See QName definition and a description of QNames as "A space-separated list of QNames". That is what Orbeon does. See an excerpt from this form that defines part of their form builder (yes, much of Orbeon is written in XForms :dark_sunglasses: ) :

<xf:setvalue event="xforms-insert xforms-delete xxforms-replace xxforms-value-changed" ... />

Alternately we'd need to have two separate actions that respond to different events which is fine too.

Yes, but given how few people author XML forms by hand, does that matter? I'm guessing even those of us who could do it tend to choose not to or at least start from a generated form definition and make edits to it rather than starting from a blank page.

One other thing that might change your mind about the attribute approach -- it only provides support for the setvalue action. We need to be able to trigger the odk:setlocation action as well and presumably some yet-to-be-dreamed-of actions in the future.

1 Like
16

Ah, before my time eh [well, not really, but I wasnt paying quite as much attention in March '18 ha :slight_smile: ] So this basically boils down to being able to support asynchronous setvalue operations (eg odk:setlocation triggered when eventually get a fix). Right @martijnr ?

Asynchronously changing the instance XML - that is, not as a direct (ie immediate) consequence of user interaction - seems like it could get messy: eg what if I'm on a question that suddenly becomes irrelevant due to some other property changing in the background, ... Have these sorta scenarios been worked thru somewhere I can catch up on?

17

That specific action does. Consider xforms-insert or setfocus as examples of actions that one might want to trigger based on an arbitrary event and that wouldn't be addressed by the attribute option.

I'm working through an implementation and will make sure those are documented.

18

Thanks a lot to you both for your thorough research and analysis! Very good finds in Orbeon and the XForms specs.

Some default attribute seems attractive. However, if we have to support events+actions anyway (for setlocation, value-change-events), and want to keep odk-instance-first-load we're not really solving anything. We're just adding convenience for XML hand-coders (and inconvenience for us poor developers).

Yes, very good point. I think it would/should not fire on those. The double event listener seems like a good solution (so die-hard XML Form coders can decide whether they want defaults in those instances).

I'm very intrigued by @LN's marvellous find of the context() function in XForms 1.1 and the interpretation of Single Node binding. I don't understand the underlying issue that they are solving with context() yet (e.g. when thinking about a calculate inside a repeat where the context is (in) the repeat instance - why did the setvalue element have this issue and a calculation did not?). Will try to wrap my head around this.

Yes I think this is okay for events + actions (though you could definitely create problems with poor form design, e.g. make sections irrelevant while the user is laboring to enter data in them). Here is the discussion.

19

That's sounding right to me.

:woman_superhero:t5:

Yeah, those specs are some dense reading. I don't think I've ever read the same sentence as many times over again. My understanding is that it's because of the way the context for setvalue's value attribute is defined. It could have been defined as the same context node as is used to contextualize the ref but it's defined as the contextualized ref itself.

In the case of calculates or constraints, the evaluation context node is always the current node. I think that's ultimately the same thing because the current node is defined by a bind's ref but binds are always at the top level so you don't have a second context to consider.

It was clear in my head but after I wrote this now I'm confused again. Will need to come back to it once again for another think-through.

20

This is why I have been advocating to only perform such activities over (and after) a few beers. I'll buy the first round at the convening... :slight_smile:

On a more serious note, I'm still worried about two threads writing to the instance XML asynchronously... At least for me, having all writes presently driven by the (single) UI thread is fairly straight-forward (and unmessy). But opening that up to mutli-threaded writes opens up a whole realm of potential synchronization issues, mutexes, ... :scream: