Spec proposal: end-geopoint to automatically capture location upon form completion

As suggested by the title, this is a proposal to add a new feature to ODK clients (ODK Collect & Enketo) to support auto-capturing the user's location upon completion (or finalization) of a form. This basically mirrors the current odk:setgeopoint action combined with current odk-instance-first-load event that is now used to auto-capture the user's location when they first begin filling in a form (or to warm up the GPS); aka start-geopoint

What high-level problem are you trying to solve?

A rationale for wanting to capture an ending geopoint to ensure that the form has actually been completed on-site, and that the enumerator hasn't simply gone off and filled it in elsewhere after starting the form. I would therefore probably envision an 'end-geopoint' would in most cases be paired with a start-geopoint question in most forms, although I dont think this needs to be a strict requirement.

FWIW an 'end-geopoint' feature has already been alluded to during the original discussions around start-geopoint; the proposal here is to finally make this a reality.

Any ideas on how ODK could help you solve it?

Yup... :slightly_smiling_face:

Proposed Implementation: Add new event type

Currently the existing odk:setgeopoint action, which is supported by both ODK Collect and Enketo, can be triggered either by odk-instance-first-load event (ie start-geopoint) or a xforms-value-changed event or a odk-new-repeat event.

An obvious approach is therefore to utilize the existing client-supported odk:setgeopoint action and introduce a new event type to represent form completion/finalization, analogous to odk-instance-first-load representing form commencement.

There are a number of potential candidate XForm 'form completion' type events that might be suitable to use for this purpose [much as there were a number of existing XForm form initialization events...]. Specifically:

xforms-submit: https://www.w3.org/TR/xforms11/#evt-submit
xforms-submit-serialize: https://www.w3.org/TR/xforms11/#submit-evt-submit-serialize

However, ODK Collect in particular somewhat decouples form finalization from actual form submission; you can finalize a form in Collect but decide to send it up/submit it later. So I might suggest introducing a new ODK event type for this - eg odk-instance-finalize - to represent the point in time at which all the values in the XForm instance have their final calculations performed, non-relevant binding's are nulled out, etc.

In keeping with existing ODK metadata XForm timeStart (aka XLSForm start) and timeEnd (aka XLSForm end), this event should ideally be timed to coincide with when the end timestamp is captured; that is, start + start-geopoint and end + end-geopoint are all captured matching chronologically (nothwithstanding waiting for GPS fix...). I note from preload attributes that end may (also?) be trigger-able from xforms-revalidate events [can anybody confirm this is true in both Collect and Enketo?]. However, I believe these revalidation events may actually occur numerous times over the course of filling in a form, eg whenever new repeat groups are added or removed. Taking yet another time snapshot is probably of nominal expense, but I'm not sure we likewise want to repeatedly poll the GPS hardware for all these events [note, another revalidation should take place at or near form finalization, which should ensure the final end timestamp remains in sync].

A custom new ODK event type odk-instance-finalize event, or equivalent thereof, for form completion is also somewhat in keeping with the decision to go with a custom odk-instance-first-load for form initialization; it avoids having to drag in a lot of baggage to ensure strict XForm 1.x spec conformance if we were to attempt to add new support for an existing defined XForm event(s).

Much like the XForm definition for start-geopoint, an obvious representation for this in an XForm might be:

<bind nodeset="/data/startlocation" />
<bind nodeset="/data/endlocation" />
<odk:setgeopoint event="odk-instance-first-load" ref="/data/startlocation" />
<odk:setgeopoint event="odk-instance-finalize" ref="/data/endlocation" />

XLSForm Changes: Add new end-geopoint question

In keeping with the existing start-geopoint implementation in XLSForm, it seems obvious to expose this new functionality in much the same manner by introducing a corresponding end-geopoint question type. This would likewise take a single name field specifying the form field to be filled in with the acquired geopoint location.

Other Approaches Considered?

1. New XPath function/calculation

Another way to ensure form values are updated at the end of a form is by using calculations, which all get continually re-evaluated during form filling. This fact could be leveraged by adding a new XPath function that polls the GPS and stores the result in a geopoint calculation result. This calculation would be continually re-evaluated, ensuring the target geopoint question had the last acquired location saved prior to the form being finalized. However, this approach was rejected because a GPS-acquisition function is impractical, due to the operation of polling the GPS ostensibly being an asynchronous operation that would requiring such a function to block awaiting a location fix. This concern has been previously raised here: Spec addition proposal: location preload - #9 by martijnr

2. Preload/Presubmit

Traditionally a small subset of 'special' form fields could be initialized in the submission response via XForm preload bindings. In a similar vein it would be possible to add something entirely new, eg jr:presubmit bindings, to similarly indicate specific metadata fields to set at a specific predetermined time using a specific prescribed (but non-XPath function!) operation.

However, ODK is somewhat moving away from using the older XForm preload metadata approach for form initialization, and now using events instead for much the same purpose. So it rather makes sense not to perpetuate the older 'preload' model even further by adding new XForm 'presubmit' bindings and attributes.

3. setvalue-type trigger

As noted earlier, it is possible (currently via XForm XML only) to trigger a odk:setgeopoint action in response to an xforms-value-changed event, much as you can trigger a set-value event (aka the XLSForm trigger column). This existing mechanism can be used, for example, to (re)save a GPS location whenever a specific question has been answered or changed; for example an acknowledge widget [which is actually a XForm trigger, which is different than a XLSForm trigger... oh my :face_with_spiral_eyes:]:

end-geopoint3.xml (1.5 KB)

The above form has been tested and works under both ODK Collect and Enketo, although it does require some manual fiddling with the final XForm XML file to accomplish. This main limitation of this approach is that there remains nothing to prevent a devious user from going back after responding to the Acknowledge and then changing the responses to earlier questions (or worse, going off-site and doing so...). So in that sense this is more a hack that can provide no guarantee the 'end' GPS location acquired actually corresponds to the location the user was when they finished the form.

Outstanding Issues

The main issue I see with auto-capturing an ending geopoint is - as is much the case with the existing start-geopoint - that it can be an asynchronous operation. In particular, when a form does not contain a start-geopoint or any other geopoint/geotrace/geoshape questions, then it may be the case that - at the end of the form - the GPS isnt 'warm' and may take several seconds to power-up and get a satellite fix. The issue then becomes how long should this be allowed to take and what happens on the client (Collect or Enketo) whilst this is taking place? Specifically, how long should the final form finalization step be put on hold and/or delay an immediate subsequent form submission?

However, this is fundamentally not that different than a problem currently faced with start-geopoint: it is entirely possible to have a very short form, eg a single yes/no question alongside a start-geopoint, which takes almost no time for the enumerator to complete and submit. Currently the behavior of start-geopoint states:

The first time that a survey with a start-geopoint question is opened, Collect will attempt to read the device's geolocation. The geolocation reading with the highest accuracy received in a 20-second window will be recorded. A location icon will be displayed in the Android status bar while the geolocation is being requested by Collect.
...
If geolocation information is unavailable, the question will be left blank.

Specifically, if no GSP fix is able to be acquired within a 20 second window then the associated form field will be left blank.

So perhaps a similar approach can be taken with end-geopoint: for a (suitably short?) period of time, form finalization might be delayed until a (suitably accurate?) GPS fix can be acquired (likewise displaying a location icon will be displayed in the Android status bar whilst doing so...). And if not then the referenced form field will, regrettably, have to remain blank. Along with a best-practice recommendation to include a start-geopoint question to first warm up the GPS whenever using end-geopoint to avoid any unpleasant delay when finalizing the form?

Happy to hear any comments (most especially from @LN and @martijnr, given your experience evolving the start-geopoint specs)

  • Gareth
3 Likes

Thanks for a thorough write up!

Since this seems to be mostly for auditing purposes, have you considered using the audit log with location capture? It would provide a lot more context in case of a change in location between form start and end.

Did you also consider exploring how we could surface setgeopoint triggered by a value change event in XLSForm? That could enable form designers to, for example, request location when the value of one of the last questions in the form is set. It would also make it possible to do things like capture location at multiple key points in a form filling session or in every repeat.

The 20s retry window is how the setgeopoint action is defined regardless of the event that triggers it.

If the location capture is going to possibly block user progress anyway, why not use a standard geopoint question with user interface?

Form finalization is already surprisingly complex and can take time so I think it’s worth thoroughly exploring alternatives!

Yup, and in fact I'm working on another spec proposal for exactly that right now! :slight_smile:

But the main issue I see with that approach - as an alternative (and as I described as one of the options considered) - is that it doesn't offer any real guarantee; unlike start-geopoint.

I basically view end-geopoint as a natural companion to start-geopoint, and a nice corollary to the established timeStart and timeEnd 'bookends' (but for form location instead of time).

If the location capture is going to possibly block user progress anyway, why not use a standard geopoint question with user interface?

True. But you could argubaly say the same for start-geopoint... unless start-geopoint is in reality just "please warmup GPS"... which if it is could have just been a simpler metadata flag instead of adding a whole new form field. [playing devil's advocate here! :upside_down_face:]

But in all seriousness, the main goal is to definitively geolocate the form when its actually finished (ala start-geopoint definitively geolocating the form when its started). Yes, this (indeed both!) can be done via the audit log, but that's a pretty heavyweight commitment. And, more practically, adding end-geopoint support to Enketo would be a lot less work than adding an entire audit log.

I've posted a proposed XLSForm spec change for this here: Spec proposal: expose xforms-value-changed event with odk:setgeopoint action in XLSForm

Feedback welcome.

I'm just posting here as well, as I made a post about triggering a calculation at the 'finalisation' stage/end stage of a form here. And although it has nothing to do with GPS/geopoint, it is linked based on the timing element.

As there are considerations for finalising forms, thought I'd just provide the link for additional context!

Thanks both!