Collect: Improve geopoint dialog so it better highlights acceptable accuracy

@LN you're right, I meant accuracy.

Background capture is useful for some contexts. In our surveys we tell people to get a good GPS signal before going into someone's house, so they have to make sure it's recorded before the interview begins.

I don't disagree, training is needed to get a good location. My hope is that we can reduce the amount of training needed. Motivation isn't our problem, it's lack of understanding of the basics behind geolocalization. By showing fewer parameters and having a good graphical (or numeric) indication of what is a good vs a bad signal we can improve data quality and shorten time spent on training. So as you said, we shouldn't remove all the relevant numbers but I suggest we display them at the bottom to still allow troubleshooting (likely with the help of a more technical colleague in the field).

My main suggestion is to find a way for an indicative progress bar based on the current accuracy (in addition to the number) which changes color once it reaches the acceptable threshold. Numeric literacy has been a problem in our experience so while we can teach "wait for the number to be smaller than 100" some people forget what the number is and become confused and uncomfortable with this process.

1 Like

FWIW I really like this approach

Thank so much for the insights into scenarios you're familiar with, @Tino_Kreutzer and @Florian_May. @Tino_Kreutzer I apologize for not following up about a call. I figured you'd have your hands full! Plus, this thread has captured a lot of valuable information so would be great to keep the conversation here as we get close.

That's really great to hear. Let's see what you think of our first round of improvements and whether you think there's more necessary. I agree that the protocol you described is probably not needed anymore in general.

We've added a progress bar as suggested:

It has 5 steps each representing 20% of the bar:

  1. No location available: Trying to get location. Please wait. (this 20% is always filled when the dialog opens)
  2. Worse than unacceptable threshold (default 100m): Unacceptable accuracy. Please wait.
  3. Between unacceptable threshold and 5m before capture threshold: Poor accuracy. Please wait.
  4. 5m before capture threshold to capture threshold: Improving accuracy. Please wait.
  5. Currently the dialog closes when the bar is filled

This is now available in beta. To enable the new experience, please go to Settings > Experimental. You'll then need a form with a geopoint question in it such as the All Widgets form from the demo server. We look forward to your feedback!

One question we have is whether 100m feels like a good default for the unacceptable threshold. It corresponds to approximately a city block. There may be cases in which that is useful enough so we do want to make it configurable. However in the contexts and with the devices we’re familiar with we would expect that getting a better accuracy would be necessary and relatively quick.

5 Likes

A few comments here from someone who trains and leads field geologists using Collect for sample collection and relies on GPS signals.

I am not sure how the "accuracy" value is actually calculated or determined (GPS chipset? AndroidOS?) as in the past we looked at PDOP or similar semi-quantitative value to determine satellite constellation geometry and therefore an assessment of accuracy... and while I can fully understand the desire to keep the UX simple and clean, please keep options available for more technical users that need to know and record GPS location information, metadata, and have customizable thresholds (or none at all).

I'm not sure if anyone has considered this, and forgive me for jumping in late to the discussion here, but a feature that would be welcome on our side of things is the ability to have position averaging be an option. This can increase precision and most times accuracy for stationary recording. Even if averaged for 10-20 seconds. Food for thought as another feature request for down the road.

Cheers,
Carl

Thanks for chiming in @Carl_Schaefer! The change described here is only in the user experience for the geopoint widget. It's mostly rearranging information that's already there. The most significant change is to add one more type of feedback to data collectors: "this point has such a bad accuracy radius that it's unacceptable." That will by default happen with 100m+ accuracy but will be configurable.

You can read more about geo-related functionality in Collect in the documentation. In particular:

The accuracy radius is an estimate of what Android calls the radius of 68% confidence: there is a 68% chance that the true location falls within this radius. This is an estimate reported by the Android system based on the available sensors (GPS, network, etc). The accuracy radius itself may be more or less reliable depending on the sensor(s) used and current conditions.

We do not currently expose raw GPS values and don't intend to unless Android functionality significantly changes. According to our testing and field feedback, Google's fused location provider does consistently provide more precise points more quickly than raw GPS. Even in remote areas, it can use things like cell towers to improve the location estimate. It is the same provider used by Google Maps.

The downside of the fused accuracy provider is that we don't know the details of how it works and it gets continuously updated by Google. As far as we know and can tell by observing its behavior, it already uses techniques like averaging so it would not be appropriate to layer on. There is a discussion on point averaging here.

Folks with very high accuracy requirements typically need to use external hardware which is used through a secondary application. You can learn more in the documentation.

3 Likes

The latest beta uses the new dialog by default and includes the satellite count.

1 Like

Thanks @ln, this is great! I tested the beta and it looks and works very well. I think some more feedback from others (looking at @chrissyhroberts) might be useful regarding the labels and customization. My first impression was that an accuracy of 12m shouldn't be called "poor accuracy" (anything >5 currently is).

I would vote on removing the "Please wait" part of the label, since whether someone should wait or not is very project dependent. I would, for example, instruct people not to wait once they reach 30m. I'd also prefer changing "unacceptable" to "very poor", since getting a 110m accuracy in a heavy thunderstorm where you hide from the elements is acceptable in my experience.

2 Likes

Just testing, looks great! The new UI feels like an improvement over the old one. Agree with Tino on wording changes.

re "autosave with good enough accuracy if perfect accuracy takes too long": as I wrote earlier in this thread, I could see the benefits of additional and optional parameters to autosave with a lower accuracy after a given time span:

  1. Wait xx seconds for perfect accuracy (<5m)
  2. Wait yy seconds for acceptable accuracy (zz=20m)
  3. Save any location after yy seconds (or not)

With 2 and 3 being controlled through optional additional parameters. This would take one critical and eternally confusing step out of our protocol. We have 400 odd folks per season doing this step 40-50k times all up.

1 Like

I believe that accuracy is a variable that form creators should be able to set - and I am not clear whether this is planned or not.

I currently have two projects. One is to track locations where travelers get sick, the other is to map food stores in densely populated areas. The accuracy of the first is easily 50 km. On the second, if I am more than 5m away, I find my points inside buildings, on the wrong side of the street, or even the wrong street, so I'd like to limit data entry to high resolution. In other words, this is a value I'd like to be able to set when I create the form, or at least when I collect data.

Thanks, Z.

@LN I think that the UI is both neat and intuitive. Looks great in dark mode.

One thing I like less is that if I try to overwrite a previously saved fix, the old data are visible whilst the new fix is stabilising. I find this a bit weird in both look and logic and I wonder if moving the getting location box up to obscure or to blur the old reading would be sensible.

Personally I would like a live updating view of the current gps fix, i.e. in small text under where it says 'satellites : n'.

like

Time elapsed : 01:04
Satellites : 15
GPS : N15°59'54", E13°40'42", 76.4m`

I totally agree with @Tino_Kreutzer about the wording and about the fact that 12m is not bad accuracy, but I'd also like full control of this as a modifier parameter like good=4, poor=10, bad=25

I also agree with @Florian_May that the optional parameters for timeouts or autosaves would be enormously helpful.

IMHO there's not much value in the bit that says Point will be saved at 5m unless this is modifiable.

Thanks again for awesomeness.

2 Likes

We've got one more addition planned for this release plus some possible language tweaks depending on the answers to some follow up questions I have. We'll be introducing an unacceptableAccuracyThreshold attribute (wording for XLSForm is to be determined) to configure when the "unacceptable" message is shown. We expect this to be particularly helpful for organizations that have high accuracy requirements. They'll be able to set it equal to the accuracyThreshold (default of 5m) so that points are shown as "unacceptable" until auto captured.

On the flip side, for contexts that never want an alarmist message, they could set it to some large value (e.g 100000). @Tino_Kreutzer I hope this addresses the concern about the "unacceptable" word which I do understand. The problem we are most trying to address with these changes are data collectors who immediately capture points and don't wait at all to improve accuracy (some analytics about current behavior at the bottom).

If a form doesn't specify an explicit accuracyThreshold then the target accuracy for capture is 5m. 5m before that threshold (so 10m with the default), the message will say "Improving accuracy. Please wait." The idea is that this is relatively neutral message. A 5m difference in the accuracy radius is a significant difference in granularity (it's 5m in every direction). We think/hope the "Please wait" will encourage actually getting to the target accuracy. It sounds like you're concerned that users would end up being stuck for a long time and feeling like they can't save? Is a point taking a really long time to get accurate something you've experienced recently? We believe this to be rare now.

"Poor accuracy" is the message shown between the unacceptable threshold and accuracyThreshold + 5. So indeed, if you have a form without an explicit accuracyThreshold, 12m will be identified as "poor". I think this helps meet the goal of encouraging folks to wait until the desired threshold is met. I agree that there are scenarios in which it would be acceptable. The question is again the one above: what bad behavior do you imagine these messages would lead to. See below for some analytics insights into what users are doing today and the behavior changes we hope for. Do you have a sense of what you'd prefer? Continuing the more neutral "improving accuracy"? Something else?

Could you set an explicit accuracyThreshold of 30 in that case? Or is it more like what @Florian_May describes that yeah, you'd like to get a more accurate point if possible but you don't want people to spend more than N seconds on it? Do you know for sure that people having to wait a long time is a problem today?

accuracyThreshold exists today but requires a new column in XLSForm and is not in the ODK docs. We plan to add it to the parameters column for the next pyxform/Central release to make it more accessible. Using the extra column will continue working so you could run with that now if you want.

I hear you. Let's finish polishing what we have now, get it out out, make sure accuracyThreshold as described above is better documented, keep an eye on analytics, and see what kinds of results you get. More configurability increases our maintenance burden and paradoxically leads to the expectation that it should be infinitely customizable for any scenario. Looking at analytics below, getting low-accuracy points is not a huge problem across points captured (but we know it hits certain contexts disproportionately, e.g. mass surveys with lightly trained staff/volunteers).

Analysis of analytics added Dec 14

We added some analytics to inform this work in the Dec 14 point release. The last month is probably not entirely typical because of several holidays but we have enough points captured to make it worth looking at. Of about 1.3 million points we have data for, 888k were captured automatically when the capture threshold was reached. That's the behavior we generally want so it's good that 68% of this sample are already doing what we hope. Of the remaining 32% points captured manually, 30% of those are captured within 2s meaning data collectors probably didn't even look at the accuracy (that's 10% of the full sample). We tried to bucket the accuracy of captured points into acceptable/poor/unacceptable but unfortunately the analytics interface has changed and I can't easily analyze that right now.

Without the accuracy buckets it's a bit hard to draw strong conclusions around what's going on. But the 68% auto capture is higher than I expected. That's good -- we don't have as broad of a problem as we'd feared. Again, we can't tell conclusively but we believe that certain organizations, devices or physical contexts are particularly hard hit by accuracy issues and those are the folks we want to help. For everyone else, we hope they just continue doing what they've been doing: wait until the point is automatically captured.

We'll know we've had positive impact if the percentage of points captured immediately goes down and the percentage of points captured automatically when the desired accuracy is reached goes up.

2 Likes

I don't have much to contribute here, but the new dialogue is definitely an improvement and much clearer to the enumerator that the value may be poor and they should wait for it to improve.

I was testing with an SM-G900F (no SIM, on wireless):

2 Likes

The new dialog is now available in v2022.1.0 and documentation is updated here. Thanks to all who have pushed to make this even better.

We ended up going back to a gray background instead of showing the activity below.

We discussed this but couldn't come up with a use for it. Do you expect data collectors to know how to recognize a correct location? Or is it more to make it clear that a point is being captured, not just an accuracy?

As your field teams work with this new experience, please consider some of the suggestions made above and my follow-up questions here and we can see about further adjustments!

We're going to add accuracyThreshold and unacceptableAccuracyThreshold as parameters in XLSForm. We are currently planning to use parameter names capture-accuracy and unacceptable-accuracy. Please let us know if you have comments on that naming. unacceptable is unfortunately hard to spell so alternatives would be welcome.

I really like it with the grey background. I think I only wanted the live updating gps fix to counteract the confusion of seeing a static location in the background.

I agree that unacceptable has unacceptable number of letter Cs in it.
how about minimum-accuracy

2 Likes

Another idea is warning-accuracy: the unacceptable threshold represents a threshold above which clients are encouraged to display a warning that the user should change conditions or keep waiting.

1 Like

Unsure if this is intentional or not, but in the new geopoint dialogue, you can't back out of it with the Android soft "<" button, only with the 'Cancel' option on the dialogue. You can back out of placement-map with "<", and take picture/markup image etc.

Good catch! This will be fixed as part of the next release (https://github.com/getodk/collect/issues/5018).

1 Like

A live feedback from an end user :slight_smile:

and in english :

:clap: :clap:

3 Likes

Hi @LN,

I just read the documentation section relative to those new parameters, wich is very clear : https://docs.getodk.org/form-question-types/#geopoint-widget
For the moment I make use of the column "body::accuracyThreshold", can I still use it until the end of the field season, before we upgrade forms to use the new parameters instead of it ?
Even if we update Collect on phones ?

Yes, and in fact you can use it until the end of time. :smile: The body:: prefix is used to directly populate a body attribute and will continue to be supported. The parameters column makes it possible to configure specific question types without having to make the survey sheet wider and with a little bit more validation. The two end up doing the same thing in the XML form.