Collect: Improve geopoint dialog so it better highlights acceptable accuracy

We've got one more addition planned for this release plus some possible language tweaks depending on the answers to some follow up questions I have. We'll be introducing an unacceptableAccuracyThreshold attribute (wording for XLSForm is to be determined) to configure when the "unacceptable" message is shown. We expect this to be particularly helpful for organizations that have high accuracy requirements. They'll be able to set it equal to the accuracyThreshold (default of 5m) so that points are shown as "unacceptable" until auto captured.

On the flip side, for contexts that never want an alarmist message, they could set it to some large value (e.g 100000). @Tino_Kreutzer I hope this addresses the concern about the "unacceptable" word which I do understand. The problem we are most trying to address with these changes are data collectors who immediately capture points and don't wait at all to improve accuracy (some analytics about current behavior at the bottom).

If a form doesn't specify an explicit accuracyThreshold then the target accuracy for capture is 5m. 5m before that threshold (so 10m with the default), the message will say "Improving accuracy. Please wait." The idea is that this is relatively neutral message. A 5m difference in the accuracy radius is a significant difference in granularity (it's 5m in every direction). We think/hope the "Please wait" will encourage actually getting to the target accuracy. It sounds like you're concerned that users would end up being stuck for a long time and feeling like they can't save? Is a point taking a really long time to get accurate something you've experienced recently? We believe this to be rare now.

"Poor accuracy" is the message shown between the unacceptable threshold and accuracyThreshold + 5. So indeed, if you have a form without an explicit accuracyThreshold, 12m will be identified as "poor". I think this helps meet the goal of encouraging folks to wait until the desired threshold is met. I agree that there are scenarios in which it would be acceptable. The question is again the one above: what bad behavior do you imagine these messages would lead to. See below for some analytics insights into what users are doing today and the behavior changes we hope for. Do you have a sense of what you'd prefer? Continuing the more neutral "improving accuracy"? Something else?

Could you set an explicit accuracyThreshold of 30 in that case? Or is it more like what @Florian_May describes that yeah, you'd like to get a more accurate point if possible but you don't want people to spend more than N seconds on it? Do you know for sure that people having to wait a long time is a problem today?

accuracyThreshold exists today but requires a new column in XLSForm and is not in the ODK docs. We plan to add it to the parameters column for the next pyxform/Central release to make it more accessible. Using the extra column will continue working so you could run with that now if you want.

I hear you. Let's finish polishing what we have now, get it out out, make sure accuracyThreshold as described above is better documented, keep an eye on analytics, and see what kinds of results you get. More configurability increases our maintenance burden and paradoxically leads to the expectation that it should be infinitely customizable for any scenario. Looking at analytics below, getting low-accuracy points is not a huge problem across points captured (but we know it hits certain contexts disproportionately, e.g. mass surveys with lightly trained staff/volunteers).

Analysis of analytics added Dec 14

We added some analytics to inform this work in the Dec 14 point release. The last month is probably not entirely typical because of several holidays but we have enough points captured to make it worth looking at. Of about 1.3 million points we have data for, 888k were captured automatically when the capture threshold was reached. That's the behavior we generally want so it's good that 68% of this sample are already doing what we hope. Of the remaining 32% points captured manually, 30% of those are captured within 2s meaning data collectors probably didn't even look at the accuracy (that's 10% of the full sample). We tried to bucket the accuracy of captured points into acceptable/poor/unacceptable but unfortunately the analytics interface has changed and I can't easily analyze that right now.

Without the accuracy buckets it's a bit hard to draw strong conclusions around what's going on. But the 68% auto capture is higher than I expected. That's good -- we don't have as broad of a problem as we'd feared. Again, we can't tell conclusively but we believe that certain organizations, devices or physical contexts are particularly hard hit by accuracy issues and those are the folks we want to help. For everyone else, we hope they just continue doing what they've been doing: wait until the point is automatically captured.

We'll know we've had positive impact if the percentage of points captured immediately goes down and the percentage of points captured automatically when the desired accuracy is reached goes up.

2 Likes