ODK Collect v2024.3 Beta: offline Entities, counter, map scale

[LN]

ODK Collect betas are an opportunity to get community feedback on upcoming releases. If you have an ongoing data collection campaign, we recommend quickly verifying your form on a test device. The release will be delayed until all reported issues are fixed. This release is currently projected for mid-October to give a lot of time for testing and we will publish multiple betas.

Joining the beta program
To join the beta program, find ODK Collect in the Play Store on your device (not in the web browser) and scroll all the way down. Please don't join the beta with a device or account actively used for data collection! In particular, note that joining the beta is account-based. If you use the same Google account across multiple devices, do not join the beta with that account.

Leaving the beta program
You can leave the beta program from the bottom of the Play Store listing at any time. Once you leave, you will get the next production update when it is released. If you need to go back to the previous production release, uninstall and reinstall the app. Please make sure to back up or submit any important data.

What to check in this release

• Entity List download and lookups have been fully reimplemented to support offline Entities. Verify your existing Entity forms and see below to try offline Entities.
• The counter question type makes it easy to tally things. Try it with this form.
• Map views now show a scale at the upper left corner.
• Try the Thai Buddhist calendar question type with this form.
• Submissions are now sent in oldest to newest order.
• The required indicator for audio questions is now shown. Thank you for reporting the issue, @Tyler_Depke!

Offline Entities preview
We have now implemented most of the functionality to support offline Entities.

Existing forms will continue to work as they do today and Entities will NOT be created/updated offline. Offline Entities will require updated forms and an updated server. You can find more details about how and when the different parts supporting offline Entities will be rolled out here.

Starting in Central v2024.2, you can opt individual forms into using offline Entities by adding an offline column to your entities sheet and putting the value yes below it.

You can also try out offline Entities using our Central staging server by scanning in this QR code from the Collect beta:

This will give you access to two forms: one to register yourself as an enumerator for the red team or the blue team and one to add items for your team. To verify offline Entities, turn off Internet access to your device before using these forms. You should be able to register yourself, add items, and see the updated team scores, all while offline. This is a small, silly example, but I do want to point out that the red team is winning.

Please note: betas provide an interface for viewing all Entities under Experimental settings so please don't use a sensitive passcode! Also note that these forms will fail to download in earlier Collect versions.

Even these simple forms demonstrate a lot of useful things you can do with Entities like preventing duplicate registrations, showing work summaries, linking Entities together, and more. The red team vs. blue team concept is silly but you can hopefully see how you could do something similar to represent and enforce quotas, for example by respondent gender or age. You can see the form definitions in this Google Drive folder.

Additional testing
You can find a full list of changes in the release notes:

• https://github.com/getodk/collect/releases/tag/v2024.3.0-beta.6

Thanks to all testers for your help!

[ahblake]

@LN

I do want to point out that the red team is winning

Not anymore!

(@yanokwa AW11 best MR2)

Related to my other comment about needing to sort lists for display/draw order, I did notice that when viewing the entity lists on device (under experimental), they're sorted in that display by UUID, not creation date!

(@LN correct link to release notes https://github.com/getodk/collect/releases/tag/v2024.3.0-beta.2)

[LN]

@ahblake

AW11 best MR2

I'll admit I generated some random data to get things started... but yes, they don't make them like they used to.

@ahblake

they're sorted in that display by UUID, not creation date

Yes, we're now well-positioned to sort Entity Lists outside the form context. We now need to decide where to go next: work on the experience to first pick an Entity from a sorted list before picking a form to fill OR work out a way to specify a sort order within a form definition. Each presents its own interesting challenges.

@ahblake

correct link

Thank you!

[ahblake]

@LN The beta won't load one of my forms with a dynamic repeat count . Emailed some example forms showing different behaviour in Collect / Enketo with repeat / nested repeat to triage.

[LN]

We have a beta 3 awaiting review that I'm pretty sure will fix that crash, thanks for reporting!

I’ll have to dig deeper into the others and will let you know what I find.

[ahblake]

I spotted beta 3 on the releases page yesterday and double checked that it wasn't available. Will test as soon as it's out!

[seewhy]

Great work again.

The counter - FAB!!! I've been hoping for this for a few years and had almost given up, so thank you for implementing it. I have a large (for me) project that is currently in progress and I decided to use a separate app to tally items (Thing Counter). The good news for me is that I can change my existing form by just adding the appearance of Counter and it works (just tested it!). Great use of screen space for multiple counters on a single page.

Map scale - excellent - really helpful in the field. Next request will be buttons to zoom in / out so that when it's wet and the pinch zoom doesn't work...

Offline entities - This looks great at an enumerator level, but I'm not quite sure how I would manage data Centrally - those conflict resolution things are a bit complex for my pretty little head! You'll see that I duplicated entries in the Blue team, not cheating, honestly, I was testing to see if it would check for duplicates... Oh, but Blue's now winning Just sayin.
As @ahblake suggests being able to order / filter the entities in the list would be helpful (you appear to have that in hand) - presumably you could draw them into another form (once finalised) using selects too - just trying to follow the Offline Entities discussions elsewhere, so that might be an overly obvious statement!

Summary - excellent to see the incremental improvements across the board to help us work more effectively. Much appreciated.

[LN]

Thanks so much for the feedback! Off topic: I only just now realized the meaning of your username, CY Good one!

@seewhy

those conflict resolution things are a bit complex

Yes, and the good news is that you should generally be able to design a workflow where conflicts are impossible or very unlikely. We have all that handling for complex scenarios when Entities can be encountered in the real world at any time and by any one but we expect those are domain specific and not all that common.

@seewhy

duplicated entries in the Blue team

The items don't have any uniqueness check! They could, but currently it's only usernames that are checked for uniqueness.

@seewhy

presumably you could draw them into another form (once finalised) using selects too

Yes, exactly! That's similar to what's going on with "enumerators" that are registered. You'll notice that as soon as you register a new enumerator (feel free to register your friends and pets if you'd like), you'll be able to use that enumerator in the "Add item" form. That's because the "Add item" form reads from the enumerators Entity List. We could have had username come from a drop down but instead we let you enter free text and then look up the value in the list to see whether it exists.

[ahblake]

I see beta 3 is out - this didn't fix the getInstanceID(...) must not be null' error.

[seewhy]

@LN

I only just now realized the meaning of your username,

Hiding in plain sight all this time but then... choosing a username phonetically similar to their name... who knew

[Thalie]

About the range slider: it is much clearer now, and I can see myself using it in cases where I would previously default to a numeric entry. For example, with integer entries, it is easy to mistype (e.g., entering 9 instead of 90), whereas with the slider, such errors are much less likely.
However, I still have a usability concern that makes me lean towards numeric widgets in some cases. It is easy to unintentionally change the selected value on the horizontal range slider when scrolling the screen if the finger starts near the scale. This could result in unintended or unnoticed adjustments. I am wondering if there is an "easy" way to make slider interactions more directional—perhaps limiting cross-sectional scrolling influence—while still maintaining smooth value adjustment?

Also decimal steps displays unrounded values in my quick test

[mathieubossaert]

@LN

The counter question type makes it easy to tally things. Try it with this form .

Wow ! Just played with the new counter appearance. And I think I achieved something that will please a lot of naturalists
Not in a dynamic way (one question needed per species) but it works fine !
Counter question type.xlsx (466,2 Ko)

[Aly_Blenkin]

@Thalie

About the range slider: it is much clearer now, and I can see myself using it in cases where I would previously default to a numeric entry. For example, with integer entries, it is easy to mistype (e.g., entering 9 instead of 90), whereas with the slider, such errors are much less likely.

Such great feedback! We are glad the slider improvement makes things easier for you!

@Thalie

However, I still have a usability concern that makes me lean towards numeric widgets in some cases. It is easy to unintentionally change the selected value on the horizontal range slider when scrolling the screen if the finger starts near the scale. This could result in unintended or unnoticed adjustments. I am wondering if there is an "easy" way to make slider interactions more directional—perhaps limiting cross-sectional scrolling influence—while still maintaining smooth value adjustment?

Also decimal steps displays unrounded values in my quick test

It would be great to see a video so we can understand what's happening a bit better. A few quick thoughts that come to mind:

• You could also disable the swipe, so they are forced to use the buttons to navigate.

• I'm also curious to know what device are you using? Touch-sensitivity can differ quite drastically.

[LN]

Beta 4 is now in review and that should fully fix the getInstanceId(...) must not be null crash. I don't know whether anyone noticed but the prior betas also dropped support for several common devices because of their OpenGL version. We have now brought back support for those.

[ahblake]

Fixed in beta 4. Thank you!

[Stephen_K_ojwang]

Such a nice show case and a very important feature to add to collect. Being that we have achieved this tally feature, it opens the next idea of ability to log in the time the counter moves from one digit to another. Of importance is the ability to know when the first frog was observed, the 5th frog, 10th and so on... Just to make it simple. I am thinking of two approaches @LN can advise if any of this is currently supported or if there is a possibility to support them in future.

1. Use an hidden calculation to recorde time at which the counter reads a specific number, maybe using once(), now() and join() functions.
2. Trace this counter actions in the audit log csv file. What do you think is the possible drawback to this approach?
   The next feature after timestamp, I know other folks may be intrested in is gps coordinate at the point say the 1st frog was observed, the 5th frog and so on... But this is not my question at per now😄. Again an optional setting addin on the widget to disable the "-" and "+" would be of interest to serve different context, especially in certain context to avoid down voting. Thanks

[ahblake]

@Stephen_K_ojwang

Of importance is the ability to know when the first frog was observed, the 5th frog, 10th and so on

Try this!
Counter question type.xlsx (576.6 KB)

Eg: I saw 9 really quickly, but accidentally counted to 10, so decremented 1, then saw 3 more.

It should work for frogs too

[Stephen_K_ojwang]

Wow very well executed👏. I really like how you play with flip flop, very simple code yet powerful🤗.

[LN]

We have just released beta 6 and it should start rolling out shortly. This should fix an issue with the first Entity in lists that @ahblake reported and also make pulldata use offline Entities. We still have a couple of issues related to offline Entities to address and believe we will start regression testing early next week.

If you use Entities, we recommend that you verify this beta with your forms. By default, offline Entities will not be turned on yet but we have made changes to a lot of the code paths for filtering Entity lists and looking up Entity values.

If you would like to try offline Entities with your forms, you can now do so with Central v2024.2. You will need to opt individual forms in. To do this, add an offline column to the entities sheet of your XLSForm and put value yes below. This is a temporary setting that will be turned on by default in the next release of pyxform. In Central v2024.3, all Entity-related forms will be automatically migrated to use offline Entities.

@Thalie

decimal steps displays unrounded values in my quick test

This is an existing issue related to how decimal values are represented by computers. We'll get to it next release or maybe in a point release! In the mean time, a larger/different step should make this less likely. Thanks for reporting.