ODK Collect v2024.2 Beta: prompt to recover, select from map line and shape styling, offline Entities prototype

ODK Collect betas are an opportunity to get community feedback on upcoming releases. If you have an ongoing data collection campaign, we recommend quickly verifying your form on a test device. The release will be delayed until all reported issues are fixed.

:sparkles: This beta includes an experimental prototype of offline Entities! You may find it useful to try if you are planning future workflows that would benefit from creating or updating Entities directly on device. For now, offline Entities are opt-in, see the end of the post to learn how. We are currently planning for production-ready offline Entities in v2024.3 which we aim to release in early fall. :sparkles:

Joining the beta program
To join the beta program, find ODK Collect in the Play Store on your device (not in the web browser) and scroll all the way down. Please don't join the beta with a device or account actively used for data collection! In particular, note that joining the beta is account-based. If you use the same Google account across multiple devices, do not join the beta with that account.

Leaving the beta program
You can leave the beta program from the bottom of the Play Store at any time. Once you leave, you will get the next production update when it is released. If you need to go back to the previous production release, uninstall and reinstall the app. Your settings will be reset but your forms will remain (though backups are always recommended).

What to check in this release

  • :boom: If Collect closes unexpectedly because of a crash or going out of memory, when you next open the form you were working on at that time, you will see a dialog asking whether you want to recover work that was automatically saved. Previously, recovery was done automatically without any indication that it had happened. You can try this by swiping up to kill Collect while you're filling out a form.
    • :recycle: It is possible for Collect to go out of memory while a form is being filled and then receive a form update when it is re-opened. Previously, the change in form version would mean your prior work could not be recovered. You will now be given the option to recover saved work even if the form has since been updated. Recovered data will be opened with the updated form version. See this issue reported by @mathieubossaert
  • :magic_wand: Dynamic relevance expressions should now update within a field list. See this issue reported by @wroos
  • :nail_care: Styling of lines and shapes in select from map. In addition to the special properties documented here, you can now set stroke and fill to colors and stroke-width to an integer.
  • :shushing_face: Text and number questions now support a new masked appearance requested by @TobiasMcNulty. When this is present, the data being input will be hidden (like in a password field).
  • :world_map: Read-only geoshape and geotrace questions should not allow edits. See this issue reported by @ahblake
  • :heart_eyes: Beautiful icons have been added to many of the capture buttons (audio, video, picture, date, time, barcode, and more). See this screenshot.

:sparkles: Offline Entities prototype :sparkles:

Offline Entities in this release are considered a prototype because we will be changing the way Entities are stored to improve performance. This means offline Entities will be erased after upgrading to a future Collect version. This prototype does not handle multiple offline updates well yet, especially when there are conflicts across multiple devices/data collectors.

We encourage those using Entities in offline environments to try this prototype with their workflows and let us know how they plan to use it so we can make sure we have considered all common needs.

To try local Entities available in follow-up forms even when offline:

  1. Explicitly tell Collect which Entity List(s) to manage locally. From the Collect main screen, tap the project icon in the upper right corner and then tap the Settings button. Scroll down to Experimental and tap on it, then tap Entities, Add entity list, and type in the name of an Entity List to manage locally. You’ll then need to resynchronize or redownload forms so that the local entity lists can be updated.
  2. Toggle on Enable local entities so that follow-up forms actually use those local Entities.

In future versions of Collect, neither of these steps will be required.

Additional testing
You can find a full list of changes in the release notes:

Thanks to all testers for your help!

10 Likes

This is so good. On implementation;

  • you can't have a purely transparent colour for a polygon fill can you? I left some values blank and they appeared in the default red.
  • is the fill rendered a lighter/ partially transparent version of the colour selected? It appears to be from a quick test but I'm not sure.
  • what are the default values for unstyled polygons?

I need to build out a palette to use now...

Edit: styling only applies to select from map doesn't it? No way to affect view/edit/create geoshape/trace? I have painted myself into a corner with a library of tilesets that have some items shown in red, and overlaying a selected geoshape is now invisible as it's also red and without vertice indicators.

Super, tried this and a read only geoshape has no visible vertices to move anymore as expected.

3 Likes

So good ! Many thanks to the team !

From my own test, if you don't set the fill property, the polygon is filled with the default "red" transparent.

First test here :
I achieve to manage showing visited/to visit places on the same select map :slight_smile:

But not in a dynamic way (I mean not with a test that defines the style to apply). Anyway this is a first workaround and the end user don't care about how it's done :wink:

  • I duplicated the all the json objects with another style.
  • I also add a property call "visited" set to 'true' or 'false'
  • And I use choice filter to show the green version (visited) or the orange version (to visit).

It will be used on the field next week :slight_smile:

The form (in french) :
presence_absence_par_maille_pour_forum.xlsx (451,4 Ko)
The geojson :
mailles_4326_presence_absence.geojson.zip (29,4 Ko)

2 Likes

I forgot we are in beta... so it will be used on the field as soon as 2024.2 will be released.

4 Likes

Second test. I started a form an then closed Collect without saving the form.
When I restart the app and try to fill the same form (or another one), I get this error :

I've just corrected the translation error (précédant -> précédent)

That's currently the case. Our thinking is that red is a more useful default. Is there a case in which you'd really want full transparency?

Correct. We'll make sure to document that clearly.

The stroke and fill are pure red (#ff0000) with opacity of 68% and stroke width 8.

Yes, that's right. Unfortunately we'd have to come up with a different spec to style geoshape/trace fields. I don't have a great workaround for you but will think about it.

Looks great! :tada:

Yes, that would be nice, wouldn't it? It's a concept we're interested in but the spec will be quite different from anything we have now.

I get this error

I think it should be fixed in beta 1! There was a problem with the date string translation.

I've just corrected the translation error (précédant -> précédent)

:pray:

2 Likes

As they are always 'filled' from a user interaction POV (i.e. tapping on edge or interior) perhaps not. pseudo transparency could be achieved by using a very pale fill colour perhaps, that would be then made 68% opacity by Collect.

It would probably be easier if I just regenerated my tiles with not-red details!

Thanks for all the other clarifications!

Edit: Happy birthday @LN :cake: !

1 Like

You could utilise entities to write a new colour to the stroke/fill fields upon completing a record? The beta offline entities functionality would reflect this on device while disconnected, and on other devices once synced. Perhaps it goes from colour A (visited = false) to colour B (visited = true, but data not reviewed/approved) to colour C (visited = true).

1 Like

Aw, thanks! I feel very lucky to know people from around the world -- I get an extra long birthday! :balloon::joy:

Absolutely. I really do hope that one day we can come up with a reasonable way to define the condition-to-color mapping somewhere so that only one property needs to be updated. Ideally we could do this for tables of entities as well. Lots of exciting challenges ahead of us!

1 Like

Oh yes it would be so nice ! :world_map:

An almost unnecessary test of the 'Stroke' style - because others have managed to get it working.

Confirming that I can use this feature from a csv with the column 'stroke'

I have just (this week) trained a whole bunch of staff with the promise that this feature is coming soon! It is REALLY useful to be able to see different lines clearly, and defined by (stroke) colour.

Thanks for implementing this feature - in the current version we just have red lines on the map and lots of head scratching!

I had been hoping the Offline layers loading feature might be included in this release - that same bunch of people can't load their base maps because their devices are locked... But then, there's always something to look forward to next time...

Running on an old Sony Z5 with Android 7.1 - a test device composting in my desk drawer :slight_smile:

3 Likes

It's great to see a screenshot of how you're planning to use the new functionality, thank you!

That's the plan. We wanted to get this first beta out to get initial feedback and catch bugs like the issue @mathieubossaert identified with French dates and will follow up shortly with another beta including the mbtiles selection.

@Grzesiek2010 and @seadowg, I want to make sure you've seen the excitement in this thread! :rocket:

2 Likes

We've just released a new beta that adds the ability to import mbtiles from device!

We had one surprise along the way which is that we can't easily filter by file extension so we can't show only mbtiles files in the file picker. For now, all files can be selected but any that are not mbtiles will be ignored without feedback. We'll work on adding that feedback.

You should be able to add multiple mbtiles at once but the file picker interface to do so will depend on your device. Generally you will need to tap the icon for the file, not the filename.

We haven't added deletion yet and will be working on that next week. If you have any other feedback on the functionality, we'd love to hear it.

3 Likes

Testing this imminently!

Edit: Tested!

I opened a project, went into Settings / Maps, tapped on Layer (toast is mostly hidden at bottom of screen in landscape mode, have to drag up, reported similar UI issue previously). (I also created a submission and opened a geowidget then used the layer picker, process was the same.)

Scrolling to the bottom gave me the 'Add layer' button, and on a Galaxy Tab S8+ / Android 14 / One UI 6.1 the system picker let me navigate to the location (SD card in a USB dock), open the folder, use the button to select all, then tap Select and they were added.

If you don't select a layer to add, you have to use the < button repeatedly to go up to the top level and then the last tap will return to the select toast. I can't see a quicker way to bail out.

Separate to adding layers, after selecting a layer when the list of layers is too long to fit on screen, it jumps to the top, so you have to scroll down again to check that it's selected.

Also, layers now appear to be sorted in order of date added, definitely not A-Z anymore, which makes finding the layer tricky when there are many and they are added at different times.

Added layers can be for project only or all projects. This is a really nice UX improvement, and is handy to quarantine tilesets. Was this function always available (the layers folder under projects\projectID\) - it's not mentioned in the docs for mbtiles?

3 Likes

Hi @ahblake
Thanks for testing and sharing your thoughts!

This is the default android behavior. As you have noticed it works in a similar way in other parts of the app as well. We will consider changing it.

You are describing your experience with the Android file picker I guess? It's the built-in file picker (not our implementation). It might work in a different way depending on your Android version but it's not something we can configure.

Yeah, it shouldn't work like that. We will try to improve it.

Good catch we will fix this.

It has been available for over 2 years now. You are right it looks like it's not described in the docs. We will fix this.

1 Like

What he said!

I've tested this on Android 7 using Shelter App (work profile with a separate Google account) - it is capable of loading tiles from the personal profile folders, which is a bonus :slight_smile:

Being able to order the tiles by date or name would be nice (best not to assume we are using the most recently added, so A-Z might the best default)

Also confirming that the dialogue is accessible and works from Project Settings and within the map layer button of a geo-widget.

This is a fantastic step forward for locked devices. Thank you for your hard work on this, great for it to nearly be at my fingertips!

3 Likes