Proposal: publish a settings key/value standard

Development
1

This proposal is a companion to OpenRosa spec proposal: add optional /client-settings endpoint.

I propose that we publish an official standard for client setting key/value pairs supported across the ecosystem. These would be used for both server- and QR code-based configuration. This would make it possible for various tools to generate configurations for ODK clients and for various ODK clients to ingest those same configurations.

I think it would make sense to use the keys and values that clients have already been using as much as possible. I propose we look at what Collect has, consider additions or modifications from what Enketo and iXForms do, see if there are any inconsistencies we want to right and then publish what we have as the standard. This might not be the standard we would come up with if we were starting from scratch but it’s a pragmatic road to greater interoperability between existing tools.

At the last TSC call, we talked about what settings exist in the ecosystem. @Martijnr described that Enketo does not currently have many client settings but that it could support some of the same ones Collect does (e.g. specifying a map basemap). Because Enketo doesn’t do multiple form management, several of Collect’s settings won’t apply to it. Enketo is a web app, so several of the kinds of things that are configured through settings in Collect are passed to Enketo through network requests.

I’ve summarized Collect’s settings below. It’s a lot. Some of these illustrate the broad range of workflows users leverage Collect within, others exist because Collect behavior changed but a setting was introduced to go back to older defaults if needed and yet others are probably no longer necessary. In going through this exercise, I identified that the documentation is incomplete and I’ll address that. I also plan to add some usage analytics in Collect.

@martijnr, @Xiphware, are there settings that Enketo or iXForms make available that should be added?

protocol: {"odk_default", "google_sheets", "other"}
server_url
username
password
formlist_url - The endpoint name to add to the base server URL. Defaults to /formList.
submission_url - The endpoint name to add to the base server URL. Defaults to /submission.
selected_google_account
google_sheets_url - Fallback Google Sheets submission URL if none is specified in a Google Sheets form def
appTheme: {"light_theme", "dark_theme"}
app_language: BCP 47 language codes. The ones supported by Collect are: {"af", "am", "ar", "bn", "ca", "cs", "de", "en", "es", "et", "fa", "fi", "fr", "hi", "in", "it", "ja", "ka", "km", "ln", "lo_LA", "lt", "mg", "ml", "mr", "ms", "my", "ne_NP", "nl", "no", "pl", "ps", "pt", "ro", "ru", "si", "sl", "so", "sq", "sr", "sv_SE", "sw", "sw_KE", "te", "th_TH", "ti", "tl", "tr", "uk", "ur", "ur_PK", "vi", "zh", "zu"}
font_size: {13, 17, 21, 25, 29}
navigation: {"swipe", "buttons", "swipe_buttons"}
showSplash
splashPath - Absolute path to an image that will be shown at application launch.
basemap_source - {"google", "mapbox", "osm", "usgs", "stamen", "carto"}
google_map_style: {1, 2, 3, 4} - Defaults to 1 (street). Other values are 2 (satellite), 3 (terrain), 4 (hybrid).
mapbox_map_style: {"mapbox://styles/mapbox/light-v10", "mapbox://styles/mapbox/dark-v10", "mapbox://styles/mapbox/satellite-v9", "mapbox://styles/mapbox/satellite-streets-v11", "mapbox://styles/mapbox/outdoors-v11"}
usgs_map_style: {"topographic", "hybrid", "satellite"}
carto: {"positron", "dark_matter"}
reference_layer: Absolute path to mbtiles file to be used as offline layer.
periodic_form_updates_check: {"never", "every_fifteen_minutes", "every_one_hour", "every_six_hours", "every_24_hours"}
automatic_update
hide_old_form_versions
autosend: {"off", "wifi_only", "cellular_only", "wifi_and_cellular"}
delete_send
default_completed - Marks records as finalized by default at the end of a form-filling session.
constraint_behavior: {"on_swipe", "on_finalize"}
high_resolution
image_size: {"original_image_size", "very_small", "small", "medium", "large"}
guidance_hint: {"no", "yes", "collapsed"}
instance_sync
analytics
metadata_username
metadata_phonenumber
metadata_email

Admin settings

admin_pw - the admin password required to view or change admin settings

All other admin settings are boolean flags that toggle functionality on or off:

edit_saved - disable editing saved forms
send_finalized - disable manually sending finalized forms
view_sent - disable viewing sent form
get_blank - disable fetching form definitions manually
delete_saved - disable manually deleting saved form definitions or records
qr_code_scanner - disable QR code configuration

change_server - disable server settings
change_app_theme - disable changing the app theme
change_app_language - disable changing the app language
change_font_size - disable changing font size
change_navigation - disable changing navigation strategy
show_splash_screen - disable changing whether the splash screen is displayed
maps - disable maps settings
periodic_form_updates_check - disable changing form update frequency
automatic_update - disable changing whether new form definitions are updated
hide_old_form_versions - disable changing whether old form versions are displayed
change_autosend - disable changing whether records autosend
delete_after_send - disable changing whether records delete after send
default_to_finalized - disable changing whether records default to finalized
change_constraint_behavior - disable changing constraint behavior
high_resolution - disable changing high resolution video setting
image_size - disable changing image size setting
guidance_hint - disable changing guidance hint setting

instance_form_sync - disable changing whether records imported from disk default to finalized
change_form_metadata - disable changing form metadata settings
analytics - disable changing the analytics setting

moving_backwards - disable any way to move back to questions that have already been visited
access_settings - disable settings access from form entry screen
change_language - disable changing form definition language
jump_to - disable jumping between questions from form entry screen
save_mid- disable manual saves from form entry screen
save_as - disable naming records
mark_as_finalized - disable selecting whether a specific record should be finalized on save

2 Likes
2

Thanks for this overview!

I found some settings in Enketo (Express and Core) that, though they are currently set centrally at the server for all Enketo users of that server, may be worth mentioning:

logo: the brand image, e.g. the Enketo, KoBoToolbox, Ona, OpenClinica logo
disable save as draft: no ability to save as draft, everything is final
validate continuosly & validateContinously: E.g. if question A has a constraint that depends on question B, this mode would re-validate question A as soon as the value for question B changes (instead of just at the end when Submitting)
validate page & validatePage: Determines whether the Next button should trigger validation and block the user from proceeding to next page if validation fails
swipe page & swipePage: determines whether to enable support for swiping to the next and previous page for forms that are divided into pages
clearIrrelevantImmediately: If clearIrrelevantImmediately is set to true or not set at all, Enketo will clear the value of a question as soon as it becomes irrelevant, after loading (so while the user traverses the form). If it is set to false Enketo will leave the values intact (and just hide the question).
themes support: which form themes the client supports
widgets: Array of widgets that are enabled in the client. See possible values here
maps: Array of map layers to show in all geo widgets in MapBox TileJSON (subset) format
text field character limit and textMaxChars: Sets the maximum allowable text field characters with a default of 2000. This settings is meant to match any back-end database limits to avoid records that cannot be submitted because the server does no accept them.
repeat ordinals: Whether to add custom http://enketo.org/xforms namespaced ordinal and last-used-ordinal attributes to repeat nodes in the model.
support > email: The email address users can contact when they experience problems with the app.
language: explicitly set the default form language

and some that can be set per form (via the form launch API)
theme: override form-defined theme
defaults[]: dynamically populate default values
go_to: scroll, flip and focus on a specific question when opening the form (XPath value)

I excluded the settings that likely don't have a use case for a non-web application.

1 Like
3

Thanks very much for these, @martijnr. I have a few follow-up questions and comments.

This is not something Collect currently supports but it could. It feels like the API should make it possible to transmit the logo image, then. OpenRosa spec proposal: add optional /client-settings endpoint currently does not do that.

In Enketo, are finalized forms uneditable? Strangely, in Collect, finalized forms are still editable unless they are encrypted. That aside, I think this is equivalent to a combination of default_completed set to true (the default) and admin/mark_as_finalized set to false.

I think this is the same as Collect's constraint_behavior "on_swipe" vs "on_finalize". It only has to do with constraints, right?

Let's say you have validate page set to true. Having validate continuously set to false would delay validation until changing pages but setting it to true would mean that constraint checking would happen on the fly within that page? In Collect, there is no such option and constraint checking always happens on the fly within a page. I think it would be quite hard to block that but I'd have to think more about it.

This is similar to Collect's navigation. That setting has three options: swipe, buttons and swipe_buttons. It sounds like Enketo's is binary. If swipe is enabled, are buttons/links also displayed? Would you consider having three options?

When you say "clear the value", do you mean the contents of an element or removing the irrelevant elements? Is this just about when the pruning happens or does it change what the submission content is?

This is probably not something that would ever be relevant to Collect.

This is mostly to specify custom widgets, is that right? Again, it doesn't seem relevant to Collect. It seems like custom widgets would inevitably be pretty client-specific so if we're going to put this in the standard, I think it should probably be enketo_widgets. Would this require shipping some files?

Collect currently supports additional layers as a single mbtiles file. Adding support for specifying a tileJSON file should be possible. We'd have to think about how it interacts with the other map settings.

Sigh. This is because of our friend Aggregate. Collect has no such setting and it has to be specified at the form level as documented at https://docs.opendatakit.org/aggregate-field-length/#id1. It could become a Collect setting as well.

This isn't supported by Collect but could/should be. I think it'd be good to make this an addition to the core spec.

This isn't supported by Collect but could be.

Does this also set the Enketo interface language or just the form language? Collect currently does not have such a setting. Form language is selected while filling out a form. I'm not sure whether setting this at the client level for all available forms really makes sense. How are users using this in Enketo, @martijnr? Why do they want to override the form definition default for a whole server?

These feel like they are more about where the form is launched from. That is, I imagine go_to is useful when some other web app launches Enketo forms for specific edits. Similarly, an external app might want to launch Collect and have it open to a certain question. That's useful functionality but doesn't feel like it goes with the others. In Enketo's case, the web app in question is probably the same as the OpenRosa server that would supply settings but in Collect's case I don't think they would be the same.

4

I've gone through and made sure that the Collect settings documentation is complete and up to date. I've then pared the settings list in my original post to just the keys and possible values if they're not self-explanatory.

Once I hear from @martijnr about some of the questions I posed above, I'll take a stab at combining settings from the two clients.

5

Late (as usual...), but here's a few more from a possible future 3rd... :wink:

Some user friendly info about the server (pretty self explanitory):

council_name = "GoMobile City Council"
council_shortname = GoMobile
support_phone = 0211234567
support_email = "Gareth Bestor<root@apple.com>"

This indicates the server is a 'test' server (which may turn on/off certin features in the client for testing purposes):

isTest = true

These are useful to ensure the app client version you are running is compatible with the server. If not you can throw a "Sorry but you must update to take latest version of Foo app" error:

min_supported_version_android = "0.3.0.0;
min_supported_version_ios = 0.9.0.0;
min_supported_version_winphone = 0.0.0.0;

(of course, till Collect is cross-platform two of these are unnecessary... :stuck_out_tongue_winking_eye: )

Branding stuff (logo could be url or possible inline base64 encoded image):

logo_url = "http://www.gomobile.co.nz/wp-content/uploads/2018/08/Logo__Text_2.png"
backgroundcolour = EEEEEE
forgroundcolor = 346BC3

These are probably getting to the more interesting stuff for turning on/of specific functional aspects of the client:

autodeleteaftersubmit = true;
autogeneratepdf = true
showhintsinpdf = true

GoMobile can create and upload a PDF version of the completed form upon submission (its a bit easier to do in the client because its already parsed out all the XForm components, question prompts, etc. So these are for customizing the associated PDF summary 'report'.

So these ones are a bit interesting, and unsual... Basically, in order to extract some common meta data from forms - specifically the final 'status' of the insepction (Pass/Fail/...), the GPS location to show it on a map, whether a reinspection is required - we identify specific 'reserved words' for control labels in the primary instance XML that will be used to populate these values. The controls can be anywhere in the form definition that's convenient, so I use a "/*[local-name()='%@']" XPath expression to find them. But I'm not sure if Collect will ever want to do anything like this....

status = "status"
location = "geopoint"
reinspect = "reinspect"

(I have a few more, but till ODK Collect incorporates a neato scheduler they're pretty irrelevant. Ha :stuck_out_tongue_winking_eye: )

6

You're more likely to find an organization that's a good value fit both for your goals and processes!

7

A URL would work for Enketo (routing the request through Enketo's server), but not sure if this is feasible for Collect.

Indeed, they are not editable. Also if the form has enabled encryption, Save as Draft is always disabled. Yes, it looks like those 2 Collect settings are the equivalent.

Both constraints and requireds. Apart from that, it looks like that Collect setting is equivalent,

In Enketo this this about re-validating question A (and showing an error immediately) if question B changes. I don't think Collect is doing that when testing with this form

Yes, in Enketo buttons are always displayed. Yes, sure three options is fine (when swipe becomes more reliable).

It means clearing the values in the model (and hiding controls, but that's always the case).

Yes, though you could disable default widgets. The latter would be the only possible rationale for including this setting in a common spec, I think. May not be any need for this, and I'd be fine with leaving it out.

Let's forget about this one. After looking it up it's really meant to be used internally as a step towards automatically syncing UI and form language (which we still don't have - but soon I think).

Yes, indeed.

1 Like
8

Forgot to mention that an XML response (instead of JSON) in line with other OpenRosa APIs would have my preference, even though a JSON response is easier to consume.

9

Can you say more about why, please? I like consistency and I like XML. My reasoning is that if we treat this is an optional extension and don't require OpenRosa headers, it's sufficiently "outside" the original API that going with the popular and slightly more compact choice makes sense. I don't feel strongly about it, though.

10

Another motivation for the JSON payload is to match the exiting Collect QR code contents. As @tomsmyth pointed out here, matching the QR code structure would be a win for tools that generate configurations (e.g. servers) because they'd only have to generate one type of settings collection and a win for Collect because it could use the same code paths to ingest settings those two ways. I recognize it's less compelling for clients that won't have QR code configuration.

1 Like
11

Oops, sorry I posted that comment in the wrong thread. Consistency with formList, xformsManifest was my reason. However, matching QR contents makes sense and I don't feel very strongly about using XML. Thanks for clarifying.