Thanks, @Ukang_a_Dickson, for leading the way to a v1.0 release of pyxform! I just looked over the project board and I think that if we could finish what is in progress that would be a great release.
The only issue that hasn't been started that I think we may want to consider doing as part of v1.0 is Generate ODK XForms spec-compliant meta block. We've discussed a way of doing it in a backwards-compatible way so it doesn't strictly need to be in a major release but I think it's a big enough change that it would fit well with the release. Of course, there's a question about whether we even consider it important enough to do at all. @martijnr?
@nribeka you have a number of great PRs open. I think that all have some feedback on them for you to address, most very small things. Will you have time to address those in the near future?
Yes, I think so. We should move to following our spec to the letter, and the spec makes sense in that respect. However, fine to do in 1.1 release as well imo.
The version output would be great to get in though (not so much because of 1.0 but because, we'd like to make use of it asap in Enketo).
I just put up a pull request for supporting last-saved at https://github.com/XLSForm/pyxform/pull/417. It's not strictly needed for a v1 release but it's functionality that we've been getting questions about and a specification that we've had approved for a while. I would really appreciate a review on that.
I gave more thought to Generate ODK XForms spec-compliant meta block and unfortunately I don't feel like it's something I can address. In fact, though I wish we could be entirely spec compliant and I filed the issue, I'd be inclined to mark this one as won't fix because the change seems too disruptive to me. Sorry, @martijnr. At the very least, I don't think it should be blocking this release.
The good news is that the version output requested by @martijnr has been merged as have all of the breaking changes we had planned other than the meta block one.
One thing I would like to suggest is that we add a warning about the change of the root node name from the filename to data. This change is going to be a really great one in the long term but in the short term, it will cause problems for form updates because all node paths will have changed (e.g. /my_cool_form/field will now be /data/field). Here is some example warning text:
If this form is an updated version of an existing form, you may get errors when uploading it to your server. Learn how to fix this at http://xlsform.org#set-root-node-name.
I like that it's non-technical and that it's not alarmist. I don't like that it's very vague and happens at the wrong time since a user won't get an error until they try to upload it. In general, this change is very hard to explain to folks who don't know about the underlying form specification. Do others think it's worth trying to anticipate the problems users will run into? I'd like to try to make this transition as painless as possible. An other idea to help the transition is to continue making the old (current) version of XLSForm online available for some time.
Note that servers which embed pyxform and know what version of the tool was used to generate the existing form version can call pyxform with a value for default_name to keep the root node consistent. They could also suppress whatever warning text we choose to add.
There are a lot of changes in this release and moving to v1 is exciting. I'd really like @Ukang_a_Dickson's blessing before the release goes out. When do you think you might be able to do a pass over the current state of master, @Ukang_a_Dickson?
So users will be able to manually set the root node name to fix an update issue? That's great. I think it is fine to have this message in pyxform for some transition time, but I think it is best placed on the servers (Ona, KoBo, Central) when uploading (with the link to xlsform.org root node name override) together with other update checks they may be doing. How about taking it out of pyxform in version 1.1?
Yes! I don't expect that it would be a problem for systems that embed pyxform, though. Those would be able to directly pass in the desired root node name when a form update is requested.
Upon reflection, I suppose this means that the folks who are most likely to be affected are those who use http://opendatakit.org/xlsform/ or XLSForm Offline to do their conversions. If that sounds right to y'all, then maybe those tools can be the ones to warn. Then we wouldn't need pyxform itself to emit any kind of warning.
I'd like to move towards a release by this Thursday. There's a lot of great unreleased functionality that will be very helpful for users and will conclude several roadmap items. I will summarize risks below so that it's hopefully easy for folks to chime in with any concerns. Thank you, @adam.butler for putting pyxform on the agenda for the @TAB call on Wednesday.
Full release notes
#357 Add list_name to generated JSON to get static choice list
Carlos Quiros @qlands (QLands Technology Consultants)
#372 Add support for Java versions that don't have dots in them
This is considered a v1 release because the XML output will change in several ways. We have done our best to verify that core ODK tools and Enketo will be compatible with these changes, including several versions back. However, older versions of core ODK tools will not be compatible with all changes. This has had negative impact on users in the past (@Batkinson, you may want to make note of these changes). Other downstream tools may be affected, particularly by the following:
#289 Do not generate dummy instance content for external instances
This changes the XML structure.
#375 Use data as the default root node name instead of the filename
This change ensures a predictable root node name. This will make it easier to share raw XPath expressions and will mean that renaming files will not change the XML output. It will also ensure shorter XML output when the filename is long. Users that need a specific root name can introduce the name setting in the settings sheet. For example, when making a permissible update to a form previously uploaded to Aggregate, users will need to use the old filename as the root node name. Tools that use pyxform directly to generate updates to forms will likely want to override the root node name.
#385 Add support for dynamic defaults
Values in the default column that include any of mod, div, *, |, +, -, [, ], {, }, (, ) will be treated as dynamic values and will bet used in setvalue actions in the resulting XML. A warning will be output to let users know that not all tools support dynamic defaults in this way. Tools that don't support setvalue will have unpredictable behavior if used with a form with a dynamic default.
#380 Add repeat instance in addition to template to the model
In Collect, the implication is that one first repeat instance will always be displayed before the user is prompted to add another one. The XML will have changed so some tools may see that as a form update. pyxform will issue a warning.
#394 Add pyxform version to XForm output
Every form generated by pyxform will include an odk:generated-by="pyxform vx.x.x" attribute on the root node name. This will be included in submissions meaning that submissions will be slightly larger in size.
#410 Output a string bind type for selects
Tools that look for select or select1 bind types will no longer work as expected.
#417 Add support for last-saved
This is risky because it extends the way that XPath references are expanded from ${} notation. Additionally, unlike all of the other PRs, it has only been reviewed by folks from one organization (myself and @yanokwa).
As I was verifying the release, I noticed another change that I had not explicitly thought about. As described above, the XLSForm filename is no longer used as the root node name. However, it is used as the default form title and form_id but it is no longer validated to ensure that it is a valid XML element name. That means that form_ids with spaces or special characters are now possible.
I have verified that Collect, Central, Aggregate and Enketo can all properly handle form_ids with spaces and special characters. I can't see any limitations on the instance id element described in the ODK XForms spec or in any other documentation so I believe that this is ok but I wanted to flag it in case anyone has concerns.