Yes, sorry, I realize that it's confusing.
This is the best way that we can honor our users by resolving a pain point
in a timely and safe way. The proposed JR change is scary and potentially
damaging; doing that will require careful safeguards, attention to the
migration path for existing users, and, all told, take much longer. Given
that (a) we will want to make efforts to be backward-compatible anyway and
(b) we would only be introducing potential problems for a set of users who
are already broken (those who want or need to be able to filter based on a
within-context repeat reference), the benefits seem well worth the costs.
I find it equally perplexing, by the way, that you would put so little
value on Enketo's ability to easily match ODK Collect's legacy behavior,
however flawed it may be. The explanation, in economic terms, is that we
have very different welfare or utility functions. We both think that we're
right -- and we may well both be right, given our perspectives.
For ODK Collect, there are a few different paths forward. And we're happy
to support any of them. But for SurveyCTO users, we're going to resolve the
pain point in the best way we know how today, then work in a reasonable way
toward a better JR. Hope that makes at least a little bit of sense.
Best,
Chris
···
On Fri, Dec 4, 2015 at 11:39 AM Martijn van de Rijdt wrote:I am confused. Sorry if I missed this plan in the previous messages. Why
would you go ahead with adding the pyxform feature for the buggy predicate
behaviour? Right now we have almost no forms out there that rely on the
bug, i.e. almost no forms with the incorrect syntax. Why would we
create them?Wouldn't it be better to add your highly-desirable feature only to
forms that have the current_always_field (or whatever it's called) flag
set? For the community-version of pyxform, I would advocate to do it like
that.On Thu, Dec 3, 2015 at 11:46 PM, Christopher Robert <crobert@surveycto.com wrote:
FYI, I've just found a way to have pyxform only use current() when
replacing ${x} references that (a) are within choice_filter expressions and
(b) point to groups or fields within repeat groups. We're going to do a
bunch of testing to see how robust that change is (in terms of not breaking
any other cases and in terms of not imposing any performance costs). If it
looks solid, then it will become a candidate for inclusion in our 2.01
release.That limits possible damage to cases that were already broken, and shifts
them to function based on the legacy JR support for current(). Our few
Enketo users won't benefit from the new ability to reference repeated
values in choice_filter expressions, but I think that's okay because
they're not losing anything. And any future fix to JR's current() behavior
will just need to be backward-compatible as discussed, based on flags in
the form model.We'll be happy to share the pyxform changes with the community, if the
broader community would like to have this option to emit functional
choice_filter expressions for repeated fields. And the pyxform is easily
changed to emit XPaths compatible with the future-desired current()
behavior (though obviously the difficulty is in the backward compatibility,
as discussed at length).Best,
Chris
On Fri, Dec 4, 2015 at 12:20 AM Christopher Robert crobert@surveycto.com wrote:
Thanks, Mitch. Some great ideas in there.
Best,
Chris
On Thu, Dec 3, 2015 at 12:52 PM Mitch Sundt mitchellsundt@gmail.com wrote:
It isn't that dire.
Not only do you need to then update 100% of your devices with a high
degree of certainty, but you also need to re-generate the XML for every
form on your server. And not even that is enough. Then you need to update
every form definition on every device.=================
Individual Device SafeguardsYes, all the devices need to be updated. And yes, this is the biggest
pain point. We CAN change the form download task so that it first scans
the form to detect unsupported revision tags, and refuses to install the
form definition on the device. And then throws up a Dialog warning the user
of the inappropriate form and prodding them to update ODK Collect. This
would preserve the older version of that individual form on that one
device. Survey CTO already added functionality to the download task to
ensure that all form attachments were downloaded before the form was copied
into the /sdcard/odk/forms directory and became 'live'. So this would be an
extra validity check before that wholesale move, allowing your non-upgraded
devices to stay 'in the past' w.r.t. form definitions they already have
until you upgrade the device.This prevents catastrophic failure of an individual device, but does
not ensure complete migration of all the devices to the latest ODK Collect.================
Device Fleet ManagementODK Aggregate, at least, does not have any mechanism to track all
devices and what their ODK Collect versions are. This is something that ODK
Manage was intended to do, but that never got implemented.I haven't looked, but there are undoubtedly 3rd party tools that can
manage fleets of devices and communicate their configurations back to a
central location, and force-push updates onto individual devices. All the
anti-theft phone-lockout tools are basically doing that.================
Form Revision ManagementYou do not need to upgrade all the forms at once.
Because the revision flags are in each form, you can keep running the
forms that require the bugs.
Even as you update your XLSForm, server and ODK Collect.Depending upon how end-to-end your solution is, when running XLSForm to
update an existing form_id, you could access your live data collection
server to retrieve the existing form_id, and then, by default, apply the
same revision tags when making updates to that form_id. And only change
those revision tags through explicit user action.That would allow users to easily edit the form to add another language
or fix a typo without much thought. It could also be expanded to run the
verification of identical data-models at that time (rather than later when
you try to upload the revision to the server) and to confirm that the form
version is appropriately larger than the existing form.On Wed, Dec 2, 2015 at 4:37 PM, Christopher Robert < crobert@surveycto.com> wrote:
Thanks all, I appreciate all of the thinking going into this.
At the risk of thoroughly exhausting everybody's patience, I'm still
not sure that we have a strategy that can be described as reasonable from
an existing user's point of view. Let me explain why.Say we release SurveyCTO 2.10 with three changes:
We add the fix to JR and have it apply whenever the appropriate
flag is set in the XForm model.We default all pyxform generation (which is basically invisible to
the user in SurveyCTO) to include the appropriate flag so that
newly-generated XML will be interpreted with the fixed JR.We default pyxform to output XPaths in choice-filter expressions
using current() so that within-context repeat references finally work. This
code, by default, relies on the fix (so paths are relative to the field,
not the group). (It also changes ALL XPath expressions, not just those
referencing within-repeat values, because we're not clever enough to be
able to distinguish the different cases in that python code.)Now say you're an upgrading user and you upgraded because SurveyCTO
2.10 also included a beautiful and astonishingly powerful web-based form
designer. You upgraded to get this new designer and you love it and now you
upload a new or revised form. That form happens to have a choice-filter
expression. Now here's where things get ugly...That form won't work on your SurveyCTO 2.01, 2.00, 1.402, etc.
devices. Probably there's some nasty XPath parsing error when you get to
where the choice filter kicks in, maybe it even crashes your Collect. That
form might have worked the day before, and maybe all you changed was fixing
a typo in a label, but now suddenly it doesn't work on all of these devices.So here's where we say, okay, the user has two choices: (1) upgrade
all devices to the 2.10 Collect, or (2) find and set the "stay in the past"
option. The second option is good in principle -- because it provides a
kind of quick save -- but then what does it take to ever migrate away from
that "stay in the past" setting? Not only do you need to then update 100%
of your devices with a high degree of certainty, but you also need to
re-generate the XML for every form on your server. And not even that is
enough. Then you need to update every form definition on every device.And if you miss a few devices and leave them on the old version, or
miss a few form definitions here or there? Cryptic errors, inability to
finalize forms, confusion, anger, frantic calls to support, etc., etc.You don't need to be a human-centered design expert to see that this
is kind of a disaster from a user-experience point of view. And, for us,
this would be multiplied across hundreds of migrating users, essentially
eating a lot of our 2016 and causing a tremendous amount of ill will.The only upgrade path for existing users that makes any sense is one
where we basically emit the "stay in the past" flag always for everyone.
You can all turn your noses up at the idea of indefinite reliance on what
was initially a bug, and I get it. But at least here at SurveyCTO, we're
not going to put our users through the kind of migration path described
above.If relying on the bug is just too unpalatable, then ultimately I don't
think that we can change pyxform to emit XPaths using current(). It's just
too fraught and we don't have a good way to manage the upgrade path without
torturing existing users. So fine, our awesome pyxform fix to magically
make within-repeat references start working is just not destined to see the
light of day. Rather, we need to make pyxform smart enough to only use
current() for paths to repeated values (which never worked in old JR's
anyway). Or we need to just provide a new solution for referencing repeated
values without current() (perhaps a new custom JR function, which everybody
loves).And in that case, sure, I'm much more relaxed about the strategies for
upgrading users since very few are likely to be using current(). I won't
want to overly torture those users, but I won't lose a ton of sleep over
them since even users who use current() now are unlikely to use it in a lot
of forms.I'll see if we can figure out how to make the pyxform change in such a
way that it only changes the XPaths for fields within repeat groups. But I
might also see if we can instead find a totally different way to solve or
work around this problem. Or, I'll see how I feel about pushing this off
for another year. Fixing choice filters to be able to work will within
repeat groups was not meant to be a huge project.Thanks again,
Chris
On Wed, Dec 2, 2015 at 2:48 PM Mitch Sundt mitchellsundt@gmail.com wrote:
w.r.t. moving forward...
Even if this particular change impacts a very small number of forms,
we should figure out how to implement non-backward-compatible changes so
that they are safe moving forward. We have no visibility into who uses ODK
Collect and ODK Aggregate and no reliable way to notify them of changes.This could be a huge deal when or if we ever revise the JR engine to
properly handle timezones, dates and times, and date and time arithmetic.Or if we find a need to change XLSForm to emit XForms features or
XPath functions that currently don't work (e.g., to use [1] or position(),
or XForms events, for example).Note that these non-backward-compatible changes extend to not just
XLSForm and ODK Collect, but also to servers. You can use the same revision
tag to indicate new or different processing to be taken on the server.=============
Managing this change requires:I believe that this requires:
(1) the XML form definition is the place to define the
feature-enabling-labels that the form requires of the device and server
processing steps.(2) when the JR engine or any tool implements non-backward-compatible
changes, it only activates them when it detects the presence of those
feature-enabling-label in the XML form definition. Omitting that
feature-enabling-label is a directive to "stay in the past".(3) ODK Collect and other tools processing the XML form definition or
the XML submission documents should warn the user (e.g., pop up a Toast) if
they are running a version that does not recognize the
feature-enabling-labels in the form's revision tag (i.e., they are given a
form that indicates an advanced feature that they are unaware of) -- this
notification informs the user that they should upgrade that tool to
correctly process the directive. Alternatively, the tool could reject the
form, Force Close or abort, which is the current behavior when ODK
Aggregate or ODK Collect gets an XForms definition that uses a new,
unrecognized, function, etc. ;-<=============
The stable upgrade path is to:(1) update your form distribution and data aggregation server(s),
(2) update your collection devices (or collection processing server
-- e.g., Enketo), and
(3) only then update the form generation tool to emit forms with the
non-backward-compatible change.=============
Specifying the Revision settings:I believe the XLS file should not contain a setting to select or
specify the revision.I believe this should be specified at the time the XLSForm converter
is executed on the source XLS file.
I.e., the web UI for these tools should have a spinner or other UI
element that enables users to choose to "stay in the past"This forces users to actively stay in the past.
The ODK site will probably achieve this with a drop-down that allows
the user to specify, e.g.,"full current functionality"
"generate a form definition compatible with ODK Collect 1.4.7 and
earlier"To allow all groups to independently advance the functionality within
their tool families, the combination of features that each of these
drop-down choices corresponds to should be documented by those groups.I.e., I see a future where a form generated on Ona for Enketo may not
be compatible with stock ODK Collect unless an appropriate compatibility
drop-down is selected.This could allow, for example, Survey CTO to implement and recoup
costs for JR date-time fixes or other XForms extensions within their
ecosystem before releasing that fix back to the wider community.I do not see any value in offering a user the choice of which
features are enabled or disabled. I suppose some sites could offer that,
but it seems overly complex. Users really just need a way to specify
compatibility to whatever older version of the tools they are using.==============
XLSForm functionalityI am fine with the XLSForm generator taking a comma-separated list of
features to enable, and passing that list through via the revision tag to
the the XML form definition.Upon reflection, I think this is the only way to manage multiple
independent parallel contributions to the various codebases. It allows
different groups to add and enable different functionalities at different
rates.We just need to maintain a global list of all of the feature labels
that every group is using so that we don't choose names that collide.And, some of these feature labels may simply pass through the XLSForm
generator and on to the downstream tools. e.g., date-time fixes might not
have any impact on XLSForm, but would alter how the JR engine processed the
form.On Wed, Dec 2, 2015 at 10:25 AM, Martijn van de Rijdt < martijn@enketo.org> wrote:
Ok, I was just wondering if it's really worth the effort. KoBo found
3 forms using current(). Not sure how many are actively used still. For
them it would seem to me easier to just contact that/those user(s)
individually to come up with a plan.I can live with the XForm attribute and XLSForm setting addition.
Chris, to respond to your question about adding support for the bug
to Enketo. I don't currently see a reason to do so, as no Enketo user has
ever relied on the bug and I highly doubt any will in the future. However,
I don't think this should be relevant to this discussion. We're all in
agreement that we want the community to move forward with the correct
syntax.On Tuesday, December 1, 2015 at 4:58:45 PM UTC-7, Christopher Robert wrote:
We have individual users with 500+ devices in the field. Updating
devices -- or even keeping track of which devices are on which versions --
is a major undertaking. In total, we're talking many thousands of devices
spread across many countries. The problem is not a small one, and our
commitment to backward compatibility is one that our users not only
appreciate but expect.ODK/SurveyCTO Collect continues to use older versions of forms to
view, edit, or finalize submissions begun under those older versions, even
when newer versions exist. So the backward-compatibility challenge is a
formidable one. If you change the way JavaRosa interprets expressions in
the kind of blanket way that you suggest, then users will be faced with the
impossible choice of running software that fails to function properly with
existing, in-progress, or old submissions, or running software that fails
to function properly with newer submissions. Multiply the difficulty of the
choice and the potential confusion of XPath parsing errors times
however-many devices there are in the field.No, this is a legitimately difficult situation that requires a
tremendous amount of care. Changing how XPath expressions are interpreted
is a big deal for the Collect world, though I can see why it would be
considerably less so in the Enketo world.Now, I will admit that there are bound to be very few users who
already use current() in choice filters. Mitch suggested the approach a few
times in the public forum, but even I failed to get it to work the first
several times I tried to follow his advice. So chances are that not many
projects are using current() today. (I know of 1-2, maybe there are 8-10
total. That's just a guess.)Enketo aside, we could -- today! -- allow people to reference
in-context repeated values by exploiting the current JR support for
current(). For some users/projects, there would be an immediate benefit.
But obviously that relies on "the bug" and so it would expand the potential
need for backward compatibility. And it would break choice-filter
compatibility between Collect and Enketo (say, for SurveyCTO users at
least).It's easy to say "just fix the bug and then this problem goes
away," but the fact is that we don't quite know how to fix it without
breaking other things; we have a candidate for a fix, but it's seriously
scary. It may take time. And meanwhile, we have already gone multiple years
with users waiting to be able to reference within-context repeated values
in choice filters. If a safe fix drags on for another year or two -- either
because it's just too hard or because nobody quite has the time -- is that
just the way it goes? I don't know. It's not obvious to me that breaking
compatibility with Enketo and relying on current behavior isn't the
welfare-maximizing choice.In the end, I believe that Mitch's suggestion re: compatibility is
maybe necessary (storing in the XML some indication of how current() in
choice filters should be interpreted). And if we totally screwed up the fix
and broke a bunch of use cases, at least it might offer some escape for
affected users (if we allowed an override on the settings sheet).So no, I don't agree that it's as simple as "fix the bug and make
people upgrade."Chris
On Tue, Dec 1, 2015 at 5:11 PM Martijn van de Rijdt < mar...@enketo.org> wrote:
I can't help thinking that we're overthinking this
backwards-compatibility issue.Ideally we have old clients that have the bug and new clients that
do not have the bug. Isn't that the case with any other major bug fix? If
an old form stops working in the new version of ODK Collect, the survey
owner will either have to upload a new version of the form or the user will
have to downgrade ODK Collect. Surely this will be a problem for some
users, but for how many? A small fraction of one percent seems like a
reasonable guess.Maybe the backwards-compatibility should be addressed just by
warning users, as Yaw suggested way above.Any 'side-effects' of fixing the bug, like with creating itemsets,
can be tested for, I presume, before deploying the fix.One way to detect at least a significant portion of the current
current() users, is to do a regex search for any usage of the
'current()/path/to/node' where current()/ is not followed by '../'. That
usage is specific to this bug to refer to a sibling node and will not occur
any more after the bug is fixed (because a question cannot have children).On Mon, Nov 30, 2015 at 11:43 AM, Christopher Robert < cro...@surveycto.com> wrote:
I wasn't going to suggest adding the different parameters to any
of the XLSForm templates -- so they wouldn't become cluttered with this
stuff. Really, it would just be an emergency option for somebody requiring
different compatibility for some component that was out of sync with their
other components. Ideally, nobody would actually have to use the settings
sheet to override any of these options.To be specific, my proposal would be that everybody
support current_always_field=false today, and that this be how pyxform
today defaults. Ideally, this would include Enketo, as Collect and
SurveyCTO's Web Collect (and anything else using the ODK JavaRosa) already
support current_always_field=false as their current behavior. Then, we
figure out how to get current_always_field=true to work in JavaRosa and
phase that in, eventually changing the pyxform default to that once we
think that enough components support it.We have thousands of devices out there using older Collect
versions, even well after their servers have been updated... so my guess is
that it's some of those users who will end up having to use the
settings-sheet option to force current_always_field=false after we've
changed the default to true.Does all of that make sense?
Chris
On Sun, Nov 29, 2015 at 6:48 PM Yaw Anokwa yan...@nafundi.com wrote:
All this sounds fine to me, but wanted to raise some potential
issues.
Discrete parameters also means folks can pick and choose what
behavior
they want and it's hard to move the entire community forward
that way.
Also, once you get past two or three, you end up with a settings
sheet
of lots of flags to toggle and that isn't a great user
experience.Yaw
Need ODK consultants? Nafundi provides form design, server setup,
in-field training, and software development for ODK. Go to
https://nafundi.com to get started.On Sun, Nov 29, 2015 at 6:47 PM, Christopher Robert cro...@surveycto.com wrote:
Having a number of discrete parameters that are overridable on
the settings
sheet makes sense to me (like "current_references_parent" or
perhaps
"current_always_field"). We could then add model parameters
and change the
pyxform default over time, but users could always override on
their settings
sheets as needed for specific compatibility.For example, our SurveyCTO pyxform for 2.10 (we've given up on
2.01 now)
could default to current_always_field=false, which would mean
that JR would
be backward-compatible and pyxform would emit relative paths
for the
choice_filter column assuming that current() references the
parent. This
would break all choice_filters in Enketo, but our Enketo users
could set
current_always_field=true in their settings sheets.For that to work, though, we'd also have to add support for
current_always_field=true in JR. We could do that as Meletis
has proposed,
which may have other side effects -- we'll have to see (e.g.,
for itemset
copying).Alternatively, we could try to convince Martijn to support
current_always_field=false, which is likely trivial. If he
would do that,
then we could push JR's support for current_always_field=true
to when we
could figure out how to do it safely. Also, it would mean
producing a
version of Enketo that supports relative paths in choice
filters in a manner
compatible with ODK Collect. Ona, Kobo, and other Enketo users
could
rejoice, able to produce forms that use current() (like those
that reference
same-repeat values) in their choice_filters. That boon to
users has to be
worth something, even if a principle is slightly violated
(supporting
something outside the XPath spec).Chris
On Tue, Nov 24, 2015 at 8:43 PM Alex Dorey dore...@gmail.com wrote:
Hi, Just catching up on the discussion. Sorry if I'm a bit
behind in my
understanding of things.Regarding the pyxform changes--
- Instead of the date, could we have the configuration
parameter describe
the specific change?So instead of:
revision="2015-11-24"
we specify something like:
current_references_parent=True (default to False with the
next pyxform
version bump)
- Could we put this configuration into the xlsform's
settings sheet? I
think this would be easier to test and configure. We could
also add a way to
override the setting with an extra argument to the CLI or
xls2json call.On Tuesday, November 24, 2015 at 3:26:18 PM UTC-5, Mitch wrote:
I agree. Having just delved into the itemset code, there are
a number of
issues that would need to be cleaned up.What I would propose is to leave JR / ODK Collect broken
as-is. This
will be future work.The python code will then be the focus of the tweaks.
(1) change:
json_survey =
xls2json.workbook_to_json(json.loads(request.POST['workbookJson']),
form_name=form_name, warnings=warningsList)
to add an optional revision argument:
json_survey =
xls2json.workbook_to_json(json.loads(request.POST['workbookJson']),
form_name=form_name, warnings=warningsList,
revision='2015-11-24')(2) change that implementation to:
(2a) store the revision string somewhere in the returned
json object.(2b) expand ${fieldname} with the new relative path
expansion only if the
passed-in revision is >= '2015-11-24' ; otherwise, use the
absolute path
expansion.(3) change:
create_survey_element_from_dict()
to emit the revision value as an attribute on the
element.===============
The JR itemset bug will be fixed at a later date.When that is fixed, we can introduce a new revision string
for that
behavior change.i.e., once we have the revision string moving through the
system, we can
begin to support non-backward-compatible changes to
behaviors in XLSForm and
JR form processing.For the time being, the ODK XLSForm converter will not
supply a revision
argument, and not make use of this pyxform enhancement. Once
the JR fix is
done, we will then support that new revision and the current
behavior,
defaulting to the new behavior, but supporting users in
generating the old
behavior.Mitch
On Tue, Nov 24, 2015 at 8:44 AM, Martijn van de Rijdt < mar...@enketo.org> wrote:
I appreciate the creative solution. It does sound like it
will become a
bit of a headache to manage and explain to users though.
Whether it's worth
it depends on how many active forms are out there that rely
on the bug. We
should get Ona's and KoBo's input on this too, I think.From Enketo's side, relative paths were never supported in
choice-filters (to my embarrassment and surprise - which
led to my
question). As a result, there is no significant issue with
backwards
compatibility in Enketo once it is fixed in JR. Not sure
how meaningful this
is to ODK Collect, but no Enketo user has ever reported
this lack of
support. I think to date it's been too hard to craft such
choice-filters.
Because of the way Enketo is integrated into KoBo, Ona,
Formhub, I'd be
surprised if there were a significant amount of those users
using relative
choice-filters.I do indeed feel strongly about following the XForms 2.0
spec here. I
think relative path support is going to be very useful in
the future, and I
agree that we need to look at the bigger picture and do
this correctly, so
it won't catch up with us.On Tue, Nov 24, 2015 at 4:24 AM, Christopher Robert cro...@surveycto.com wrote:
Okay, well, this is turning into a bigger and bigger
project and we may
well miss the window for our 2.01 maintenance release
(and/or make the
changes so extensive and scary that we cannot in good
conscience include
them in a maintenance release).However, your idea for making JR sensitive to the revision
value
emitted by pyxform opens a new possibility: we could
release the pyxform
change with 2.01 so that users can begin to benefit from
the relative paths
(relying on today's current() behavior), then assume that
when JR is "fixed"
it will only be fixed for XML emitted by newer pyxforms
that would have
updated their paths to account for the fix (using your
revision-value idea).
The primary drawback to this approach is that we would
basically break all
cascading-select filter expressions for SurveyCTO+Enketo
users (since
Martijn clearly wants to implement the "fixed" version of
current() rather
than the JavaRosa-compatible version of current()).Meletis also found another issue relating to the
evaluation context in
jr:choice-name() (
https://github.com/opendatakit/opendatakit/issues/1180),
so we have that to contend with as well. If we can't
safely resolve the
context for jr:choice-name(), then that would present
another major drawback
and further lessen our enthusiasm for this entire
adventure.Best,
Chris
On Mon, Nov 23, 2015 at 5:45 PM Mitch Sundt < mitche...@gmail.com> wrote:
And, if it isn't clear, to preserve backward-compatible
operations,
XLSForm should not expand ${fieldname} with a relative
path unless the
revision value is the newer one; if the revision is not
specified, it would
emit absolute paths like it currently does.I also switched to use a date value for the revision value
("2015-11-23") which allows simple lexical comparisons.On Mon, Nov 23, 2015 at 1:03 PM, Mitch Sundt < mitche...@gmail.com> wrote:
w.r.t. backward compatibility
I too am concerned.
A proposed solution:
- Add a command line argument or other mechanism to
request that
XLSForm inject a 'revision="2015-11-23"' attribute into
the element.
e.g.,This is used to alter the behavior of the XForms engine
to
accommodate backward-compatible bug behaviors.We can't identify the bugs (e.g,
'legacy="itemset-parent"'), since
that would break older forms run on the new ODK Collect
releases.
We have to advance the revision code as bugs are fixed.I think an integer value is fine for this.
I don't anticipate making a lot of revisions, and I
think the
revisions would be sequentially applied (you can't
pick-and-choose what bugs
you include).In the future, this would allow us to re-do the
date-time behavior.
Modify the XLSForm tools (e.g., the online form
generator) to
enable the user to select what revision they want to
emit (much like we did
last year with the JR form eval choice setting within
ODK Collect). We can
default to emitting the latest revision if the user does
not make a choice.Augment JR to look for this revision attribute and
store/restore
it in the FormDef binary object.Based upon this value, alter the behavior of the JR
itemset
evaluation to fix the path bug.
- Modify ODK Validate to scan for syntax that may be
impacted by a
revision (to advise whether to change something). Not
sure how complex that
might be. May not be always possible?
w.r.t. impacts of change
Thank you Meletis for researching when this change
occurred.I will have to ponder what effect the parent/node change
would have
on itemset copying. Hadn't thought that there was any
relationship between
the two, but perhaps there is.Yaw also reported that the Cascading Select itemsets on
opendatakit.appspot.com are not working with the 1.4.7
rev 1053 release (and
they work but give odd 'null' resolutions on 1057). This
is in a related
area of this same code, so I will hopefully gain a
better understanding of
dynamic itemsets as I uncover the causes of that problem.
Mitch
On Mon, Nov 23, 2015 at 5:59 AM, Christopher Robert cro...@surveycto.com wrote:
So just to be clear, the "fix" we're testing here
reverts the entire
context within that populateDynamicChoices() function,
to be the field
instead of the parent -- so the effects are potentially
much wider than just
what current() returns. Potentially anything and
everything that depends on
context can change now, since the context will shift
from parent to field.This means that we revert a change from 5 years ago and
alter the
context from now on. Neither Meletis nor I feel
terribly confident that we
can entirely predict the consequences to backward
compatibility. We're
testing the things we can think to test, but honestly
it's XML users in the
broader ODK community who face the greatest risks here;
SurveyCTO users by
and large use XLSForm and thus the potential XPath
syntax consequences are
sharply limited.Are you guys very sure that you want to take this path?
I can tell
you now that if we make this change and then there is
some outcry after the
next ODK Collect release, SurveyCTO Collect will very
likely stick with the
new change (since we will have changed pyxform and
we'll then have too many
backward-compatibility concerns to then reverse the
change).Thanks,
Chris
On Mon, Nov 23, 2015 at 5:23 AM mel...@surveycto.com wrote:
Just for the history's sake, I found the time where
the behavior
changed. It was at March 11, 2010:Take a look only in the changes of the
"populateDynamicChoices"
method. The context reference used to be the "current"
field, but after the
changes above it started be the field's parent (by
adding getParent()).
Perhaps there was a reason for that change, but
perhaps not, but I think all
of us agree that this should be fixed, even after 5
and a half years.Meletis
On Monday, November 23, 2015 at 12:08:47 PM UTC+2, mel...@surveycto.com wrote:
Hi Mitch,
Please take a look at the code comment in line 31
here:It clearly says: "ref of the control parent
(group/formdef) of
itemset question"So, for some reason, they wanted it to point to the
parent. I also
see that when the dynamic choices are populated, the
evaluation context is
wrapped around that parent (contextRef object),
that's why current() refers
to the parent group when evaluating itemsets nodesets.I will run a few tests by changing the contextRef
field value, I
just wanted to mention that in case you remember a
reason for that code
comment.Meletis
On Friday, November 20, 2015 at 8:49:12 PM UTC+2, Mitch wrote:
Yes, I think it should be a one-line change; just
don't know
where.Mitch
(I do jar-updates over Christmas / New-Years.
Tedious, but
doesn't require a whole lot of attention.)On Fri, Nov 20, 2015 at 10:36 AM, < mel...@surveycto.com> wrote:
Hi Mitch,
Thank you for all the details, I guess I was
missing some of the
context of the original issue.We will give it a shot to see if we can come up
with a few ideas
how to fix the evaluation content of the itemset
nodeset. I just hope it
will not be like last year with the "form
processing logic" in JR, otherwise
I believe that every Christmas will be a JR
Christmas.Meletis
On Friday, November 20, 2015 at 7:29:14 PM UTC+2, Mitch wrote:
The form was attached in one of my earlier replies.
On Fri, Nov 20, 2015 at 9:27 AM, Mitch Sundt mitche...@gmail.com wrote:
And you see this in constraint expressions where
you write:. > 6
I.e., current()/. should refer to the field, and
not the group
containing the field, as it currently does when
evaluating expressions in
the itemset nodeset.On Fri, Nov 20, 2015 at 9:26 AM, Mitch Sundt mitche...@gmail.com wrote:
The issue is that there is a discrepancy in the
meaning of
'.' across the different contexts of: calculate,
relevant, , and
these itemset nodeset expressions.In other usages, you need to reference this
other field with
current()/../filterfield ; but in the nodeset
expression, you would
reference it with current()/./filterfieldI.e., the notion of '.' is different when used
within a
nodeset expression and when used within a
constraint expression or an
field.See the attached NBiggestOfSet form. For the
label, it has:<select1ref="/NBiggestOfSet/education/edu_level">
<label>What is <outputvalue="current()/../referred_attendee_name"/>'s
level of education?
...For a constraint, it has:
<bindcalculate="/NBiggestOfSet/attendee/name[position(..)=int(current()/../referred_attendee)]"
nodeset="/NBiggestOfSet/education/referred_attendee_name" readonly="true()"
type="string"/>
These each use the current()/../ to reference
the group (
/NBiggesetOfSet/education ).But, within the nodeset definition (if this form
used one),
you would need to use
current()/./referred_attendee_name to reference a
field in that group.On Fri, Nov 20, 2015 at 8:22 AM, < mel...@surveycto.com> wrote:
Hi Mitch,
I think I'm missing something but I think that
nothing is
broken and that the only issue is that the
expression
current()/../filterfield in your example above
should be just changed to
current()/filterfield. If you make that change,
everything should work
because it will be resolved to:/data/group1/filterfield
Why should a "../" be present there? We have
used
current()/somefield successfully in one of our
test forms and it works fine.
And I think it would also work great in your
example. Wouldn't it? Or am I
totally out of context?Meletis
On Friday, November 20, 2015 at 5:35:16 PM UTC+2, Mitch wrote:
Hi Meletis,
That's not the issue --
The issue is that if you use a filter
condition in the
&g--
Revolutionizing data collection since 2012.Enketo https://enketo.org/ | LinkedIn
http://www.linkedin.com/company/enketo-llc | GitHub
https://github.com/enketo | Twitter
https://twitter.com/enketo | Blog http://blog.enketo.org/--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
You received this message because you are subscribed to the Google
Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
You received this message because you are subscribed to a topic in the
Google Groups "ODK Developers" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit-developers/nN8HTA5pUIM/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
opendatakit-developers+unsubscribe@googlegroups.com.For more options, visit https://groups.google.com/d/optout.
--
Revolutionizing data collection since 2012.Enketo https://enketo.org/ | LinkedIn
http://www.linkedin.com/company/enketo-llc | GitHub
https://github.com/enketo | Twitter https://twitter.com/enketo
| Blog http://blog.enketo.org/--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
