Referencing prior variables using current group ID

ODK developers,

We are running up against a limitation that has been periodically raised in
this group (but always left unresolved): we would like to reference prior
field values conditional on the current iteration number of a repeat group.
For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

*1. ODK Validate won't validate these position() calls. *The "return
position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
seemingly because the repeat group's nodeset doesn't have any nodes in it
when called in the context of ODK Validate. This obviously has to do with
how ODK Validate loads and walks through a form, and I'm perplexed as to
what's going on.

*2. You can't use the current position to reference a value in a prior
repeat group. *For example, you can use
"/formid/repeatgroup1[position()=3]/name" to reference the name field in
the third iteration of repeatgroup1, and you can use
"position(/formid/repeatgroup2)" to reference the current position when
inside repeatgroup2 -- but
"/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
not work from inside repeatgroup2. When in the midst of indexing
repeatgroup1, the context doesn't seem suitable for getting your position
in repeatgroup2. If, instead, you stash the current repeatgroup2 position
into a calculated "repeatgroup2/pos" field, you can use that field within
the group -- but unfortunately not as part of indexing repeatgroup1; in the
indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
you want to reference. So, at the moment, we seem tantalizingly close to
being able to reference a prior repeat group from within a current repeat
group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

ยทยทยท On Fri, May 3, 2013 at 10:14 AM, Christopher Robert wrote:

ODK developers,

We are running up against a limitation that has been periodically raised
in this group (but always left unresolved): we would like to reference
prior field values conditional on the current iteration number of a repeat
group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

Great stuff Chris - this is a feature that I am sure will be useful to a
lot of people. I have been trying to replicate the two position examples
you gave i.e. full Xpath and (..), to express in a note field e.g. "You are
entering person number 2 details" (where 2 is the iteration number of the
current repeat group). Unfortunately I can't get the note to show the
iteration number. Perhaps you have a simple form that you can share to show
us how you achieved this?

Cheers
Dan

I have traced the ODK Validate problem, and have filed issue 822 (
https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

ยทยทยท On Mon, May 6, 2013 at 3:23 PM, Christopher Robert wrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

*1. ODK Validate won't validate these position() calls. *The "return
position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
seemingly because the repeat group's nodeset doesn't have any nodes in it
when called in the context of ODK Validate. This obviously has to do with
how ODK Validate loads and walks through a form, and I'm perplexed as to
what's going on.

*2. You can't use the current position to reference a value in a prior
repeat group. *For example, you can use
"/formid/repeatgroup1[position()=3]/name" to reference the name field in
the third iteration of repeatgroup1, and you can use
"position(/formid/repeatgroup2)" to reference the current position when
inside repeatgroup2 -- but
"/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
not work from inside repeatgroup2. When in the midst of indexing
repeatgroup1, the context doesn't seem suitable for getting your position
in repeatgroup2. If, instead, you stash the current repeatgroup2 position
into a calculated "repeatgroup2/pos" field, you can use that field within
the group -- but unfortunately not as part of indexing repeatgroup1; in the
indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
you want to reference. So, at the moment, we seem tantalizingly close to
being able to reference a prior repeat group from within a current repeat
group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert <crobert@surveycto.com wrote:

ODK developers,

We are running up against a limitation that has been periodically raised
in this group (but always left unresolved): we would like to reference
prior field values conditional on the current iteration number of a repeat
group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

Hi Dan,

I use a calculate field, within each group, to calculate the position. Then
I refer to that calculated field.

See, for example, the form attached. Note that I had to patch JavaRosa in
order to be able to validate the XLSForm conversion -- so you probably
won't be able to convert the .xls file with XLSForm yourself. To try it,
you can use the .xml file, which was itself converted by my XLSForm (which
now has the patched JavaRosa).

Best,

Chris

SamplePositionForm.xls (31.5 KB)

SamplePositionForm.xml (3.87 KB)

ยทยทยท On Tue, May 7, 2013 at 12:38 PM, dj_bridges wrote:

Great stuff Chris - this is a feature that I am sure will be useful to a
lot of people. I have been trying to replicate the two position examples
you gave i.e. full Xpath and (..), to express in a note field e.g. "You are
entering person number 2 details" (where 2 is the iteration number of the
current repeat group). Unfortunately I can't get the note to show the
iteration number. Perhaps you have a simple form that you can share to show
us how you achieved this?

Cheers
Dan

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Picking back up on this thread... Since there is no current way to
dynamically index prior fields, you can't easily refer to an earlier
repeat-group's fields in later repeat groups. This is a real hindrance.

As a solution, I have added a repeatedfield() function to JavaRosa's
built-in function set. It takes two parameters: a reference to a field
within a repeat group, and an index number to fetch. Together with the
corrected support for position(..) (
https://code.google.com/p/opendatakit/issues/detail?id=822), this allows
you to easily reference a prior repeat group's fields from within a later
repeat group.

Here is the required addition to XPathFuncExpr.java:

} else if (name.equals("repeatedfield") && args.length == 2) {
if (argVals[0] instanceof XPathNodeset) {
XPathNodeset nodeset=(XPathNodeset)argVals[0];
int index=toInt(argVals[1]).intValue()-1;
if (index >= nodeset.size()) {
return XPathPathExpr.unpackValue(null);
}
else {
return (nodeset.getValAt(index));
}
} else {
throw new XPathTypeMismatchException("repeatedfield(): first parameter does
not refer to repeated field");
}

Attached is an example of the kind of form that becomes possible with this
repeatedfield() function, coupled with a functioning position() function.

Since this is highly-sought-after functionality, would the core team
consider adding these patches to the ODK JavaRosa fork?

Best,

Chris

SampleMultRepeatForm.xls (33 KB)

ยทยทยท On Tue, May 7, 2013 at 11:41 AM, Christopher Robert wrote:

I have traced the ODK Validate problem, and have filed issue 822 (
https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

On Mon, May 6, 2013 at 3:23 PM, Christopher Robert crobert@surveycto.comwrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

*1. ODK Validate won't validate these position() calls. *The "return
position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
seemingly because the repeat group's nodeset doesn't have any nodes in it
when called in the context of ODK Validate. This obviously has to do with
how ODK Validate loads and walks through a form, and I'm perplexed as to
what's going on.

*2. You can't use the current position to reference a value in a prior
repeat group. *For example, you can use
"/formid/repeatgroup1[position()=3]/name" to reference the name field in
the third iteration of repeatgroup1, and you can use
"position(/formid/repeatgroup2)" to reference the current position when
inside repeatgroup2 -- but
"/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
not work from inside repeatgroup2. When in the midst of indexing
repeatgroup1, the context doesn't seem suitable for getting your position
in repeatgroup2. If, instead, you stash the current repeatgroup2 position
into a calculated "repeatgroup2/pos" field, you can use that field within
the group -- but unfortunately not as part of indexing repeatgroup1; in the
indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
you want to reference. So, at the moment, we seem tantalizingly close to
being able to reference a prior repeat group from within a current repeat
group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert < crobert@surveycto.com> wrote:

ODK developers,

We are running up against a limitation that has been periodically raised
in this group (but always left unresolved): we would like to reference
prior field values conditional on the current iteration number of a repeat
group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

Chris,

Thanks for the examples. Am not too bothered about the XLS conversion.
Works exactly as you say - thanks so much!

Cheers
Dan

Chris,

The core team is either on vacation or doing a deployment. I'm sure
they'll patch this one in when they come back.

Yaw

ยทยทยท -- Need ODK help? Go to http://nafundi.com for custom features, form design, implementation support, and user training for ODK.

On Tue, May 28, 2013 at 3:00 AM, Christopher Robert crobert@surveycto.com wrote:

Picking back up on this thread... Since there is no current way to
dynamically index prior fields, you can't easily refer to an earlier
repeat-group's fields in later repeat groups. This is a real hindrance.

As a solution, I have added a repeatedfield() function to JavaRosa's
built-in function set. It takes two parameters: a reference to a field
within a repeat group, and an index number to fetch. Together with the
corrected support for position(..)
(https://code.google.com/p/opendatakit/issues/detail?id=822), this allows
you to easily reference a prior repeat group's fields from within a later
repeat group.

Here is the required addition to XPathFuncExpr.java:

} else if (name.equals("repeatedfield") && args.length == 2) {
if (argVals[0] instanceof XPathNodeset) {
XPathNodeset nodeset=(XPathNodeset)argVals[0];
int index=toInt(argVals[1]).intValue()-1;
if (index >= nodeset.size()) {
return XPathPathExpr.unpackValue(null);
}
else {
return (nodeset.getValAt(index));
}
} else {
throw new XPathTypeMismatchException("repeatedfield(): first parameter does
not refer to repeated field");
}

Attached is an example of the kind of form that becomes possible with this
repeatedfield() function, coupled with a functioning position() function.

Since this is highly-sought-after functionality, would the core team
consider adding these patches to the ODK JavaRosa fork?

Best,

Chris

On Tue, May 7, 2013 at 11:41 AM, Christopher Robert crobert@surveycto.com wrote:

I have traced the ODK Validate problem, and have filed issue 822
(https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

On Mon, May 6, 2013 at 3:23 PM, Christopher Robert crobert@surveycto.com wrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However, you
can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what we
want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

  1. ODK Validate won't validate these position() calls. The "return
    position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
    causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
    seemingly because the repeat group's nodeset doesn't have any nodes in it
    when called in the context of ODK Validate. This obviously has to do with
    how ODK Validate loads and walks through a form, and I'm perplexed as to
    what's going on.

  2. You can't use the current position to reference a value in a prior
    repeat group. For example, you can use
    "/formid/repeatgroup1[position()=3]/name" to reference the name field in the
    third iteration of repeatgroup1, and you can use
    "position(/formid/repeatgroup2)" to reference the current position when
    inside repeatgroup2 -- but
    "/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
    not work from inside repeatgroup2. When in the midst of indexing
    repeatgroup1, the context doesn't seem suitable for getting your position in
    repeatgroup2. If, instead, you stash the current repeatgroup2 position into
    a calculated "repeatgroup2/pos" field, you can use that field within the
    group -- but unfortunately not as part of indexing repeatgroup1; in the
    indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration you
    want to reference. So, at the moment, we seem tantalizingly close to being
    able to reference a prior repeat group from within a current repeat group --
    but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert crobert@surveycto.com wrote:

ODK developers,

We are running up against a limitation that has been periodically raised
in this group (but always left unresolved): we would like to reference prior
field values conditional on the current iteration number of a repeat group.
For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

FYI, I have renamed the function from repeatedfield() to repeated-field(),
in order to be more consistent with other built-in functions. Hopefully
this will be something that can be integrated into the ODK JavaRosa fork so
that it doesn't become a SurveyCTO-only thing.

Best,

Chris

ยทยทยท On Tue, May 28, 2013 at 12:00 PM, Christopher Robert wrote:

Picking back up on this thread... Since there is no current way to
dynamically index prior fields, you can't easily refer to an earlier
repeat-group's fields in later repeat groups. This is a real hindrance.

As a solution, I have added a repeatedfield() function to JavaRosa's
built-in function set. It takes two parameters: a reference to a field
within a repeat group, and an index number to fetch. Together with the
corrected support for position(..) (
https://code.google.com/p/opendatakit/issues/detail?id=822), this allows
you to easily reference a prior repeat group's fields from within a later
repeat group.

Here is the required addition to XPathFuncExpr.java:

} else if (name.equals("repeatedfield") && args.length == 2) {
if (argVals[0] instanceof XPathNodeset) {
XPathNodeset nodeset=(XPathNodeset)argVals[0];
int index=toInt(argVals[1]).intValue()-1;
if (index >= nodeset.size()) {
return XPathPathExpr.unpackValue(null);
}
else {
return (nodeset.getValAt(index));
}
} else {
throw new XPathTypeMismatchException("repeatedfield(): first parameter
does not refer to repeated field");
}

Attached is an example of the kind of form that becomes possible with this
repeatedfield() function, coupled with a functioning position() function.

Since this is highly-sought-after functionality, would the core team
consider adding these patches to the ODK JavaRosa fork?

Best,

Chris

On Tue, May 7, 2013 at 11:41 AM, Christopher Robert <crobert@surveycto.com wrote:

I have traced the ODK Validate problem, and have filed issue 822 (
https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

On Mon, May 6, 2013 at 3:23 PM, Christopher Robert <crobert@surveycto.com wrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

*1. ODK Validate won't validate these position() calls. *The "return
position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
seemingly because the repeat group's nodeset doesn't have any nodes in it
when called in the context of ODK Validate. This obviously has to do with
how ODK Validate loads and walks through a form, and I'm perplexed as to
what's going on.

*2. You can't use the current position to reference a value in a prior
repeat group. *For example, you can use
"/formid/repeatgroup1[position()=3]/name" to reference the name field in
the third iteration of repeatgroup1, and you can use
"position(/formid/repeatgroup2)" to reference the current position when
inside repeatgroup2 -- but
"/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
not work from inside repeatgroup2. When in the midst of indexing
repeatgroup1, the context doesn't seem suitable for getting your position
in repeatgroup2. If, instead, you stash the current repeatgroup2 position
into a calculated "repeatgroup2/pos" field, you can use that field within
the group -- but unfortunately not as part of indexing repeatgroup1; in the
indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
you want to reference. So, at the moment, we seem tantalizingly close to
being able to reference a prior repeat group from within a current repeat
group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert < crobert@surveycto.com> wrote:

ODK developers,

We are running up against a limitation that has been periodically
raised in this group (but always left unresolved): we would like to
reference prior field values conditional on the current iteration number of
a repeat group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

Just to close the loop on this, in case anybody was following the
discussion...

We released an implementation of this new JavaRosa function in SurveyCTO
today:

indexed-repeat(repeatedfield, repeatgroup, index): References a field or
group that is inside a prior repeat group. The first parameter specifies
the prior field or group in which you are interested; the second specifies
the prior repeat group within which thefield or group of interest is
located; and the third specifies the instance number, within the prior
repeat group, to use. For example, the calculate expression
"indexed-repeat(${name}, ${names}, 1)" will return the first name available
when the "name" field is inside a prior repeat group named "names". From
inside a later repeat group, the calculate expression
"indexed-repeat(${name}, ${names}, position(..))" will pull the xth name
from the prior repeat group, where x is the instance number of the current
repeat group (e.g., if currently in the fourth instance of a repeat group,
it will return the fourth name from the earlier repeat group). See the
multiple-repeat
sample form
for a simple example. If you need to reference a field or
group within multiple nested repeat groups, you can supply additional
parameters to indicate the instance numbers to use for each level of
nesting. For example, the calculate expression "indexed-repeat(${name},
${families}, ${familynumber}, ${names}, ${membernumber})" will pull a
particular family member's name when family member names are inside a
repeat group that is itself inside a repeat group of families. See the second
multiple-repeat sample form
for a much more complex example that includes
three levels of nested repeat groups.

Both sample forms are attached.

We revised the implementation in consultation with Mitch, and he has the
code for this. So, hopefully it -- or a better version, if somebody builds
one -- will become part of the core.

Best,

Chris

SampleMultRepeat2Form.xls (35.5 KB)

SampleMultRepeatForm.xls (33 KB)

ยทยทยท On Tue, May 28, 2013 at 12:00 PM, Christopher Robert wrote:

Picking back up on this thread... Since there is no current way to
dynamically index prior fields, you can't easily refer to an earlier
repeat-group's fields in later repeat groups. This is a real hindrance.

As a solution, I have added a repeatedfield() function to JavaRosa's
built-in function set. It takes two parameters: a reference to a field
within a repeat group, and an index number to fetch. Together with the
corrected support for position(..) (
https://code.google.com/p/opendatakit/issues/detail?id=822), this allows
you to easily reference a prior repeat group's fields from within a later
repeat group.

Here is the required addition to XPathFuncExpr.java:

} else if (name.equals("repeatedfield") && args.length == 2) {
if (argVals[0] instanceof XPathNodeset) {
XPathNodeset nodeset=(XPathNodeset)argVals[0];
int index=toInt(argVals[1]).intValue()-1;
if (index >= nodeset.size()) {
return XPathPathExpr.unpackValue(null);
}
else {
return (nodeset.getValAt(index));
}
} else {
throw new XPathTypeMismatchException("repeatedfield(): first parameter
does not refer to repeated field");
}

Attached is an example of the kind of form that becomes possible with this
repeatedfield() function, coupled with a functioning position() function.

Since this is highly-sought-after functionality, would the core team
consider adding these patches to the ODK JavaRosa fork?

Best,

Chris

On Tue, May 7, 2013 at 11:41 AM, Christopher Robert <crobert@surveycto.com wrote:

I have traced the ODK Validate problem, and have filed issue 822 (
https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

On Mon, May 6, 2013 at 3:23 PM, Christopher Robert <crobert@surveycto.com wrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

*1. ODK Validate won't validate these position() calls. *The "return
position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
seemingly because the repeat group's nodeset doesn't have any nodes in it
when called in the context of ODK Validate. This obviously has to do with
how ODK Validate loads and walks through a form, and I'm perplexed as to
what's going on.

*2. You can't use the current position to reference a value in a prior
repeat group. *For example, you can use
"/formid/repeatgroup1[position()=3]/name" to reference the name field in
the third iteration of repeatgroup1, and you can use
"position(/formid/repeatgroup2)" to reference the current position when
inside repeatgroup2 -- but
"/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
not work from inside repeatgroup2. When in the midst of indexing
repeatgroup1, the context doesn't seem suitable for getting your position
in repeatgroup2. If, instead, you stash the current repeatgroup2 position
into a calculated "repeatgroup2/pos" field, you can use that field within
the group -- but unfortunately not as part of indexing repeatgroup1; in the
indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
you want to reference. So, at the moment, we seem tantalizingly close to
being able to reference a prior repeat group from within a current repeat
group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert < crobert@surveycto.com> wrote:

ODK developers,

We are running up against a limitation that has been periodically
raised in this group (but always left unresolved): we would like to
reference prior field values conditional on the current iteration number of
a repeat group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris

1 Like

Congratulations Chris....

This will be a great addition! Hope it is easy for the core team to add
into a future release of ODK!

Hi Chris,

This is very useful in referencing prior IDs. Do you think that in the repeat, one can also reference prior IDs conditional on some certain features associating with the IDs. For example, I have IDs of household members, but I only want to repeat them for all female household members, or for children under 5 or for adults above 10 of this household. Is there a index can be added on this one?

Also, I cannot transform the XLS form to XLM. How to insert the "required addition to XPathFuncExpr.java" you mentioned above to use this indexed-repeat(repeatedfield, repeatgroup, index) code? I am using the XLS form converter downloaded from https://code.google.com/p/opendatakit/downloads/list?can=1&q=&colspec=Filename+Summary+Uploaded+ReleaseDate+Size+DownloadCount and it seemed that this was clearly outdated (uploaded in December 2012).

Thank you.

ยทยทยท On Friday, June 7, 2013 6:22:15 AM UTC-4, Christopher Robert wrote: > Just to close the loop on this, in case anybody was following the discussion... > > > > > We released an implementation of this new JavaRosa function in SurveyCTO today: > > > > indexed-repeat(repeatedfield, repeatgroup, index): References a field or group that is inside a prior repeat group. The first parameter specifies the prior field or group in which you are interested; the second specifies the prior repeat group within which thefield or group of interest is located; and the third specifies the instance number, within the prior repeat group, to use. For example, the calculate expression "indexed-repeat(${name}, ${names}, 1)" will return the first name available when the "name" field is inside a prior repeat group named "names". From inside a later repeat group, the calculate expression "indexed-repeat(${name}, ${names}, position(..))" will pull the xth name from the prior repeat group, where x is the instance number of the current repeat group (e.g., if currently in the fourth instance of a repeat group, it will return the fourth name from the earlier repeat group). See the multiple-repeat sample form for a simple example. If you need to reference a field or group within multiple nested repeat groups, you can supply additional parameters to indicate the instance numbers to use for each level of nesting. For example, the calculate expression "indexed-repeat(${name}, ${families}, ${familynumber}, ${names}, ${membernumber})" will pull a particular family member's name when family member names are inside a repeat group that is itself inside a repeat group of families. See the second multiple-repeat sample form for a much more complex example that includes three levels of nested repeat groups. > > > > > Both sample forms are attached. > > > > > We revised the implementation in consultation with Mitch, and he has the code for this. So, hopefully it -- or a better version, if somebody builds one -- will become part of the core. > > > > > Best, > > > > > Chris > > > > > > On Tue, May 28, 2013 at 12:00 PM, Christopher Robert wrote: > > > > > Picking back up on this thread... Since there is no current way to dynamically index prior fields, you can't easily refer to an earlier repeat-group's fields in later repeat groups. This is a real hindrance. > > > > > > As a solution, I have added a repeatedfield() function to JavaRosa's built-in function set. It takes two parameters: a reference to a field within a repeat group, and an index number to fetch. Together with the corrected support for position(..) (https://code.google.com/p/opendatakit/issues/detail?id=822), this allows you to easily reference a prior repeat group's fields from within a later repeat group. > > > > > > Here is the required addition to XPathFuncExpr.java: > > > > } else if (name.equals("repeatedfield") && args.length == 2) { > > > > > if (argVals[0] instanceof XPathNodeset) { > > XPathNodeset nodeset=(XPathNodeset)argVals[0]; > > > > > int index=toInt(argVals[1]).intValue()-1; > > > > > if (index >= nodeset.size()) { > > return XPathPathExpr.unpackValue(null); > > > } > > else { > > return (nodeset.getValAt(index)); > > > > > } > > } else { > > throw new XPathTypeMismatchException("repeatedfield(): first parameter does not refer to repeated field"); > > > > > } > > > > Attached is an example of the kind of form that becomes possible with this repeatedfield() function, coupled with a functioning position() function. > > > > > > Since this is highly-sought-after functionality, would the core team consider adding these patches to the ODK JavaRosa fork? > > > Best, > > > > Chris > > > > > > > > On Tue, May 7, 2013 at 11:41 AM, Christopher Robert wrote: > > > > > I have traced the ODK Validate problem, and have filed issue 822 (https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly. That is the primary issue standing in the way of people referencing the current repeat index. > > > > > > > The one-repeat-group-referencing-another problem remains less clear. > > > Chris > > > > > > > > > > On Mon, May 6, 2013 at 3:23 PM, Christopher Robert wrote: > > > > > > Use of the position() function has been mentioned several times, in the context of related questions. It turns out that position(), called without any parameters within a repeat group, seems to always return 1. However, you can call position(..) (or pass the full XPath to the repeat group node itself, instead of ".."), and the returned value seems to be exactly what we want: the iteration number of the current repeat group. That's wonderful news, because it means that we can reference the current iteration number inside a repeat group. > > > > > > > > > However, there are two limitations at present: > > > 1. ODK Validate won't validate these position() calls. The "return position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception, seemingly because the repeat group's nodeset doesn't have any nodes in it when called in the context of ODK Validate. This obviously has to do with how ODK Validate loads and walks through a form, and I'm perplexed as to what's going on. > > > > > > > > 2. You can't use the current position to reference a value in a prior repeat group. For example, you can use "/formid/repeatgroup1[position()=3]/name" to reference the name field in the third iteration of repeatgroup1, and you can use "position(/formid/repeatgroup2)" to reference the current position when inside repeatgroup2 -- but "/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does not work from inside repeatgroup2. When in the midst of indexing repeatgroup1, the context doesn't seem suitable for getting your position in repeatgroup2. If, instead, you stash the current repeatgroup2 position into a calculated "repeatgroup2/pos" field, you can use that field within the group -- but unfortunately not as part of indexing repeatgroup1; in the indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration you want to reference. So, at the moment, we seem tantalizingly close to being able to reference a prior repeat group from within a current repeat group -- but not quite there. > > > > > > > > I suspect that there are easy fixes for both of these limitations. Does somebody more familiar with ODK's JavaRosa/ODKValidate implementations have any suggestions? Again, I'm happy to make and test the fixes, but it's not yet obvious to me where the true sources of these problems are. > > > > > > > > Thanks, > > > Chris > > > > > > > > On Fri, May 3, 2013 at 10:14 AM, Christopher Robert wrote: > > > > > > > ODK developers, > > > We are running up against a limitation that has been periodically raised in this group (but always left unresolved): we would like to reference prior field values conditional on the current iteration number of a repeat group. For example: > > > > > > > > > ${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the repeat group, agplot2 on iteration 2, and so on. > > > > Likewise, we would like one repeat group to be able to reference values from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to pull the GROUPNUMth value of agplot, from within a prior repeat group. > > > > > > > > > I am not worried about syntax, or even about the XLSForm part of this. Working out appropriate XPath logic and JaraRosa support seems to be the major stumbling block. Right now, you can't even reference the current repeat group's iteration number (i.e., the number that shows at the top of the entry screen, to indicate which iteration number you are on). > > > > > > > > > We are happy to do the heavy lifting on this, and to contribute the solution to ODK/JavaRosa. However, we could very much use guidance from those of you who have already thought about this. > > > > Any sage advice? > > > Thanks, > > > Chris

Hi Eva,

You can use http://opendatakit.org/use/xlsform/ to convert XLS files to
XML. The downloadable version is, as you found, out-of-date relative to the
online version. When it was initially posted, I guess that nobody committed
to keeping it forever up-to-date. You can find the sources at
https://github.com/UW-ICTD/pyxform.git, though, and anybody -- including
yourself -- might volunteer to keep the downloadable version up-to-date.

It sounds like you want to do some quite advanced things with your form.
What you want to do is entirely possible, but will require a fair bit of
work to develop and test. In a prior email, you mentioned having a repeat
that collects names and genders, then wanting to ask follow-up questions
for one or more females. You might do that by:

  1. After the repeat, having a series of name1, name2, etc. fields that pull
    the names from the repeat into a series of calculate fields. You'd use
    indexed-repeat() for that.

  2. Let the user choose a female from the list by putting ${name1},
    ${name2}, etc. as the labels on the choices sheet. Filtering this list
    might be difficult, so it might be best to just list all names and let them
    choose a female.

  3. Have a constraint that uses indexed-repeat() to check the gender of the
    selected individual. Don't allow males to be selected.

Better yet, you could have a second repeat group that has a repeat_count of
count(${firstgroupname}), to ask follow-up questions for each individual.
Using indexed-repeat() and position(..), you could pull the gender from the
first group into the second group, then you could make your follow-up
questions relevant only if gender is female. Thus, you could have a series
of follow-up questions that only appear for females.

Again, though, these are complex forms and will require a fair bit of work.
In SurveyCTO, we have some sample forms that provide some of the basics
(e.g., for having one repeat group follow another, to ask follow-up
questions), but even with SurveyCTO there might be advanced modifications
you'd want to make. If it ends up being more than you can comfortably
program yourself, you could hire a consultant to help. If you email me, I
can recommend a few options.

Best,

Chris

ยทยทยท On Tue, Sep 3, 2013 at 12:09 PM, wrote:

Hi Chris,

This is very useful in referencing prior IDs. Do you think that in the
repeat, one can also reference prior IDs conditional on some certain
features associating with the IDs. For example, I have IDs of household
members, but I only want to repeat them for all female household members,
or for children under 5 or for adults above 10 of this household. Is there
a index can be added on this one?

Also, I cannot transform the XLS form to XLM. How to insert the "required
addition to XPathFuncExpr.java" you mentioned above to use this
indexed-repeat(repeatedfield, repeatgroup, index) code? I am using the XLS
form converter downloaded from
https://code.google.com/p/opendatakit/downloads/list?can=1&q=&colspec=Filename+Summary+Uploaded+ReleaseDate+Size+DownloadCountand it seemed that this was clearly outdated (uploaded in December 2012).

Thank you.

On Friday, June 7, 2013 6:22:15 AM UTC-4, Christopher Robert wrote:

Just to close the loop on this, in case anybody was following the
discussion...

We released an implementation of this new JavaRosa function in SurveyCTO
today:

indexed-repeat(repeatedfield, repeatgroup, index): References a field or
group that is inside a prior repeat group. The first parameter specifies
the prior field or group in which you are interested; the second specifies
the prior repeat group within which thefield or group of interest is
located; and the third specifies the instance number, within the prior
repeat group, to use. For example, the calculate expression
"indexed-repeat(${name}, ${names}, 1)" will return the first name available
when the "name" field is inside a prior repeat group named "names". From
inside a later repeat group, the calculate expression
"indexed-repeat(${name}, ${names}, position(..))" will pull the xth name
from the prior repeat group, where x is the instance number of the current
repeat group (e.g., if currently in the fourth instance of a repeat group,
it will return the fourth name from the earlier repeat group). See
the multiple-repeat sample form for a simple example. If you need to
reference a field or group within multiple nested repeat groups, you can
supply additional parameters to indicate the instance numbers to use for
each level of nesting. For example, the calculate expression
"indexed-repeat(${name}, ${families}, ${familynumber}, ${names},
${membernumber})" will pull a particular family member's name when family
member names are inside a repeat group that is itself inside a repeat group
of families. See the second multiple-repeat sample form for a much more
complex example that includes three levels of nested repeat groups.

Both sample forms are attached.

We revised the implementation in consultation with Mitch, and he has the
code for this. So, hopefully it -- or a better version, if somebody builds
one -- will become part of the core.

Best,

Chris

On Tue, May 28, 2013 at 12:00 PM, Christopher Robert < cro...@surveycto.com> wrote:

Picking back up on this thread... Since there is no current way to
dynamically index prior fields, you can't easily refer to an earlier
repeat-group's fields in later repeat groups. This is a real hindrance.

As a solution, I have added a repeatedfield() function to JavaRosa's
built-in function set. It takes two parameters: a reference to a field
within a repeat group, and an index number to fetch. Together with the
corrected support for position(..) (
https://code.google.com/p/opendatakit/issues/detail?id=822), this allows
you to easily reference a prior repeat group's fields from within a later
repeat group.

Here is the required addition to XPathFuncExpr.java:

          } else if (name.equals("repeatedfield") && args.length ==
  1. {
                  if (argVals[0] instanceof XPathNodeset) {

                          XPathNodeset

nodeset=(XPathNodeset)argVals[0];

                          int index=toInt(argVals[1]).intValue()-1;




                          if (index >= nodeset.size()) {

                                  return

XPathPathExpr.unpackValue(null);

                          }

                          else {

                                  return (nodeset.getValAt(index));




                          }

                  } else {

                          throw new

XPathTypeMismatchException("repeatedfield(): first parameter does not refer
to repeated field");

                  }

Attached is an example of the kind of form that becomes possible with
this repeatedfield() function, coupled with a functioning position()
function.

Since this is highly-sought-after functionality, would the core team
consider adding these patches to the ODK JavaRosa fork?

Best,

Chris

On Tue, May 7, 2013 at 11:41 AM, Christopher Robert < cro...@surveycto.com> wrote:

I have traced the ODK Validate problem, and have filed issue 822 (
https://code.google.com/p/opendatakit/issues/detail?id=822) accordingly.
That is the primary issue standing in the way of people referencing the
current repeat index.

The one-repeat-group-referencing-another problem remains less clear.

Chris

On Mon, May 6, 2013 at 3:23 PM, Christopher Robert cro...@surveycto.com wrote:

Use of the position() function has been mentioned several times, in the
context of related questions. It turns out that position(), called without
any parameters within a repeat group, seems to always return 1. However,
you can call position(..) (or pass the full XPath to the repeat group node
itself, instead of ".."), and the returned value seems to be exactly what
we want: the iteration number of the current repeat group. That's wonderful
news, because it means that we can reference the current iteration number
inside a repeat group.

However, there are two limitations at present:

  1. ODK Validate won't validate these position() calls. The "return
    position(((XPathNodeset)argVals[0]).getRefAt(0))" in XPathFuncExpr.java
    causes a "java.lang.ArrayIndexOutOfBoundsException: 0 >= 0" exception,
    seemingly because the repeat group's nodeset doesn't have any nodes in it
    when called in the context of ODK Validate. This obviously has to do with
    how ODK Validate loads and walks through a form, and I'm perplexed as to
    what's going on.

  2. You can't use the current position to reference a value in a prior
    repeat group. For example, you can use
    "/formid/repeatgroup1[position()=3]/name" to reference the name field in
    the third iteration of repeatgroup1, and you can use
    "position(/formid/repeatgroup2)" to reference the current position when
    inside repeatgroup2 -- but
    "/formid/repeatgroup1[position()=position(/formid/repeatgroup2)]/name" does
    not work from inside repeatgroup2. When in the midst of indexing
    repeatgroup1, the context doesn't seem suitable for getting your position
    in repeatgroup2. If, instead, you stash the current repeatgroup2 position
    into a calculated "repeatgroup2/pos" field, you can use that field within
    the group -- but unfortunately not as part of indexing repeatgroup1; in the
    indexing context, JavaRosa doesn't know which repeatgroup2/pos iteration
    you want to reference. So, at the moment, we seem tantalizingly close to
    being able to reference a prior repeat group from within a current repeat
    group -- but not quite there.

I suspect that there are easy fixes for both of these limitations. Does
somebody more familiar with ODK's JavaRosa/ODKValidate implementations have
any suggestions? Again, I'm happy to make and test the fixes, but it's not
yet obvious to me where the true sources of these problems are.

Thanks,

Chris

On Fri, May 3, 2013 at 10:14 AM, Christopher Robert < cro...@surveycto.com> wrote:

ODK developers,

We are running up against a limitation that has been periodically raised
in this group (but always left unresolved): we would like to reference
prior field values conditional on the current iteration number of a repeat
group. For example:

${agplot${GROUPNUM}} would pull the agplot1 field on iteration 1 of the
repeat group, agplot2 on iteration 2, and so on.

Likewise, we would like one repeat group to be able to reference values
from prior repeat groups. This might look like ${agplot[${GROUPNUM}]} to
pull the GROUPNUMth value of agplot, from within a prior repeat group.

I am not worried about syntax, or even about the XLSForm part of this.
Working out appropriate XPath logic and JaraRosa support seems to be the
major stumbling block. Right now, you can't even reference the current
repeat group's iteration number (i.e., the number that shows at the top of
the entry screen, to indicate which iteration number you are on).

We are happy to do the heavy lifting on this, and to contribute the
solution to ODK/JavaRosa. However, we could very much use guidance from
those of you who have already thought about this.

Any sage advice?

Thanks,

Chris