Infinite loop and dynamically delete already selected options from list

[mathieubossaert]

Godd evening to all, hope everybody is ok.

I face a problem I can't resolve.
We drive a field servey of plants species inventory. Species list are different from on site to another (I use choice filter to manage it).
For the moment I builded my form in a non generic, hard to modify way : -> A select widget for the abundance of each species

Reading yesterday some great recipes on the forum, I tried and I achieved to merge two great ones from @LN and @jules_rugwiro :

Returning a list of multiple select questions which doesn't contain a previously selected choice

Hello Green, It's possible to achieve this. you will need to use filters as indicated below: Involved columns are "choice-filter" in survey and "filter" in choices. Step one: Define the choices (name and label) in the choices and then add an extra column called "filter" of which the content is the copy paste of the "name" column. Step two: create your variables (select_one and select_multiple) as desired Step three: add the column "choice_filter" and pass the following function for the sel…

and

XLS form- Repeat select_one question until "A" is selected

To achieve something like this, you can define the repeat count in terms of your desired condition. This indefinite-repeat XLSForm shows an example of that technique. The key part is that the count is set to if (${count} = 0 or (${person}[position()=${count}]/choice != '' and ${person}[position()=${count}]/choice != 'a'), ${count} + 1, ${count}) edited 6/2020 to use ${} for repeat reference instead of XPath path This says that if there are currently no repeats or your ending c…

So my colleagues can now loop in the species list until the select 'no more species' AND the previously select species get out of the list, so the can not count the same species two times.
This works really fine.

Now I want to include it into a bigger repeat group (lines 13 & 29 in the attached xls file)
indefinite_repeat_and_remove_already_selected_from_species_list.xlsx (10.8 KB)
This repeat group concern the locations visited on the field.

But when I try it I get a error from xlsformonline:

Error: ODK Validate Errors:

Something broke the parser. See above for a hint.
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
The following files failed validation:
${indefinite_repeat_and_remove_already_selected_from_species_list}.xml
Result: Invalid

I do not have any idea about the reason but I am not an xlsform guru

[LN]

I haven’t had a chance to look at your form yet but I’m guessing that you are trying to do this in a nested repeat, is that right? If so, you are running into a Collect bug that we hope to publish a fix for this week.

[mathieubossaert]

Yes exactly A first repeat of geopoint and inside an infinite repeat for species.
Great to ear that it will probably work !
Now I remember I red something about this bug... but I am not sure... This will be a very useful form for us.

[mathieubossaert]

May be I was too optimistic in my last message because I get the error when I try to transform the xls file into xml, not in colect...

[LN]

@mathieubossaert

I get the error when I try to transform the xls file into xml

Yes, that makes sense. The error is in JavaRosa which both use. We'll aim to publish a pyxform/XLSForm Online fix soon as well.

[LN]

Quick update here. We ended up doing a Collect point release without this fix because I noticed some problems with it and was pulled into other time-sensitive work. I hope to get back to it by the end of the week.

[LN]

Collect v1.26.3 is now out and should fix this issue.

However, I have since realized that the sample form can cause an infinite loop in Collect when using the hierarchy view or finalizing. To avoid this, you can change the count expression to add a check for blank values: ${count} = 0 or (${taxons_observes}[position()=${count}]/espece != '' and ${taxons_observes}[position()=${count}]/espece != 'maille_suivante'). I have updated the original sample form accordingly. Please give it a try and let us know how it works for you, @mathieubossaert!

[mathieubossaert]

For the moment I still can't transform the xls to xml with XLSForm Online.

[LN]

Ah, indeed, I had converted the form some weeks ago and forgot you'd run into that!

indefinite_repeat_and_remove_already_selected_from_species_list.xml (10.1 KB) is the XML I got by making the change described above and running the converter without the Validate step. I'll try to get XLSForm Online updated sooner than later.

[mathieubossaert]

Thanks @LN it works perfectly !

[mathieubossaert]

I spoke too fast.
The form logic is ok but I realized that the form has not been saved (Collect is still finalizing / saving the instance).

I have to kill Collect (1.26.3) and when I restart it, I am unable to load the form while the scrren freeze on "Loading"

[LN]

Do you remember when I said that there was a change needed to the form design to make sure there wasn't an infinite loop on finalization? I sent you the converted form without making that change. So sorry about that.

Hopefully this one works better: indefinite_repeat_and_remove_already_selected_from_species_list.xml (10.2 KB)

[mathieubossaert]

Works great. Thanks @LN

[mathieubossaert]

I am facing a problem with this form. I am preparing the 2022 field season and I would like to use this generic form, as the number of species and sites is now too big to use a none generic way.
On the collect side, the form is well finalized at the end of data collection (as I said in the last message).
I try it today and go forward with submissions. And I realize that species code (select_one especes) is empty.

I can't understand why so I think it could be a bug ?

Here are the form and submission data archive
suivi_maille_generique.xlsx (14.1 KB)
suivi_maille_generique.csv-3.zip (1.5 KB)

[mathieubossaert]

It is quite weird, when I add a calculate field containing ${espece} this calculation is ok and this field is well submitted to Central, but select_one still empty.

suivi_maille_generique.xlsx (14.1 KB)

[mathieubossaert]

Thanks to @LN I managed to get it work :

Choice that was filtered out of items to choose from does not appear in submitted data

What's going on is that when the form is finalized, the choice list is recomputed for every select and values that are no longer available are cleared. By finalization time, the accumulator (previously_selected_strc in your case) includes every single selection ever made and so all answers are cleared. It's possible that there's been a change of behavior here but I don't think so. When I've written this kind of form, I've always explicitly made sure the choice_filter includes the choice for the…

Here is the final form !
suivi_maille_generique.xlsx (14,2 Ko)
Edit : because I forgot to "unhide" the geopint question

[willydauphin]

Hi !
I found another solution to do that trick ! The main idea is to concat the current choice (Q1) with an unique id (position(..)) (Q0) in order to compare the current selection of a repeat group and the other ones (C1). The choice_filter exludes the current selection (Q1) from the list (C1) with the functions substring-before and substring-after. Then, it only check if the name was already selected or not.

By using the functions substring-before and substring-after, we are able to edit previous selections without any duplication.

This is the trick (working on Kobo Collect and Enketo) :

survey :

choices :

I hope it will help someone. See you

William

[willydauphin]

Hi,
The previous solution worked only for name values with the same length or really differents (for example the values "on" and "one" would not have worked). This another version is working with a concatenation of a special character ('#' in red). It's not exhaustive but for me... it's enough ! You can of course extand the character with a unique sequence ('#$^%ù~ [...] +=').

survey

It could be great to limit the number of repetition with the size of Q1 list. Any idea to achieve it ?

Looking to get your feedback

See you,
William

[mathieubossaert]

Thanks @willydauphin and welcome to the ODK community forum ! When you get a chance, please take some time to introduce yourself here
Thanks for sharing, I will try it with Collect. Lazy question : could you share an xlsform ?

@willydauphin

It could be great to limit the number of repetition with the size of Q1 list. Any idea to achieve it ?

You might take a look at this discussion and the proposed trick

[willydauphin]

Thanks @mathieubossaert for your reply !

Yes I can share xlsform !

Solution :
test_choice_filter.xlsx (13.6 KB)

Solution (with a "dynamic" repeat-count, according to this) :
test_choice_filter_with_limit.xlsx (13.8 KB)

Unfortunately, I'm wondering if it's possible to limit the number of repetitions. With the second form, I only fix the number of repetitions. I would like to have someting like '.<${Q1_length}' but it looks like impossible or seems to be a very bad way. Any idea ?

Thanks again