# Add distinct function for use with entities **URL:** \ **Category:** Ideas\ **Tags:** odk-xforms\ **Created:** [September 4, 2023, 12:26pm UTC](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945 "2023-09-04T12:26:00Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![TomJ](https://getodk.b-cdn.net/letter_avatar_proxy/v4/letter/t/dfb087/32.png) [@TomJ](https://forum.getodk.org/u/TomJ)\ **Post date:** [September 4, 2023, 12:26pm UTC](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945/1 "2023-09-04T12:26:00Z")
**What high-level problem are you trying to solve?** With entities now available, there are some form logic operations that cannot be achieved with the existing set of form functions. i.e. The "trees" dataset, which is populated with entities created from a form "Tree registration form". In the form "Tree follow-up Form", we want to filter down the "trees" dataset by the property "species", so in a subsequent question a user only has a limited number of options to select from. We want this to dynamically respond to entities with new species in the dataset, rather than having to update the definition. There is not currently a way to build a distinct list of "tree/species" directly within the form to use in a select\_one question. This can be solved by utilising multiple datasets, i.e. "trees" & "species", however this adds complexity to manage, especially since a form can currently only create a single entity. Previously, this type of distinct operation was rolled into the form definition ( e.g. using Excel functions), however entities should remove the need to update definitions for this reason. **Any ideas on how ODK could help you solve it?** By adding the function [distinct-values](https://www.w3.org/2005/xpath-functions) This would allow for a nodeset that contains only the "species" values to be calculated, and the distinct values of that nodeset used in the select\_one question. Alternatively, by extending support for regular expressions to allow groups, not just a boolean as the result of the calculation. **Upload any helpful links, sketches, and videos.** ---
**Author:** ![ahblake](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/ahblake/32/22392_2.png) [@ahblake](https://forum.getodk.org/u/ahblake)\ **Post date:** [October 15, 2024, 5:03am UTC](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945/2 "2024-10-15T05:03:29Z")
> [@Calculate unique items outside a repeat](https://forum.getodk.org/t/calculate-unique-items-outside-a-repeat/50040/2): > > This feels closely related to [Add distinct function for use with entities](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945) This function or something like it would be useful indeed! It would allow dynamic 'reverse choice filtering' more easily possible. Currently I am creating a join of all/some of my dataset where a certain field matches (eg for the big entity list, join all `site` values for my entity items where `status = incomplete`), and then setting my choice filter on the parent (eg select site) to search the join for matching items and exclude others (so if only Site A, Site C, Site D, Site Z appear in the join, then my choice list for 'select site' will only show these matching options and not other sites that don't have incomplete items). This results in a join that can be thousands of items in size and is not performant (particularly in Enketo). An alternate to the `search(distinct-values(join()))` or however it's implemented, would be as above, eg if a unique nodeset of `site` values [Site A, Site, C, Site D, Site Z] from the large entity list (`biglist.csv`) could be created, and the question 'select the site' as `select_one_from_file biglist.csv` would present these options - using parameters such that `value=site, label=site` and choice filter like `distinct-values(instance('biglist')/root/item[status='incomplete']/site)`, this removes the need for the parent list(s) altogether (though I'm pretty sure my expression is written incorrectly!) ---
**Author:** ![chun\_hing\_yap](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/chun_hing_yap/32/13598_2.png) [@chun\_hing\_yap](https://forum.getodk.org/u/chun_hing_yap)\ **Post date:** [October 17, 2024, 12:11am UTC](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945/3 "2024-10-17T00:11:15Z")
I also wish to have a distinct-values(nodeset) function to only returns unique items in the nodeset. The use case can be as straightforward as counting distinct values, or more advanced, such as serving as a choice list in select\_one/select\_many questions ---
**Author:** ![Xiphware](https://getodk.b-cdn.net/user_avatar/forum.getodk.org/xiphware/32/2587_2.png) [@Xiphware](https://forum.getodk.org/u/Xiphware)\ **Post date:** [December 30, 2025, 5:46am UTC](https://forum.getodk.org/t/add-distinct-function-for-use-with-entities/42945/4 "2025-12-30T05:46:42Z")
> [@chun\_hing\_yap](#): > > have a distinct-values(nodeset) function to only returns unique items in the nodeset This is actually already possible (!) - that is, retrieving only the _unique_ values from a nodeset - but it currently only works in Enketo, not Collect, by using the [preceding](https://www.w3schools.com/xml/xpath_axes.asp) XPath axis. This ‘axis’ is one which Enketo supports but Collect doesn’t [yet… 😁] To give an concrete example, this is a deployable XML XForm that basically includes a **choice-filter** that will filter out any duplicates in the `select_one` choice list [that is, after also including an **allow\_choice\_duplicates** settings option to stop it from complaining about the duplicates in the first place]. [UniqueSelect.xml](https://forum.getodk.org/uploads/short-url/rDmEgMpe8ZY74k52x4XeOWWboe0.xml) (3.1 KB) The Secret Sauce™ is in the nodeset filter of the final select1, which removes any duplicate entries: `` ` ` ` change 'item/name' to 'preceding::item/name' in XML` ` ` ` ` ` ` `` Unfortunately, because Collect doesn’t (yet) support this `preceding::` axis, you cant actually put the desired choice-filter in your XLSForm without pyxform complaining because it runs everything thru javarosa once to validate these XPath expressions, which barfs when it sees this filter expression in your XLSForm **choice-filter** column (with a somewhat obscure error, I might add 🤨 ): ![Screenshot 2025-12-30 at 6.11.45 PM](https://getodk.b-cdn.net/uploads/default/original/3X/8/6/86d97b9384fb2bc98c97ac0ef2fc26a3a9ffc5d2.png) So, instead, you have to manually insert the desired filter into the resulting _XForm_. But once you do this you can then deploy your XForm form to Central and run it under the web viewer/Enketo [I actually put ‘`not(./name=item/name)`’ in my original XLSForm, which converts fine, and then change it to ‘`not(./name=preceding::item/name)`’ in the resulting the XForm, per my hint]. This is my _unmodified_ XLSForm containing some duplicated choices: [UniqueSelect.xlsx](https://forum.getodk.org/uploads/short-url/CbGENhZc3AJLsqGsu6dQOcuo3I.xlsx) (18.4 KB) ![Screenshot 2025-12-30 at 6.19.55 PM](https://getodk.b-cdn.net/uploads/default/original/3X/0/2/02404f4c6932723ef47e7bb4d1af77c835b72e17.png) ![Screenshot 2025-12-30 at 6.20.26 PM](https://getodk.b-cdn.net/uploads/default/original/3X/1/4/14b134214b0915dd36b0d991324c275ac6d2978b.png) Once you deploy the _modified_ resulting **XForm** to Central and run it, the duplicate choice names are removed/filter from the nodeset, so the select1 question now only displays the _unique_ entries: ![Screenshot 2025-12-30 at 6.18.15 PM](https://getodk.b-cdn.net/uploads/default/original/3X/f/6/f6a7b5a9a790074a22cc886586a8ce2239d47502.png) This XPath 1.0 preceding axis trick should basically accomplish the equivalent of the XPath 2.0 [distinct-values()](https://www.w3schools.com/xml/xsl_functions.asp) function. I dont know what the relative effort is between in implementing a new _axis_ in javarosa vs adding a new custom _function_, but perhaps it is worth considering given that Enketo already supports this [and I think supporting an actual `preceding` axis in Collect’s XPath processor might open up some rather interesting further tricks we could do with it… @LN? 🤔] Again, this is a bit of a PITA to accomplish right now because you have to bypass pyxform and manually tweak the final XForm, but this node uniqueness/distinct preceding filter approach does work in Enketo today, at least for choice lists. And I suspect if the preceding axis is supported, it could probably be similarly applied to entity nodesets as well.