Difference in form behaviour when retrieving data from an entity or from an external csv

When trying to prepopulate data for a select_one question either from an entity or from an external csv I incounter a difference in behaviour I do not understand.

The from "pulldata_from_entity" retrieves the data from an entity (entity.csv) in the field "producteur_nationalite" as intended. If no data is available to retrieve (when a new entity is created per example) the user can choose in the dropdown referring to an external csv 'pays.csv'.

This works as intended.

In the form "pulldata_from_csv" I want to achieve the same behaviour, retrieving data not from an entity but from an external csv (external_csv.csv) and, if no data can get retrieved, let the user choose from a list reffering to a different external csv (localites.csv).

The from fails to pull data from "external_csv" to "exploitation_agricole_localite". The dropdown remains always empty.

If the type is "text" the data is pulled without problem. This is also the case if the external csv from which data is pulled from is the same as the one used in "select_one_from_file", even if the type is "select_one_from_file". But this is not what I intend.

I also noticed, that if 'name' in the calculation in the form "pulldata_from_entity":

pulldata('entity','producteur_nationalite','name',${exploitation_code_choix})

is replaced with a specific field of the entity (like e.g.: 'exploitation_code_choix'), the form fails to pulldata from the entity.

Thank you for your support!

pulldata_from_entity.xlsx (667.6 KB)
pulldata_from_csv.xlsx (642.1 KB)

Hi @dast

It's pretty hard to really understand this behavior without having the .CSVs (media file and entity list) at hand, I would request you to please add them here, maybe with the dummy data in case the real data is sensitive?

Hmm, also I had a question, why is "parameters" for the question "exploitation_agricole_label" is mentioned as "value=label" in "pulldata_from_csv"...?

I've had to do this (in my case) with entity lists, as I don't want to return the UUID in my submissions, so instead I have another field 'name-value' which was the name in my pre entity CSVs. (Can cause other complications elsewhere if you are using dependent entity lists...)

@dast without seeing your form plus CSVs (and entity list export to CSV), could it be a mismatch between the csv name, the entity list name (UUID) and the label value? I was trying to use a 'last-saved' value in my form and realised that the last saved value is the one returned by value=another_column and not the UUID/name.

Thanks @MinimalPotato and and @ahblake for your feedback.

I worked on the example forms to illustrate the issue.

This is the form that works well with an entity and the csv exported from the entity:

pulldata_from_entity.xlsx (644.0 KB)
pull_data_from_entity.csv (828 Bytes)
pays.csv (179 Bytes)

And these for the form linked to an external csv which does not work as desired, because the field exploitation_agricole_localite (select_one_from_file localites.csv) is not prepolutated by pulldata() from the external csv.

pulldata_from_csv.xlsx (642.2 KB)
localites.csv (162 Bytes)
external_csv.csv (1.5 KB)

@dast without seeing your form plus CSVs (and entity list export to CSV), could it be a mismatch between the csv name, the entity list name (UUID) and the label value? I was trying to use a 'last-saved' value in my form and realised that the last saved value is the one returned by value=another_column and not the UUID/name.

Yes, it is somehow related to how the field 'name' in the pulldata calculation relates to the entity. But 'name' cannot be used in pulldata() if it relates to an external csv.

I just don't grasp the whole logic behind this to come up with a solution strategy.

I also noted that this form logic only works with pulldata() but not with instance().

Thank for your ideas and best regards, Daniel

The pulldata for exploitation_agricole_localite will return values like 'Village 1' but the values in the localities.csv are like 'Village One' so that's why it doesn't work. Your other pulldata expressions did work in the form as provided (to text fields)

This shows the non-working Localite pulldata, then a working Localite pulldata, and then a replacement of all pulldata with instance lookups. Tested in Enketo and Collect:
external_csv2.csv (1.7 KB)
external_csv_o.csv (1.5 KB)
localites_o.csv (172 Bytes)
localites_a.csv (165 Bytes)
localites2.csv (657 Bytes)
pulldata_from_csv2.xlsx (644.4 KB)

1 Like

Many thanks for your extensive testing and the solution provided @ahblake !

This means, the values from external.csv must exactly match the values in localites.csv?

In our case this would create difficulties in the field as enumerators can add a new village name (if they select "other" from the localites.csv in the form equivalent to external.csv) when registering (in our case) a farmer and the village is not yet registered or the form not yet synchronised with the server. Off-line entities could provide a possible solution.

1 Like

If you want to prepopulate a select, yes, the values must match exactly or it won't select an option. In your case it was a :person_facepalming: Village One != Village 1 that stopped it from working.

If they might not match, you could show the lookup value as a note / in a hint to let the enumerator select the closest match.

Offline entities would let your enumerators register a village name to a 'village' entity list and then immediately use that offline entity village item in another form.

1 Like