Include list of images from CSV as media filenames in form

[seewhy]

1. What is the issue? Please be detailed.
I am seeking to include images in my form based on a list in a csv. This is in the hope of being able to update the csv and upload the relevant media to Central if/when that list of images changes. This would be instead of having to include the list of images / filenames in the choices sheet (using the 'image' column). I've already fallen over with 'special columns' so maybe I'm making a similar mistake.

I asked KAPA the following question and very confidently it came up with the answer below:

how can I include image choices from csv to display in the form with the images being stored in the media folder Okay, not the most eloquent of questions, I'm not very familiar with AI... But KAPA says:

You can do this with an external CSV and the image column in the choices sheet.

1. In your external CSV

Add a column that contains the image filenames (e.g. img_file):
name,label,img_file
a,Option A,a.jpg
b,Option B,b.jpg
c,Option C,c.jpg

Put the actual image files (a.jpg, b.jpg, c.jpg, …) in the form’s media folder (e.g. myform-media/ in Collect, or upload them as media attachments in Central). [Form styling]
2. In your XLSForm – choices sheet

In the image column, put the CSV column name that holds the filenames (not the filenames themselves):
choices

list_name	name	label	image
mylist	a	A	img_file
mylist	b	B	img_file
mylist	c	C	img_file

This tells ODK to pull the image filename from the img_file column in the CSV for each choice. [Dynamic media from CSV; Video in choice list from CSV]
3. In your XLSForm – survey sheet

Use a select question that draws from that list (and uses the CSV via search() or select_one_from_file, as you already do):
survey

type	name	label	appearance
select_one mylist	my_choice	Select an option

As long as:

the CSV is correctly referenced (e.g. via search() / select_one_from_file), and
all image files named in the CSV are present in the form’s media folder / uploaded as media,

the choices will display with images pulled from the CSV. [Including media in choices]

2. What steps can we take to reproduce this issue?
I tried to follow the answer and created a form (I only picked the top 3 images to put in the choices sheet)
ImagesFromCSV.xlsx (106.5 KB)

and imagelist.csv
imagelist.csv (909 Bytes)

How can I upload the images in the image list? - Central doesn't recognise the images when I try to drag them to the draft form (see screenshot). Maybe I'm doing something fundamentally stupid (wouldn't be the first time!) but I thought I was following KAPA. If it is suggesting that we need to essentially duplicate the lists, but not include the filename in the choices sheet I'm not sure I see the advantage - and it doesn't appear to work. And it's not what I'm hoping to do. Maybe it is 'logical' in that the xml needs to include every option within the choices sheet, and the external CSV doesn't get processed by Central to generate the options?

3. What have you tried to fix the issue?
Have a look at the form...

I can 'fix' this by putting the imagelist in the choices sheet - Central then asks for all the files, but that defeats my purpose as I need to update the form definition. I know I can do that using excel formulae but I was looking for another way that doesn't require the xlsx to be updated and resubmitted to Central...

4. Upload any forms or screenshots you can share publicly below.

[ahblake]

This is a bugbear of mine too, having to spill filenames from entity lists into all forms that use them, to get those media uploaded for use, with definition updates for every media change.

Here's some forum history where others have hit the same issue:

Upload media images via odk central that are not part of xlsform choice list/survey directly

1. What is the issue? Please be detailed. I have designed a xlsform where I display image for note based on the previous selection. So the I upload media file through central, I can't see the list of those images. as they are loaded through dynamic variable. how can i push the those images in media folder of form 2. What steps can we take to reproduce this issue? 3. What have you tried to fix the issue? If i use google drive and upload form, I can simply copy the files in media folder and …

Using images with multiple-select from CSV

I wanted to revisit this topic to see if it's affecting other people. My use case: I have large reference CSVs that contain a list of items and some data about them, plus media filenames I want to show these media for reference (via a filtered select and image & big-image and tap to enlarge) or deliver them as dynamic images (ideally also with supressed 'take picture' & 'choose image' buttons) for annotation. Currently in order to do this, I need to also include an unused choice list that i…

Pull data with photos

@TobiasMcNulty This is a pain point for me as well, I have CSVs that reference many images, so also need to force them into Central. Here's what I do: Add a choice list, eg ref_images to your form, then under the image (and perhaps big-image) column put the filename - I use formulas to build the label / image / big-image values from the name. important : Add a select_one to the form at the very start/end that references this choice list, but has relevance set to false. If you don't do this, t…

Retrieving Dynamic Media from Entity

1. What is the issue? Please be detailed. Summary Essentially my question is how do I create a select multiple or select one question that looks like this: https://docs.getodk.org/form-question-types/#select-widget-with-no-buttons-appearance but uses media collected in a different form using entities? It is important to note that when tapping the answer shown as an image, an audio should play associated with that image. I only need this to work in Collect and don't need it to work in Enketo. …

Dynamic media - video in choice list from dynamic CSV

What is your primary reason for wanting to use an external list in this case? How many choices do you have? We don't currently really support images this way either because Central does not have a way of attaching arbitrary media files to a form. You can see how folks like @ahblake overcome this limitation at Pull data with photos - #6 by ahblake What we'd like to do longer term is have a concept of media attached to entities. That will make it possible to keep the media in sync with their co…

[seewhy]

Yep, I've come across most of these and not always clear whether things are solved or in the pipeline (partly because the trail goes cold, partly because things develop without people revisiting the posts - of which I am equally guilty), which is why I tried KAPA...

So I guess we're still where we are, but KAPA seems to disagree. I certainly couldn't get the 'answer' to work as described, although I don't think it's quite what I was looking to do - I'm closer to your bugbear, although not actually trying to integrate entities at this stage.

Thanks as always for your encyclopaedic input (or would that be wikipaedic?)

[ahblake]

Did kapa provide links to back up that response? It might indicate where that came from (docs, a forum post etc)

I agree the response seems both wrong and no more useful than just including a list of filenames in a choice list

[seewhy]

It did include links, but they don't seem to have transposed in the copied text above. One was from the docs (https://docs.getodk.org/form-question-types/#including-media-files-in-choices) and the others were forum posts, [Dynamic media from CSV; Video in choice list from CSV] Dynamic media - video in choice list from dynamic CSV

Glad it's not just me that thinks the answer might not be entirely accurate... It wasn't meant to make things up!

[ahblake]

Ok, so it looks like it's coming from the xlsform.org page via that forum post, it's not totally out of date (it includes the ODK xlsxform template from 2023), but it's not exactly current and complete.

@Grzesiek2010

Dynamic selects from pre-loaded data is a functionality that was implemented a long time ago (10 years) and it was developed by Dobility, Inc. (as part of SurveyCTO) according to the copyright headers. As far as I remember we have never touched that feature and generally it exists totally outside of the rest of the form spec.

@LN

We don't currently really support images this way either because Central does not have a way of attaching arbitrary media files to a form.

Just for fun (because I don't use search() appearance and this way of referencing CSVs), I downloaded the example form & CSV from xlsform.org and added a column as described to the csv (myimage with a.jpg, b.jpg, c.jpg). Knowing that Central wouldn't ask for these files, I added a choice list fakelist that had one choice with a.jpg under image and called that list with a non relevant select in the form.

In your choices worksheet row, you may also include a .csv column name in the image column. If you do, the image filename to use will be pulled from the specified .csv column.

Outcome - much the same as you found, with a Webforms bonus;

• Central called for

  • column name myimage as an attachment, type image (I added an image against it just to see... )
  • a.jpg type image ( uploaded)
  • fruits.csv type Data File (uploaded)

• In Enketo the form loaded with an autocomplete dropdown type appearance which could never display an image, but the selections for the question were ... and name, not the CSV content (the documentation does state it won't work)
  

  

• In Webforms only name is an option, and the csv column name image appears!
  

  

• In Collect the form loaded with a basic select appearance (single column) and a.jpg did appear and the others showed the File PATH is missing message. The CSV column name image doesn't show anywhere.
  

  
  • I added a second select removing the search appearance to see if anything would change, but it won't validate;

The XLSForm could not be converted: Question 'fruit' uses 'search()', and its select type references the choice list name 'fruits'. This choice list name is referenced by at least one other question that is not using 'search()', which will not work: 'fruit2'. Either 1) use 'search()' for all questions using this choice list name, or 2) use a different choice list name for the question using 'search()'.

So, my take on that part of the documentation is: It was written pre Central where you could upload whatever you like, so if you used Collect and not Enketo/Webforms, and did correctly upload all the filenames to Aggregate (or manually transfer the files to the device definition media folder), it 'works' (for some values of work). The search() approach is only for Collect, and only if you have an absolutely massive dataset that won't be performant via select_one_from_file.

And @seewhy for your case, the only current way to get it to work is to include the filenames in your xlsxform as well as the CSV, and if you don't want to modify the form, you'll have to include a onetime list like 0001.jpg 0002.jpg ... 9999.jpg and assign those to rows in your CSV - but you'll still have to update the definition in Central in order to modify the attachments so this isn't much better... You might also want to the kapa response to downvote it to oblivion for future queries.

BTW When I mentioned entity media earlier, this describes me perfectly, and I think would also solve your issue :

@LN

Generally, when folks want to attach CSVs that reference media, the CSVs really represent lists of entities that have some broader lifecycle in their project so we'd like to address the media need at the entities level. That would mean we could, for example, have media captured with one form automatically available in other forms representing other workflow steps.

[seewhy]

Well, I am seriously impressed! I'm coming straight to you next time

Thanks for chasing that down and trying the various permutations. So this is the answer, but not the solution to my problems...

I am being careful what I wish for in the Entity realm - I think it could place a massive load on servers and bandwidth. Although it wouldn't apply to you @ahblake because you are diligent (see above!), I suspect that we could end up responsible for burning through a data centre or two if people were able to include media in their entity lists - we know already that there are requests for VERY large lists, and I don't know how many people 'prune' their lists or how often. Maybe if there was a reasonable limit on the number of entities that could have associated media I might be less averse.

For now, I'll feed my images through the choices sheet...

[LN]

Sorry neither Kapa nor the forum threads were all that helpful, @seewhy

@seewhy

Thanks as always for your encyclopaedic input (or would that be wikipaedic?)

+1, always amazing, @ahblake!

@seewhy

I suspect that we could end up responsible for burning through a data centre or two if people were able to include media in their entity lists

We share your concern! And really a list of images from CSV is the same as images attached to Entities -- as soon as we move into a more dynamic way of specifying images we get to a place where users might not realize just how many images they're transferring. This is why we've reluctant to go deep in this before we have robust ways to limit what Entities individual users see, Entity archiving, progressive updates and other functionality aimed at reducing the burden you describe.

@seewhy

not always clear whether things are solved or in the pipeline

We'll try to do better with this -- we have so much good stuff to build these days that we're finding it hard to keep up with the forum!

[seewhy]

Thanks for your thoughts - and I'm glad it's not just me that worries about the unintended consequences of new toys

@LN

We'll try to do better with this -- we have so much good stuff to build these days that we're finding it hard to keep up with the forum!

It is good stuff too!

To be fair I think that some of that needs to be on us, and was in part why I posted - anyone hoping to do the same might now get deflected earlier. So we all need to reflect on when our own posts go out of date or are solved - guilty as charged. It's interesting to see when old threads that have 'gone cold' get picked up, hoping for a solution to have been quietly found... Can Kapa be tasked with cleaning up some dead or open ends? Not sure whether it would have real value though. And sometimes those are good reminders when they pop up...

[LN]

@seewhy

Can Kapa be tasked with cleaning up some dead or open ends?

Not as-is but that's an interesting idea for an extension.

One thing to note is that we only train the Kapa bot on threads that have an answer or were started by the core team. If a thread was started by a community member and was never marked as resolved, Kapa doesn't know about it at all. We do this as a way to control quality but it does make for some blind spots. For example, it's not entirely clear whether we should consider this thread resolved but I think marking @ahblake's answer at Include list of images from CSV as media filenames in form - #6 by ahblake as the solution is helpful.