Tag/link pictures with defined field

one of the things in which I find ODK Collect a bit weak -IMO- is the relation between the data collection form and the collected images. The "link" thing is a bit clunky (as already discussed here), even more when you want to manage the data extenally (e.g. in R) because you don't have -or, at least, I didn't find yet- a unique key contained in the link that relates directly to the image name enabling a merge between the two sources.

I definitely don't have knowledge of how google works for the creation of such links but, if it was possible, I would suggest to add a form field (e.g. survey plot ID) to the instance name (form name_date_time-plot_ID.jpg).
Another ossibility might be to allow to add the field info as a visible tag in the picture (e.g. bottom-left corner).

IMO, these solutions would allow for a quicker visual check on the pics instead of checking the long and complicated uuids or link by link.

short update: I found this thread (https://github.com/getodk/xforms-spec/issues/128) only now but it seems stuck since quite a while

Hi @spono,

Since you mention ODK Collect, the data structure of submissions while still on the device is one folder for each submission, with attachments nested within. So you have all media attachments neatly kept together with the original submission XML.

Once you upload the submissions to ODK Central, the submissions will contain the image (or other media) filename as value of a media field. Unless there are duplicate filenames, this is all the linkage you need.

Since you mention R, I assume you use ruODK::odata_submission_get. Here, the media will be downloaded automatically for you to a destination folder, and the value (initially just the filename) will be adjusted to the absolute path to that file. The adjustment of the media filenames is an ruODK thing.

Caveat: odata_submission_get's default download destination for media files is here::here("media"), which depends on which user runs that script.
If you run odata_submission_get yourself from within an RStudio project, the destination will be relative to your project's workdir.
If you run the download e.g. from a cronjob (example) inside a Docker image (example), the script will be run by user root and the default destination will be /root/media.

I will give you two examples of what to do with the media files.

Embed photos in a table

You can subsequently use that path to link directly to the image, e.g. embedded in a reactable.
You can see a real-world implementation of this here.

I will annotate my example a bit, although this is out of scope of the original question:

  • basename(value) is the filename of the actual submission.
  • coldef_img builds a clickable image thumbnail.
  • This example assumes a dataframe mydata with photos in columns photo1 and photo2.
  • I rebuild the image filepaths with fs::path("media", basename(value)) as relative paths so they work in a Shiny app.
  • In a Shiny app, you must set the resourcePath or Shiny will not find the local images even if the path is right. Example: add_resource_path("media", app_sys("inst/media")) means:
    • Relative to the workdir, media attachment files are stored in a folder inst/media. Using inst is an R package convention to provide non-standard file resources.
    • Files in inst/media are now available to the Shiny app under the relative path media.
coldef_img <- reactable::colDef(
    cell = function(value) {
      img_src <- fs::path("media", basename(value))
      image <- tags$img(src = img_src, width = 100, height = 100)
      tags$a(href = img_src, target = "_", image)
    }
  )

  mydata %>% 
    reactable::reactable(
    searchable = TRUE,
    sortable = TRUE,
    filterable = TRUE,
    columns = list(
      photo1 = coldef_img,
      photo2 = coldef_img
     # your photos here
    )
  )

Embed photos in a map popup

See https://docs.ropensci.org/ruODK/articles/odata-api.html#map for a working example.
You see that the work to link media attachments as images in the map popup happens in the definition of the popup.

So I agree, turning a spreadsheet or table with file paths and a bunch of media files in a folder into something visually appealing takes some effort.

Of course, all of the above is biased through my own experiences and work context.
I'd be interested to hear your story!
Do I understand you correctly that you want more natural foreign keys in the media filenames, such as submission ID?
How would a good use case look in your context?

4 Likes

wow...that's quite a lot to digest: a HUGE thank for such a reply!

(sorry but I was out of the office)

actually, I'm finding myself in my first experience of managing "big" amounts of image-data coming from ODK...and till now I always did in a pretty rough way (basically: by hand). I have no experience with Central or ruODK (I found it out only a couple of weeks ago) because 99% of times I use ODK for myself or really small shared projects.

My pretty basic idea [you've gone well beyond my imagination..! :wink: ] is to have the chance of choosing/compose the name of pics while they're memorised in Drive...thing that -as I understand- is handled by Central once data is dowloaded through it, right?
But might be already memorised using a field from a form? or, at that stage, depends from Android/Google?

Interesting!
So since you mention (Google) Drive and Central, which one do you use?

If you use Central, the media questions will contain as value the filename of the media attachment. So if question "sample_photo" captures an image, the value might be something like "DSCN0001.jpg".

The easiest way for you to get to the submission data is the manual export.
Once you export the submissions from ODK Central as ZIP (including media) and extract the ZIP archive, you will find the submission data in a CSV file called {form-id}.csv, and the media attachments in a folder called "media". The CSV will have a column "sample_photo", and you will find the photos in the "media" folder.
If you want to build some kind of visualisation which requires the photo files in the media folder to be linked to the submission record (a row in the CSV), you can modify the file path in the submission data by prepending the media folder, e.g. change "DSCN0001.jpg" to "media/DSCN0001.jpg" e.g. using an Excel frmula. (Mind path delimiters - Windows \ vs Unix and Mac / and absolute vs relative paths).

The relevant docs on submission export are here for ODK Central and ruODK::submission_export.

I hope this sheds some light on how to link the photos to the submissions. I believe the above will work for your intended use cases without the need for custom (regognisable) file names.

Once you have a setup that works nicely for you, feel free to share it here in the showcase category so others can learn from it!

-at least till now- I use Google Drive...that's why I didn't know about the possibilities using Central...I'll have to take a look, even though too require quite an effort for small projects. I'll check it out: thanks!

1 Like