The topic of this maundering post is ways of visualizing what we are trying to accomplish with an ODK questionnaire form.
My goal is to share two such examples and to spark conversation re: What tools & workflows do you use to visualize form contents and flow?
At the June 2023 ODK Summit I shared example visualizations of the logic and flow of two ODK forms that our firm has developed in collaboration with the American Red Cross for the 5-Point Plan project. (link to first; link to second) The XLSforms were coded by my teammate, Mary Kay Trimner (@marykaytrimner). The diagrams were conceived and constructed by my teammate Caitlin Clary @cclary. The flow of the household interview forms was developed and refined over several years and several large pilot surveys involving a dozen or so contributors from the Red Cross and my team at Biostat Global Consulting. The printed visualizations that I hung up in the room at the Summit are 4-5 feet tall because the forms are complex.
I’m not making a strong claim that these forms are the best way to collect these data nor that these diagrams are the best way to visualize the workflow. That could also be accomplished by a series of smaller figures, starting with a simple overview and then a series of smaller figures with details in several logical chunks. But both the forms and the diagrams have been quite useful, so I offer them as conversation starters.
We’ve used these diagrams (and previous versions) for several purposes: a) to agree amongst our own team how the interview should flow; b) to communicate with our Red Cross client team about that flow; c) to jointly think through the detailed implications of suggested changes to the flow; and d) to test the form, moving intentionally through every one of the many possible paths that the interview could take through the form.
These diagrams are focused on representing the user experience of navigating through the form’s many paths, and so they elide some of the complexity of what’s going on under the hood in the XLSform.
Our diagrams were built using Microsoft PowerPoint, which has several strengths and at least one limitation.
Strengths:
- Everyone on my team can work with PowerPoint graphics – we are already up the learning curve – and there’s no need to learn and remember yet another set of syntax.
- PowerPoint is extremely flexible, giving the user control over the canvas size and placement, shape, size, color, font, and annotations used to convey meaning.
- It can generate a high-quality PDF that can be printed on very large form sheets or poster material.
Limitations:
- We’re not aware of many software tools that would allow users to use syntax to generate PowerPoint diagrams like the examples we’ve shown, or even a rough first draft of such a diagram (e.g. putting blurbs of text into text boxes and arranging them roughly on the canvas). The .pptx file format is documented, and a few tools give programmers an option of building a rudimentary layout, but the ability is far from ubiquitous.
There are other tools designed explicitly for drawing diagrams from syntax. I’ve spent some hours since the Summit playing with https://mermaid.js.org/. In fact, I created a semi faithful reproduction of one of Cait’s diagrams, using mermaid. (link) You can render it by copying and pasting the source into https://mermaid.live. (Be sure to toggle the ‘pan and zoom’ setting on, and click the full screen button to render a version that you can zoom into. An exported .png is here.)
I find mermaid to be unsatisfactory for several reasons.
Strengths:
- Good for small diagrams
- Extensive but not exhaustive set of options for shapes and connectors
Limitations:
- Need to learn/remember/refer to a new set of syntax.
- User controls the content but has somewhat limited control over placement.
- Not possible (as far as I can see) to use numerous text shading conventions in a single box.
- Not possible (as far as I can see) to bring in a .png as a diagram element. (e.g., the various types of 5PP images on our diagram to show how the house should be chalk-marked, depending on interview disposition)
- Larger diagrams do not (as far as I can see) always render well.
At the Summit, I asked if anyone has any automated tools for pulling out various fields from an XLSform document and visualizing them. I repeat the question now, here, to this larger audience: Have you found or developed any tools for helping to diagram aspects of your ODK forms?
It sounded like several people had done small batches of hand-crafted diagrams using several tools, but had not built something generic for making finished diagrams, or -- and this seems more achievable -- for simply putting the elements of a form onto a canvas in preparation for crafting a useful diagram.
Modest future endeavor
Cait tells me that an R package named officer lets you use syntax to write to a PowerPoint canvas. We are starting a small self-funded project to develop a simple generic tool to extract key form elements and arrange them on a slide to serve as Step 1 in preparation for visualization. Because the tool won’t know your ultimate aims (or your esoteric formatting preferences) it won’t connect or color (much) but will simply fill the canvas with rows and columns of text boxes holding the form elements and then you can connect and annotate and re-arrange to suit your needs.
We will post the code on GitHub and share an announcement and a few examples when we have a first draft ready to share.
In the meantime, I would love to have you post examples of useful visualizations you have made of your forms and share how you made them and what purposes they served. And if you have other tools that you use to write from syntax to PowerPoint graphics, I would be interested to hear about those, as well.
Respectfully,
-Dale