Form linking

Ideas
1

This post describes a feature which exists in JHU Collect, which was forked from ODK Collect some time ago. At the time of this writing, JHU Collect is 95 commits ahead and 2013 commits behind ODK Collect. The goal would be to get this feature into ODK collect.

What is the general goal of the feature?
The goal of this feature is to allow forms that are related to each other to be "linked". The most important feature of this "linking" is that some data should be shareable between them.

What are some example use cases for this feature?
A common use case is a household questionnaire that would be connected to one or more household member questionnaires.

What can you contribute to making this feature a reality?
JHU Collect is used by PMA2020. Myself and @jpringle are frequent users. However, I do not know what our capacity is / will be in the near future for contributing towards this. We are currently transitioning to a new grant. I believe we will know more in the coming months about what our capacity for this under the new grant would be.

Technical description and further details
In the source file there is a save_form and save_instance binding. The corresponding fields exist in the source XlsForm.

A standard use case is the following:

  1. The parent form is a used to fill in a household roster to identify people for further follow-up.
  2. Children forms are generated for those who are identified, and certain data is pre-populated in the children forms.

PMA2020 uses this scheme to interview a household (household questionnaire) and identify eligible females for later questioning (female questionnaire) on topics like reproductive health. The household questionnaire might have the following fields:

  • 1 hidden string type (similar to a calculate), with the value of fq-v1 in the save_form field of the XlsForm. The value "fq-v1" represents the form_id in the settings worksheet of the child form. In this case, it would be short for "female questionnaire, version 1".
  • 1+ fields (I believe any type is possible), with values of the structure XML_ROOT/PATH/TO/NODE in the save_instance field of the XlsForm. The "XML_ROOT" would be found in the xml_root field of the settings worksheet of the child form. If, for example, my "XML_ROOT" was "FQ" (female questionnaire), and I had a question with the name first_name in a group called confirm_info, the following value would be entered in save_instance: /FQ/confirm_info/first_name.

Given this setup, we would expect the following behavior in JHU Collect:

  1. An interviewer would enter a household roster in the household questionnaire. Upon saving it, she would receive a toast notification saying that the form had been saved, and 1+ other forms were created.
  2. The interviewer would navigate to the new forms or edit forms section. There, she would see a single item that she could select which, in ODK Collect, would simply be the name of the form, which she would press her finger on to enter the form. In JHU Collect, tapping this option would expand into a list of the parent form (household questionnaire) and any child forms (female questionnaire(s)).
  3. She opens a child form.
  4. When navigating to the questions for which the path to that question was designated in the save_instance in the parent form, the question would be pre-populated with whatever the value was in the parent form (household questionnaire).
  5. If she changed that information in the child form (female questionnaire), the change would be synced with the corresponding value in the parent form.
  6. Once completing all of these linked forms (parent form and all child forms), she would be able to submit to the server.

Some features of form linking are the following:

  1. The parent form may contain eligibility logic to create the child form. If in the parent form, answers are changed such an existing child form is no longer eligible, the child form is deleted.
  2. The child form may contain eligibility logic such that if further information shows ineligibility, the child form will be deleted.
  3. The parent form determines the instance name of the generated forms upon creation.
    • Example: PMA2020 uses this so that in the instance chooser list, the instance name shows the age and name of the eligible female.
  4. The parent form may send values to the child form. These values are linked, so that an update in one form triggers an update in the other form.
    • Example: PMA2020 uses this so that name and age can be updated in the female form.
  5. In the instance chooser lists for "edit saved form," "send finalized form," "view sent form," and "delete saved form," linked instances are shown together.
  6. In "send finalized form," linked instances are sent to the server together.
  7. In "delete saved form," deleting a parent instance deletes all child forms

We at PMA2020 have used the form linking functionality only in the context of repeat groups. If a single repeat group has an associated child form and the repeat group is deleted, then the child form is deleted.

In the future, it may be useful to allow different child forms with different eligibility and to allow form linking outside of repeat groups.

2

this feature is amazing and will solve a lot of issues users raise in this platform.

3

Thanks for posting this, @Joseph_E_Flack_IV!

I had written up a long response with lots of over-engineered approaches, but then I asked for @LN's feedback and she asked me why repeats wouldn't work. And I think she's right. :sweat_smile:

It seems like repeats have the primitives that you need. You can create children, they can be named, the values sync, they get sent at once, etc. Is there a reason why they wouldn't work?

Is it the UI on the client side? Is it the way it's stored in Aggregate? Is it the form design?

4

@yanokwa, I agree that the repeats solution is a way to solve this problem. The reason we prefer the linked form approach is because it helps to separate things. Filling in a household roster within a household questionniare and interviewing eligible people for follow up are separate tasks.

If for example we had four follow-up interviews to do after finishing the household questionnaire (we would finish the head of household portion first), then we would need to navigate hierarchy view correctly to get to the correct repeat group. With advancing forward or backward screen to screen, it is possible to end up in the wrong repeat group. We see this as error-prone for the enumerators we hire. To answer your question, the reason we don't use repeat groups is a combination of form design and the UI in ODK Collect. The way data is stored in Aggregate does not factor into our reasoning.

2 Likes
5

Thanks for the feedback, Yaw. From a conceptual standpoint, you are correct. But I concur with James that this is a use case based served by a different approach. Separating things out as forms rather than repeat groups seems to be better UX and greatly decreases error rates that we would see. I don't have anything quantitative regarding those error rates, but this is the feedback I get from our staff who interact with the interviewers more directly, and I've firsthand seen the trouble that a lot of our interviewers have with navigating form hierarchies.

2 Likes
6

Hello @yanokwa and others,

In our previous discussions, I believe we left it open ended as to how a "form linking" feature (such as currently in JHU Collect) might get into ODK Collect.

My latest news update is that we have decided to allocate the labor internally. It could change, but it's probable that it would be James and/or I that implement this feature.

Basically we'd make a new feature branch from our new fork of the ODK Collect trunk. And then we'd re-add / re-implement the form linking features that are in JHU Collect.

This is sort of self-interested though. Basically we want to upgrade to the latest version of ODK Collect. The easiest way to do this is simply to put our features in ODK Collect and not maintain an active fork anymore

That being said, I'd like to ask a couple of questions:

  1. Is this still something that the ODK steering committee would approve? The addition of this feature?
  2. Are there any constraints / requirements that we should know about in advance? If so, is this something to discuss over text or over voice?
1 Like
7

I want to UP this

form linking adds other benefits:

  • slice down big questionnaire to increase performance
  • create reusable questionnaire snippet that ensure coherence between questionnaire / avoid question duplication
  • create much easier data to process (do not need to handle the number of repeats)
  • make data processing workflow much easier: ex what ever the "main" questionnaire use a death report could have its own processing workflow