The longitudinal data collection feature in the form of entities was an exciting feature that opened new possibilities in data collection with ODK.
I was able to design a workflow using entities and maintain a data collection spanning a few months across multiple LGAs in rural, sub-rural and urban areas. Here is the workflow in parts:
Thanks to a fellow ODK Insider and colleague @Certified_Bashir for the help in implementing and maintaining the workflow.
Part 1: The ''Registration'' Form
This is the first contact form with the entity to be collected; it is the form that adds an entity to the entities table, and it uses some values from the submission to create the initial table.
The entities table saves some data points that uniquely identify the entity, e.g., lga, ward, and GPS (which is subsequently used for geofencing), along with variables that could potentially change.
Part 2: The ''Follow-Up Survey'' Form
The forms used subsequently in the study are used to collect similar data over time, which is where ODK entities shine for longitudinal studies and where the workflow begins to take shape. The forms are administered periodically.
- Entity Selection
An entity is selected using its label, which is set using the baseline form; other details are used to verify the entity by pulling some attributes and displaying them to the enumerator. They have to acknowledge to move forward,
- GPS Validation
The next level of validation is a location check. The enumerator has to capture their GPS location. Then, using the location captured on baseline from the entities, the points are compared using the distance function; every follow-up form uses this validation check before being administered. This ensures that the enumerator will have access to the form only when in the designed structure.
- Comparison of variables
Thanks to the forms having access to the entities, another important aspect of the workflow is the ability to display previous values to the enumerator, which allows them to provide context if/when there are changes, using the lookup function. A section is dedicated to pulling these data elements and then referencing them on the label of similar data points.
- Entity Update
After submission, each form creates or updates an entity variable. Most of the variables that were compared are then updated, an example of such variable is the number of people who spent the night in the household.
- Entity polygon
The entities worked on have a geographical footprint; their sizes differ, some significantly larger than others. With the help of entities, a separate form that selects the entity, a geoshape question is used to capture the shape of the entity and update the variable on the entity table.
Using the recently updated geofence function, the point captured is validated to ensure it falls within the polygon, which accommodates all entity sizes.
Part 3: ODK Setup and Data Flow
Entity registration form
survey sheet
| type | name | label | save_to | calculation |
|---|---|---|---|---|
| select_one lga | lga | Select lga | lga | |
| select_one ward | ward | Select ward | ward | |
| select_one structures | structure | Select structure | structure_name | |
| calculate | structure_label | jr:choice-name(${structure},'${structure}') |
||
| geopoint | gps_location | Capture the location within the main structure | location | |
| select_one supervisor | supervisor | Select your supervisor code | last_supervisor | |
| date | reg_date | Registration date | first_visit | |
| calculation | date | last_visit | ${reg_date} |
|
| begin_group | section_a | |||
| • | Section A | |||
| • | ||||
| • | ||||
| end_group |
choices sheet
| list_name | name | label |
|---|---|---|
| structures | structure_1 | Structure 1 |
| structures | structure_2 | Structure 2 |
| structures | structure_3 | Structure 3 |
| supervisor | supervisor_1 | Supervisor 1 |
| supervisor | supervisor_2 | Supervisor 2 |
| supervisor | supervisor_3 | Supervisor 3 |
entities sheet
| list_name | label |
|---|---|
| structures_list | ${structure_label} |
The form above creates an entity table called structure_list. The label of the structure is returned and used as the label for the entity added. Each submission adds an entity with its label and the following: lga, ward, structure_name, structure_label, location, last_supervisor, first_visit and last_visit
Follow-up form
survey sheet
| type | name | label | calculation | relevant | save_to |
|---|---|---|---|---|---|
| select_one_from_file structures_list.csv | structure | Select structure | |||
| calculate | structure_label | instance('structures_list')/root/item[name = ${structure}]/structure_label |
|||
| calculate | lga | instance('structures_list')/root/item[name = ${structure}]/lga |
|||
| calculate | ward | instance('structures_list')/root/item[name = ${structure}]/ward |
|||
| note | Confirm structuredetails: Structure: ${structure_label} LGA: ${lga} Ward: ${ward} | ||||
| calculate | instance('structures_list')/root/item[name = ${structure}]/location |
||||
| geopoint | structure_gps | Capture the GPS coordinates of the structure. | |||
| calculate | gps | round(distance(concat(${structure_gps},;,${gps}),2) |
|||
| note | distance | Your distance to the selected structure is ${distance} meters. You are out of range | ${distance} > 50 | ||
| begin_group | section_a | Section A | ${distance} <= 50 | ||
| • | |||||
| • | |||||
| • | |||||
| select_one supervisor | supervisor | Select your supervisor code | last_supervisor | ||
| date | visit_date | Select the date of visit | last_visit | ||
| end_group |
entities sheet
| list_name | name | update_if |
|---|---|---|
| structures_list | ${structure} | true() |
There are multiple follow-up forms, and all follow a similar structure to that above. For each of the follow-up forms, the entity is selected, some attributes are pulled and displayed to the user to verify the entity. The location is also pulled and compared to the supervisor's current location. For these follow-up forms, they need to be within a 50-meter radius of the structure to proceed. The entity is also updated with the name of the supervisor and the data visited, which is possible by specifying true() under the update if column on the entities sheet.
If they are out of the 50-meter radius, they receive an error message and the distance between them and the facility, which also helps to let them know if they are close to the structure or not.
Images Form
Most of the data points collected require an image to be captured for verification and tracking over time.
A separate form was created and deployed alongside a corresponding follow-up form, another supervisor was responsible for capturing the images as the main follow-up form is administered.
This form would update the entity table with the name of the supervisor who administered the image tool.
Polygon form
survey sheet
| type | name | label | calculation | save_to |
|---|---|---|---|---|
| select_one_from_file structures_list.csv | structure | Select structure | ||
| calculate | structure_label | instance('structures_list')/root/item[name = ${structure}]/structure_label |
||
| calculate | lga | instance('structures_list')/root/item[name = ${structure}]/lga |
||
| calculate | ward | instance('structures_list')/root/item[name = ${structure}]/ward |
||
| select_one yn | structure_fenced | Is the structure fenced | fenced | |
| select_one yn | gate | Does the structure have a gate | gate | |
| note | The next question will allow you to capture the shape of the structure. Follow the steps below to capture the shape of the facility. 1. Click on 'Get polygon' 2. Click on Add Location 3. Select Manual location recording 4. Move to the 4 corners (or more) of the structure and record a point at each corner 5. When you are done, save it using the floppy disk icon at the bottom right | |||
| geoshape | structure_polygon | Trace the perimeter of the facility | structure_shape | |
| date | visit_date | Select the date of visit | last_visit | |
| select_one supervisor | supervisor | Select your supervisor code | polygonization_by |
choices
| list_name | name | label |
|---|---|---|
| yn | yes | Yes |
| yn | no | No |
entities
| list_name | entity_id | update_if |
|---|---|---|
| structures_list | ${structure} | true() |
The form above is used to collect the polygon of the structure, some are fenced, making it easier to pick the points for the perimeter, others are not. The supervisors have to rely on the staff in those structures or the people of the community.
With the polygon of each structure stored, now the validation is more accurate, ensuring that a supervisor is within the perimeter of the structure before their forms are enabled.
The form below shows an updated version of the GPS validation using a polygon instead of the radius.
| type | name | label | calculation | relevant |
|---|---|---|---|---|
| calculate | structure_polygon | instance('structures_list')/root/item[name = ${structure}]/structure_shape |
||
| geopoint | gps | Capture the GPS coordinates of the structure. | ||
| calculate | distance | round(distance(concat(${structure_gps},;,${gps}),2) |
||
| note | Your distance to the selected structure is about ${distance} meters. You are out of range | not(geofence(${gps}, ${structure_shape})) | ||
| begin_group | section_a | Section A | geofence(${gps}, ${structure_shape}) | |
| • | ||||
| • | ||||
| • | ||||
| end_group |
Part 4: Additional forms and features
With the possibility of connecting a single table in multiple forms with entities, other features were given birth to. One of the notable ones was the entity locator.
- Entity Locator Form
The structures in the study are not usually easily located, especially ones in semi-urban and rural areas, a facility locator form that pulls the location and extracts the longitude and latitude, then dynamically creates a google maps URL that opens a navigator to the structure to guide the enumerator or a supervisor to the structure - (I wanted to use the ex: to open the google maps or another maps application on the device but couldn't in time, so I went with this work around, should be a section under support I guess)
Here is the structure of the form
| type | name | label | calculation | relevant |
|---|---|---|---|---|
| select_one_from_file structures_list.csv | structure | Select structure | ||
| calculate | structure_label | instance('structures_list')/root/item[name = ${structure}]/structure_label |
||
| calculate | lga | instance('structures_list')/root/item[name = ${structure}]/lga |
||
| calculate | ward | instance('structures_list')/root/item[name = ${structure}]/ward |
||
| note | Confirm structuredetails: Structure: ${structure_label} LGA: ${lga} Ward: ${ward} | |||
| geopoint | gps | Capture the GPS coordinates of the structure. | ||
| calculate | structure_gps | instance('structures_list')/root/item[name = ${structure}]/location |
||
| calculate | lat | selected-at(${structure_gps},0) |
||
| calculate | long | selected-at(${structure_gps},1) |
||
| calculate | distance | round(distance(concat(${structure_gps},;,${gps}),2) |
||
| note | Your distance to the selected structure is about ${distance} meters. | ${gps}!="" | ||
| calculate | lat_long | concat(number(${lat}),,,number(${long})) |
||
| calculate | gps_url | concat(https://www.google.com/maps/search/?api=1&query=,${lat_long}) |
||
| note | Click to see the [location in maps] (${gps_url}) | |||
Just in time, as ODK Central enabled deletion of submissions, any submission from that form was deleted, as it was only required to be a utility form for navigation, not data collection.
- Entity Update Form
This feature was not implemented in the study, but was a workaround to a limitation we faced with entities.
Even though ODK Central allowed for editing submissions, any edit made on the form was not reflected in the entity table [add ODK docs highlighting that limitation]. The entity update form will be an EXACT replica of the form used in any follow-up visits, but without any constraints as the finalised form used in the field, no required questions, no relevant questions, and no location validation. This form is only available to the supervisory team and managers.
When an edition needs to be made on the entity, the supervisor picks the form that was used with the error, selects the entity, populates only the questions that need to be edited, and then submits. This will then update the entity with the only question answered.
Part 5: Challenges and Solutions
The workflow came with some challenges, we were able to find a workaround for most of them and others, we adopted them the way they were:
- Editing a form does not update the entities table. We employed the workaround of editing both simultaneously, and later had the idea of using a form dedicated to update, as highlighted in part 4; although that wasn't deployed in the live data collection
- Enumerators found it difficult to locate the structures as they were constantly rotated to reduce familiarity bias; the facility locator tool served as a guide to access the structures
- Although the geofencing feature ensured the form opened within the structure, it does not prevent filling the form outside the fence after accessing it. The forms were then opened within a window during the expected times of reporting, except for the entity locator
- The images captured for each data point made the tool longer and delayed submission due to the sizes of the images; the tools were then separated into 2 -
follow-up tool A & follow-up images tool A