Here is the library I referred to. It's pretty robust and is being used in OpenSRP, DHIS2 tracker and I believe CommCare soon.
I've been chatting with @Ivangayton / HOT OSM about how to resume some of the past work around improving the geo widgets that @zestyping was such a key part of, and also develop the ideas for the work into a more cohesive vision. There were community discussions around this at the most recent ODK Convening in late 2019, and a variety of forum discussions. We're just wrapping up on an initial document to describe the aspirations and so it has not been discussed with the TSC yet. The hope was to get additional eyes on the ideas and work with the TSC to figure out the technical feasibility and fit with the ODK project. Then if we can agree on a clear scope and plans, move on to identification of resources to make happen.
EDIT: I think there is a lot of overlap between the above and our proposed roadmap ideas.
As @danbjoseph alludes to, the general space of selecting a geo feature as part of the data collection process is now being taken up more actively by the TSC. The geo future directions doc gives a good overview of the various themes that users have brought up related to all things geo and that the dev team would like to take up. The map-centered form workflow section in that doc is closely related to what is being described here and what @danbjoseph has started fleshing out in the doc linked above.
@Matt_Berg, could you please share a writeup of the specific project/workflow that is triggering this request? A good example of such a writeup is @J_D1’s View GPS point on Map for Blank Forms - #10 by J_D1. I think the biggest decision to make is whether to treat this as a subset of entity-based data collection or not and getting a clear understanding of what workflows need to be supported would really help. Specifically, the first post in this thread suggests attaching the geo features data to a single form. @danbjoseph describes a model where the feature selection happens before form selection (e.g. select a waterpoint and then select what form to fill about it) which means features data has to come from outside the context of a single form.
There are excellent notes from the last TSC call here. @Ukang_a_Dickson brought up various related projects and I imagine you’ve probably synced up (I mention this because @Matt_Berg and @Ukang_a_Dickson are both from Ona).
Since there are a lot of moving pieces in this general space, there would definitely need to be detailed user stories leading to a specification and iteration with the TSC. @Matt_Berg, what is your timeline and would you be up for working towards a specification for the TSC to consider?
Regarding Kujaku, the systems that incorporate it are all entity-centric and I assume that those systems use it as a way to visualize and collect data about those entities. Without an entity context, that pushes more data/workflow management onto users. @Matt_Berg do systems that incorporate the library build their Kujaku configs on the client or server? Regardless, leveraging the library might be possible. An immediate question that would need to be explored is how the existing Collect point/trace/shape question types that use the Mapbox API coexist with the Kujaku lib. Kujaku pulls various functionality from Collect like offline mbtiles management, location provider management and both rely on the same underlying Mapbox lib but Kujaku has an older version. I think that almost certainly means conflicts.
The Technical Advisory Board has assembled a working group exploring entity-centered workflows. It's likely that this would include this kind of mapping. The group is reviewing use cases and process for making bigger shifts to the tools as well as possible targeted funding opportunities.
The Entity-Based Data Collection Working Group was formed, met, came up with a high-level project plan, and is now seeking funding. Geo will eventually be central to what we have in mind but is currently looking like it will be layered on top of other upfront work. In the meantime, Ona has expressed renewed interest in what @Matt_Berg described above.
We propose adding a map
appearance and geojson support to select_one_from_file
. Initially the map
appearance would only work with geojson files but we could expand it to work with all select_one variants (see extensions below). select_one_from_file
with a geojson file and without the map
appearance would provide a text interface identical to how select_one_from_file
works today.
The user experience provided by the map
appearance would be similar to a geopoint with the placement-map
appearance. In the form, question text and a button with text “Select place” would be displayed. If a feature has previously been selected, its label (from the name
property, see below) would be displayed.
When the “Select place” button is tapped, a map view would be launched. All features from the geojson file would be displayed on top of the basemap and any reference layer configured on the device. Instead of being able to select an arbitrary location, the user would be able to tap on any feature defined in the geojson file.
geojson files used with select_one_from_file
:
- MUST have a .geojson extension (not .json)
-
MUST define a single top-level
FeatureCollection
-
MUST define a property called
id
for each feature in the collection (features without this property will be silently ignored; see extensions below for more flexibility which could be later introduced) -
MAY define a property called
name
for some or all features in the collection which will be used as the feature’s label (Note: I proposename
because I’ve seen that in several geojson file examples. An alternative would betitle
which is specified by simplestyle. interested in opinions here!) - MAY define any number of additional custom properties for some or all features which will be displayed when a feature is selected (and eventually available to form logic, see extensions)
Point
s would be displayed as markers like the ones used for the filled form map . Additional feature types could be supported as described in the extensions below.
All of the features would be of the primary app color (blue-green) when unselected. The feature selected would be displayed in red to match the existing geo widgets. In the case of points, the marker would also increase in size (same as the filled form map).
When the quick
appearance is specified in addition to the map
appearance, tapping on a feature would select it immediately and the map screen would close.
When the quick
appearance is not specified and a feature is tapped, a bottom sheet would be shown as on the filled form map. If the feature has a property called name
, that property’s value will be shown as its label (above the horizontal rule). If there is no such property, the id
/underlying value will be displayed as the label. Any properties in addition to id
and name
will be displayed in a table below the label.
When a feature is selected and the save icon is tapped, that feature’s id
/underlying value would be saved as the question’s value, overwriting any previously-saved value (same as geotrace/shape). When the trashcan icon is tapped, the current selection is abandoned and any previously-saved value remains (same as geotrace/shape).
When a geojson file is used without the map
appearance, the id
property would be used as the underlying value/name
and the name
property would be used as the display value/label
.
An additional requirement is the possibility to specify a feature not defined in the geojson file. For now, we propose pushing that to form design: the form definition could include label text that prompts data collectors to leave the question blank if the feature isn’t on the map. If the value is blank, a series of questions could be asked to define the new feature. Once we have the basic requirements in place, we’ll try this and see if it’s sufficiently clear.
Extensions:
- Add feature types
-
MultiPoint
,LineString
,MultiLineString
as outlined circles with a white center separated by lines as for the geotrace widget -
Polygon
,MultiPolygon
as outlined circles with a white center separated by lines as for the geoshape widget with the addition of a light shading of the enclosed area
-
- Any property defined in the geojson file could be accessed in expressions the same way that values from other datasets are. For example, if a field called
the_road
has typeselect_one_from_file roads.geojson
in an XLSForm, therepair_state
property of the road selected could be accessed with expressioninstance('roads')/root/item[id=${the_road}]/repair_state
. This would make it possible to do things like constrain the features that can be selected based on the current location or filter the features shown based on some condition (e.g. only show hospitals that have some specialized team) - Add flexibility to the geojson requirements:
- if there is no property called
id
, look for anid
foreign member (sibling of type: https://datatracker.ietf.org/doc/html/rfc7946#section-6.1) - allow specifying the property to use as the underlying value in the parameters column of the XLSForm (e.g.
name=the_id
) - allow specifying the property to use as the label in the parameters column of the XLSForm (e.g.
label=the_label
)
- if there is no property called
- Support
map
appearance onselect_one
. This would also require specifying eitherlat
andlon
columns or ageojson
column from theparameters
column of the XLSForm. (XForms spec might be weird; additional children foritemset
? This is why we propose introducing themap
appearance rather than only using the filename extension) - Support
map
appearance onselect_multiple_from_file
/select_multiple
- Allow styling using the simple style spec. This would make it possible for the geojson file to specify things like color for individual features.
I'm super excited to read this. Two initial trains of thought. I'm sure there will be more. Please excuse the rambling as I work through my thoughts.
[1] I'm trying to think through how I would get survey answers back into the geospatial dataset. That is, if I have a FeatureCollection
of buildings. And they all have some assortment of the additional custom properties (outside of the must have id
). If a feature has building=yes
+ shop=deli
and then my survey data is that actually shop=cheese
+ also levels=3
then how do I get my output FeatureCollection
that has that feature with all three of the right tags building=yes; shop=cheese; levels=3
. This is probably a matter of an external process/script that merges the data spreadsheet download from Central with the source GeoJSON?
The scenario where we may only be adding details, like a damage assessment where we are adding a damage_level=*
custom property that no feature has yet, seems pretty simple in then getting back to your desired output for analysis (basic join of a survey data table to the FeatureCollection in a GIS software on the id
field).
A mapping scenario in which we might be adding/ editing/ removing custom properties, seems more complicated. If someone clicks a feature with amenity=place_of_worship
then we have a questions asking them if they can add or update religion=*
and denomination=*
. If a feature has shop=*
then we ask if they can add or update name=*
. And so on... and then we want to bring together the starting FeatureCollection data with the assortment of new/ changed/ removed attributes.
[2] When would a user need to see the vertices of a polygon/multipolygon? The geoshape widget in ODK is for editing the shape and so you need to know where the points are. In the OSM iD editor it shows the points when a shape is selected for editing. But otherwise it just shows the border and the translucent fill.
Was it my birthday yesterday ?...
Yes it really was
I believe this would enable using either a choice filter to filter out features based on a previously-captured location, right? So you could require someone capture their location, then once they do that, they only see features that are say 20 meters of their location?
Edit: Yes because I see now that you also said...
Yes, exactly. With this model the merging would have to be customized for your specific form(s) and context. In an entity-based world more of this would (will?) be done by the system. With the extension about accessing properties for use in the form, you could do things like have a shop
select1
question and use the property's value (deli
in your initial example) as the dynamic default.
I promise you this is a true story. I looked at the first image you shared and it made perfect sense. Then I stared at the second one for a while and I couldn't tell why you were sharing it. It was only when I came back to your post the second time that I noticed that there are 7 rectangles that are features. Weirdly, this illustrates one reason why I think it's worth having the vertices -- it allows us to have one more visual cue for features. There are so many different basemaps that we won't always get good contrast. Another reason is that it's just simpler implementation-wise to have a single representation. Is there a reason the vertices would be a problem?
Wait, I can't tell whether this is a joke or not!! If not, happy birthday! Either way, I'm glad we were able to get you a suitable gift!
It was not a joke
Out of the topic but do you mean another way to update media on phones than publishing a whole new version ?
Very excited that this is moving forward! I can't speak unilaterally for HOT, but I do know that we (HOT) have a strategic interest in supporting exactly this: I think at the leastsometime next year we could supply in-kind (dev) support, and maybe more. Shall we book a call?
Amazing!!
Yes, the idea with an entity-based model is that entities would be synced between Collect and the server outside of the context of a single form. So you could have 10k entities and if only one changes, only that one would be updated. Forms would be independent of the entities they act on and wouldn't need to be updated at all in case of entity changes. Hopefully that's where we're going. As you can imagine, it's quite a different model. For now what we've described here would still be the existing model of data updates being tied to forms.
@Ivangayton that's exciting! Will get in touch soon. I think this work as outlined here is ready to move forward but there are lots of related areas where some geo expertise would be very beneficial. In particular, @danbjoseph keeps reminding us that with something like this, offline custom basemaps are even more important.
We have made good progress on the implementation here. Given the level of interest, we have taken on several of the extensions described above. In particular, we are making the map
appearance work with any select_one
variant that specifies a geometry
column. That means you could specify a choice list in the choices
tab with a geometry
column or attach a csv
with a geometry
column and use those in a select with map
appearance. Note that this introduces a new convention: the geometry
column is always used to map features when the map
appearance is used. For the choices tab or a csv, the column value would be points in ODK format (lat lon alt acc
) and eventually also lines and shapes in ODK format.
Because of the way we built that generalization, properties defined in the geojson file will be accessible in expressions the same way that values from other datasets are (I gave a useful example above).
In parallel, @Lindsay_Stevens_Au has added the ability to specify the property to use as the label in the parameters column of the XLSForm (e.g. label=the_label
) and that just needs to be reviewed (PR).
Given what we've learned during our implementation, I propose one additional change: in the case of a select from an external geojson file, defaulting to the title
property being used as the label. I think we'll want to support styling with the simple style spec sooner than later and so it makes sense to follow it minimally to start with (as opposed to using a name
property as I'd previously described).
I look forward to any comments on the proposed expansions and changes I've described above. We still have to wire things together and do quality assurance. We hope to have a beta available by the end of the month.
Had a good conversation with @Ivangayton and @danbjoseph with some ideas on where we could use some geo-related help. In particular, they will look into offline vector tiles.
Nice! Is that a possible feature for Build?
This is extremely exciting news! I can immediately think of many uses for this.
No map
appearance yet but you can now try your geojson files in v2022.2 Beta 1 to check that we are parsing them correctly. In particular, it should be robust to extra elements.
Any geojson file with .geojson
extension, a top-level FeatureCollection
and points only should work. The points should have an id
property (or top-level id
) and a title
property. In an XLSForm, add a question of select_one_from_file my-file.geojson
. You'll need to use https://getodk.org/xlsform/ or Central v1.4.2+ to convert it. You should see the select in ODK Collect and it should look like a normal select. You should be able to access the geometry
and any other property in the usual way. For example, instance("my-file")/root/item[id=${selected}]/geometry
See a small example: geojson.zip (8.3 KB)
Beta 2 is rolling out and includes support for the map
and map quick
appearance for select_one
and select_one_from_file
! See the post above for a description of how to try it out. There's a sample form with geojson
above. This form is an example of an a select with map
appearance and a set of locations defined directly in the form.
One thing we are interested in feedback on is performance. If anything feels slow to you, please let us know more about your scenario, the choices you want to load and the basemap you want to use.
More on the beta here. We are currently planning to release the week of August 11th.
Thanks to those who have tried out the beta! I want to draw your attention to some test forms in the beta thread which provide ideas on how to use this feature.