Show previously collected geospatial data as reference layer

Ideas
1

Hi all,

I have a form in ODK collect where I use a ‘geoshape’ question to collect polygons. The polygons are then stored in an entity list. During a monitoring, we want to be able to modify the polygon (in ODK Collect) by selecting it from the entity list and re-do the mapping. Ideally, the former (erroneous) polygon is plotted on the map of the geoshape environment so that the enumerator can see what was the shape of the former polygon was, and how this should be modified. Ideally, the shape itself is modified, but I can also work with just plotting the former polygon on the map inside the geoshape and add a new shape. Is this possible? So (in resume) my question is: Can I add a geometry (polygon, line, point) from an entity list inside the geoshape question? And how could this be done? Thanks for clarifying.

1 Like
2

You can't (yet) include other geometry in a geo question, so one effort intensive option is to use submitted geometry to create a tileset that will display and allow creation/ modification of your polygon.

Less effort and hopefully acceptable would be to select the entity item somehow (select, barcode, pick geometry from map...) and then have a triggered calculation in your geoshape question that looks up the geometry value you stored against the entity. The user can then view and modify as needed.

1 Like
3

@ahblake,

Thanks. Yes, I already explored working with a imagery tileset as a background but creating these tilesets is too labour intensive for our case and it also requires some additional tasks (downloading those tiles every time they go to the field) from our enumerators (hundreds) which -I suspect- will not work in practice.

I read somewhere about the ‘pulldata()’ function in the calculation field. That could work(?). I will try. Thanks again.

4

I prefer instance lookups over pulldata, but both would work. There's an example instance expression in the official form template to refer to

5

@ahblake,

I can’t get the pulldata() option working. I have attached an excel of which I thought should be a solution. But this is not working. Is this approach the appropiate direction….?

odk_forum_integrate_polygon_geoshape.xlsx (18.6 KB)

6

Try replacing this calculation:
pulldata('monitoring_trees','geometry','search_label',${org_site_selection_monitoring})

with this:
instance('monitoring_trees')/root/item[name=${org_site_selection_monitoring}]/geometry

and add a trigger so it won't keep being recalculated, overwriting any geometry changes, as this:
${org_site_selection_monitoring}

But put them directly in the geoshape field, no need to use a helper calculate, and the geoshape doesn't need the placement-map appearance. And then save_to the entity geometry field (unless you have intermediate validation steps).

Of note - your form asks the user to walk around the field, assume auto recording / capturing GPS coordinates - if you want them to modify a polygon you would have to explain for the modification case to manually drag the points to make it more accurate/correct, otherwise delete points until all inaccurate points are gone and then start walking and auto collecting points.

odk_forum_integrate_polygon_geoshape.xlsx (17.4 KB)

1 Like
7

Thank you @ahblake for sharing all these insights on what’s possible right now!

@Edmonds, it sounds like dynamic default likely does what you need in this case and should work today! You might also be interested in exploration we’re doing around showing existing geometry as context for shape/trace questions!

For those interested in displaying existing geometry as a reference while mapping new ones, we have a few open questions we’re exploring and would love your thoughts. If others have similar challenges, please share the scenarios you’re designing for and any feedback on the questions below.

Scenario examples

  • I'm capturing one of a farmer's fields, it's helpful for me to see the fields that have already been captured, especially if I'm not the one who captured the existing ones, so that I don't accidentally re-visit one or capture a geometry that overlaps.

  • I'm capturing the location of a household I'm visiting, it's helpful for me to see all households that have been visited so that I don't accidentally re-visit one

Open questions to better understand the needs

  • What would help enumerators understand that previously mapped geometry is reference, not something to edit?

  • When viewing a reference geometry, what (if any) additional information would be useful to access in the field?

  • Are there cases where enumerators need to see mbtiles or multiple sources of map context at the same time? If so, what are they and why?

  • How much control do enumerators need over showing or hiding reference geometries while mapping?

  • Are there situations where too much map context could interfere with the task?

  • When a map opens with both new and existing geometries, what should it prioritize showing? What is most helpful for the user to help orient?

1 Like
8

I recalled this was brewing away, hence my '(yet)' :wink: , looking forward to it! I'll ruminate on the questions.

3 Likes
9

Hi @Aly_Blenkin

first of all I wish the team all the best for 2026 !

A first round :

What would help enumerators understand that previously mapped geometry is reference, not something to edit

Its appearance (specific stroke color, its thickness, transparency)
The ability to show/hide the "layer" of references geometry

When viewing a reference geometry, what (if any) additional information would be useful to access in the field?

Not easy. Something meaningful for sure. Regarding the first scenario, you're thinking about showing objects that have been mapped by others, so, for example, the name/id of the enumerator, the creation date, the "type" or any characteristic of the object

Are there cases where enumerators need to see mbtiles or multiple sources of map context at the same time? If so, what are they and why?

I think so, for example the parcels we own and the plots we follow...
But the most important thing for me is to better contextualize the new object I am mapping and to avoid mapping the same part of reality (one map context) twice, and the avoid geometry cleaning into the database. For example, mapping land cover, the forest I am mapping can share its border with a crop field I already mapped but can't overlay it.

How much control do enumerators need over showing or hiding reference geometries while mapping?

This could help for sure, for example at a small scale if there is a lot of geometries that hide the background map.

Are there situations where too much map context could interfere with the task?

Maybe depending on the scale of the map but also if two many objects have been mapped in the past.

When a map opens with both new and existing geometries, what should it prioritize showing? What is most helpful for the user to help orient?

The background map more than already mapped geometries.

1 Like
10

@ahblake

Thank you so much! It works! Indeed, as you say: I need to consider whether it is practicle in some cases to drag points to another location manually. That could be a frustrating process in the field. Rather remap the entire polygon than dragging numerous points to another location. But in some other cases where minor adjustments are needed (or walking around the field again would be very time consuming), this manual adjustment may be very efficient. However, in any case, spatial context is missing by for example neighbouring polygons. So ideally, you would like to see the entire layer (showing all polygons, defined by a filter) so that the enumerators can see when their mapped polygon is overlapping a neighouring polygon. This is particularity valid for projects that have to map smallholder systems. I will answer the questions of @Aly_Blenkin .

2 Likes
11

@Aly_Blenkin

  • What would help enumerators understand that previously mapped geometry is reference, not something to edit?

Answer: Color maybe. Showing non-editable polygons (reference layer) in another color then the editable polygon. Besides that: I see that at present, the editable polygon already appears as editable (by showing the vertices). This already explains to users which polygon is editable and which not.

  • When viewing a reference geometry, what (if any) additional information would be useful to access in the field?

Answer: Name or ID of the site. Or any other prefered label.

  • Are there cases where enumerators need to see mbtiles or multiple sources of map context at the same time? If so, what are they and why?

Answer: In our case: (up to date) true color satellite imagery as a background layer would be ideal. This is useful to check whether your polygon boundaries make sense. Especially valid in agro(forestry)/smallholder projects. But a global satellite imagery dataset might be too date ‘heavy’ and slow down devices.

A work-around for satellite imagery as a background layer could be to integrate a button that gives you direct connection to a imagery server where enumerators can define their AOI and then directly download imagery on to their device (keeping required technical skills as low as possible).

Besides having satellite imagery as a background layer, having a polygon (polygons already mapped for the project) layer as context is most useful (showing neighbouring polygons)

  • How much control do enumerators need over showing or hiding reference geometries while mapping?

Answer: not that much. Keep it as simple as possible. The main focus should be that the polygons are ‘clean’: no self-intersection errors, no overlap with other polygons (snapping function…?), no duplicate poygons, polygons that ‘make sense’ (no needles, no strange shapes etc). This can be improved by: high resolution satellite imagery as background layer AND/OR polygon layer as a reference layer AND/OR functions that check the quality of the polygons during mapping (e.g. checks on self-intersections, overlap etc.)

  • Are there situations where too much map context could interfere with the task?

Answer: Maybe. Keep it (as) simple (as possible) is my motto…:wink:

  • When a map opens with both new and existing geometries, what should it prioritize showing? What is most helpful for the user to help orient?

Answer: Color works best I think to distinguish. And/or using filled (50% transparent) polygons and (100%) transparent (only lines) could also be a good way to distinct layers.

1 Like
12

Open questions to better understand the needs

  • What would help enumerators understand that previously mapped geometry is reference, not something to edit?
    • Clear styling differentiation, while preserving styling (stroke/fill/thickness) that is applied to the geometries to identify them - a styling type applied to the capture/edit geometry not allowed to geometries perhaps (eg line style dashed)
    • agree with @mathieubossaert toggling it on/off for clarity, similarly to selecting tilesets.
    • additionally, the reference geometry should be filterable in some manner, don't necessarily want to see every item, maybe only those within a certain distance of the user or of a certain field value (for the 'building' set, show only 'hospital') etc
  • When viewing a reference geometry, what (if any) additional information would be useful to access in the field?
    • if the form could identify desired entity field(s) to display it's in the user's hands. But how is this actioned - if taps are creating points, tap-drag/tap-hold-drag is moving points, what action indicates 'don't create or move a point but tell me about the existing geometry I'm interested in?
  • Are there cases where enumerators need to see mbtiles or multiple sources of map context at the same time? If so, what are they and why?
    • Yes. for example a tile layer with drawing/imagery content combined with an entity geometry layer showing existing features to allow matching / aligning them or not creating duplicates as you can see it already exists.
  • How much control do enumerators need over showing or hiding reference geometries while mapping?
    • A simple toggle on/off in the map view would cover most cases, and filtering would be controlled by a question(s) beforehand
  • Are there situations where too much map context could interfere with the task?
    • if there are many and it slows the map down or they don't cluster and it's hard to see any map information as it's blocked by the reference information (but toggle off if so?)
  • When a map opens with both new and existing geometries, what should it prioritize showing? What is most helpful for the user to help orient?
    • existing = reference set and new = 'add/edit' geometry? the new geometry should be drawn at the top level / be the most visible. From bottom to top I guess it's: basemap / tileset / reference / add-edit.
2 Likes
14

As usual, I would say 'What @ahblake said'.

Some tricky problems well described regarding add /edit vs display info (especially if for any reason you DO want to overlap with existing geometry - a geo-Venn diagram! - scenario - existing geometry might show field boundary or habitat type, new geometry is for a sub-habitat, or species extent).

One possibility would be to have a 'toggle edit / info mode' button? But the screen is going to be filling up with all the icons! Maybe a multi-toggle button that cycles through:
new/edit mode || existing geometries visible || existing info shown on tap / mouse.

Base maps (offline layers) are really useful for ground truth of previously digitised geometries (desktop), and can be more flexible in terms of styling, to avoid confusion with ODK geometries.

It would be helpful for any styling of existing (submissions and entities) vs new geometry to be consistent - also between collect, Enketo and Web Forms

I'm starting to wonder if geo-widgets need to have a bespoke menu to show / hide the icons and reduce clutter while creating / editing / navigating? Is that possible with Collect and Web Forms (probably not with Enketo for well rehearsed reasons)

2 Likes
15

Thank you to everyone who shared their feedback! We are working on the spec and will share the full details next week. In the meantime, here is a quick update on our direction:

In scope :green_circle:

  • Read-only display: Points, shapes, and traces will be visible while mapping new ones

  • Visual distinction: Clear UI so users know these background geometries are not editable

Out of scope (for now) :red_circle:

  • Metadata display: Showing info like creation date or enumerator name for reference

  • Real-time overlap constraints: The map won't "block" an overlap in this version

  • Layer toggling: Users won't be able to switch these layers on/off manually in the app

We are still refining the colours, stroke weights, and transparency to ensure the best map legibility. Current thinking is that form designers will opt-in to this behaviour, and previously mapped geometry will be displayed by default. Stay tuned for the full spec next week!

Here is a little sneak peek :eyes:

Mockups of previously mapped shapes in purple with the active polygon in blue.
Mockups of previously mapped shapes in purple with the active polygon in blue.1504×1032 262 KB

Let us know if you have any questions or feedback so far!

4 Likes
16

Of course I have questions!

  • Is this for Collect / Web Forms or both?
  • Is this currently only for geoshape questions or also geopoint/trace?
  • Will any styling against to the reference geometry layer be preserved? eg keep pin character/colour and change size/transparency or line/shape stroke width/colour & fill but make the line dashed and the fill more transparent etc.
  • form designers will opt-in to this behaviour, and previously mapped geometry will be displayed by default - these appear to contradict?
  • How will it be enabled and how will a geo dataset be referenced
    • further, will the dataset be filterable - eg dataset of 10000 households, but only show those that are in the selected village / within a geofence / only those with children etc
    • can >1 dataset be shown, eg polygons with farmers fields and points with plant locations.
  • If the reference geometry is from an entity list, and the collection is save_to-ing that entity list, in a repeat, will previous repeat geometries be added to the layer of existing geometries? (Assume a new submission will use the updated entity list so the prior submission values would be shown. Mid form I'm not sure...)
  • Users won't be able to switch these layers on/off manually - does this affect offline layers in any way? So can have a basemap/offline layer/reference geometry layer all with the collect new geometry.

I assume a 'snap-to' pre-existing vertices when creating a new geometry is way out of scope for now - for the case of creating nicely adjacent polygons, like field to field that doesn't overlap at all @dast ?

This is very exciting, it will be super handy to be able to see geometries while collecting them. For some users it might completely remove the need for offline layers.

Edit: if this is all addressed in the spec I'll cool my jets and wait!

1 Like
17

Totally agree from a user perspective: This will be such an important feature for us and our planting partners! Very happy with this:-)

2 Likes
18

@Aly_Blenkin

This looks already great and would cover much of our usecases.

Snapping/overlapping with previously captered geodata:
If such a feature could be turned on/off in the XLS Form, fine. But I would not include too many features and keep it simple and basically offer a visual control of what the enumerator is capturing at the moment against what has been captured before. Maps basically work as a visual tool and should be configured as such. And if it’s about polygones of farms of smallholder farmers (as in our case) who do not have land-titles: who can tell if the previously captered polygone shows the true border or the one the enumerator is capturing at the moment? Or do none of both show the truth? The real world can be very complicated :face_with_spiral_eyes:

Features to dispay in the screen in Collect
Of course it would be nice and informative to define in the XLSForm a column that is used to display an ID or name on the map of previously captered polygons. But this could also end up in overpopulated screens in Collect.

A toggle button to switch the display of previously captered polygons and/or their display names on and off would be nice to have, but not an ultimate priority.

Filter which features are shown on the map
There sould be a filter option in XLS Form to control what exactly is displayed. It might not in all cases (but in most) be the geodata from the same column in the entity list the geodata is saved to.

But what is shown on the screen-shots combined with the new features to prevent polygon errors already covers the need of 90% of our usecases. Great. Thank you for this exiting development.

Just one last thing
Please consider adding a feature to the XLS form that allows to specify at form level the time interval at which individual points are recorded in automatic mode. Depending on requirements, this interval may need to be shorter (for very small polygons) or longer (for large polygons). Enumerators often forget to set this in Collect.

1 Like
19

Again, great feature. Huge thanks to the people that worked on this.

My first question would also be if there could be a filter on user level. We have +180k planting sites. I am afraid when we would push all of those 180k sites to all the devices of our enumerators would seriously slow down their devices. On the other hand, users need to see ALL their sites. So a filter on device level (user level?) that reduces the number of sites downloaded from the entity list would be ideal. As far as I know, entity list are always pushed entirely to all devices and filters are arranged in the ODK forms.

Anyway, I first need to test it.

1 Like
20

Amazing conversation so far! Thanks for all the feedback.

A lot of these questions we will answer with the spec proposal, so I’ll hold off on answering a few of these. I’ll focus on some of the more product and user experience questions for now.

@ahblake

Is this for Collect / Web Forms or both?

This is focused on Collect right now. How big of a need is this for Web Forms as well?

Is this currently only for geoshape questions or also geopoint/trace?

For shapes, points and traces :sparkles:

Will any styling against to the reference geometry layer be preserved? eg keep pin character/colour and change size/transparency or line/shape stroke width/colour & fill but make the line dashed and the fill more transparent etc.

Yes, we want to preserve the styling. There are some tradeoffs to this because it might be confusing to differentiate, but we also know that if there was the decision to style geometry then it has meaning and we want to respect those intentional design decisions.

form designers will opt-in to this behaviour, and previously mapped geometry will be displayed by default - these appear to contradict?

Form designers will opt-in to this behaviour to display previously mapped geometry. We are thinking end users might not have the ability to toggle initially, so it will be displayed for them by default is what I meant.

That said, I am exploring options for what a toggle experience could be like so we can get a better understanding of complexity to implement. Stay tuned ; )

I assume a 'snap-to' pre-existing vertices when creating a new geometry is way out of scope for now - for the case of creating nicely adjacent polygons, like field to field that doesn't overlap at all @dast ?

Snap to grid is out of scope for now, but we do understand the value!

@dast

Features to dispay in the screen in Collect
Of course it would be nice and informative to define in the XLSForm a column that is used to display an ID or name on the map of previously captered polygons. But this could also end up in overpopulated screens in Collect.

A toggle button to switch the display of previously captered polygons and/or their display names on and off would be nice to have, but not an ultimate priority.

Great feedback and I’ll share more about the toggle ideas soon!

@Edmonds

My first question would also be if there could be a filter on user level. We have +180k planting sites. I am afraid when we would push all of those 180k sites to all the devices of our enumerators would seriously slow down their devices.

Appreciate you sharing this use case and this is top of mind for us! We certainly understand how valuable it would be for enumerators to filter. We will share the plan and how that might impact performance in the next few days when we share the XLS Form design.

1 Like
21

For me now, not much of a need, it was more curiosity. As WF matures and offers feature parity it would be great to also have.

Looking forward to the spec, all sounding very positive