Converting .shp to .svg and .geoJSON for creating selects question from image/map

[Tyler_Depke]

1. What is the issue? Please be detailed.

I just started playing around with Automatically create geographical itemset.csv for external cascading select which was quite helpful for making an ordinary select_one_from_file using the autocomplete appearance, the using some calculate fields for reverse cascade as shown below:

Now I’m wondering if it could be taken a step further to produce two questions that I think would also be helpful to others. They are getting at the same underlying data, but essentially, instead of choosing from a large dropdown list, users could either select from map or select from image using an SVG. Having a script that could convert all this data from tables/.shp files to an SVG and geoJSON of a countries sub-regions I think would be extremely helpful to the community. Has this been done before?

For my purposes currently, I’m trying to do this for Enketo/Public Webform, not Collect.

1. Selects via SVG would show the .svg image with different regions/prefectures/sub-prefectures as selectable with their names? Similar to how this map is displayed. Can this type of SVG be referenced by multilingual image columns?

   I remember seeing a good example using the 50 US States, but cannot seem to find this reference.

2. Select One from Map. It seems like there are a couple different routes one could go. I tried automating the creation of geojson files from the .shp files that were originally retrieved with the Rscript linked above, but this is the first time I’ve ever used R and have no idea what I’m doing.

Similar to this R Script, ideally these could be done in R or python without needing to download a large GIS package. I’m just trying to think about automating this workflow in a way that the most amount of people could use it.

2. What steps can we take to reproduce this issue?

3. What have you tried to fix the issue?

I tried running this script made via chatGPT, but it looks like it’s creating the same item over and over again in a single .geojson.

library(sf)

setwd("/Users/tyler/Documents/myexamplepath")
shp_files <- list.files(pattern = "\\.shp$", full.names = TRUE)

for (shp in shp_files) {
  shp_data <- st_read(shp, quiet = TRUE)
  
  # Explode MultiPolygons into Polygons
  shp_data <- st_cast(shp_data, "POLYGON")
  
  geojson_file <- sub("\\.shp$", ".geojson", shp)
  st_write(shp_data, geojson_file, driver = "GeoJSON", delete_dsn = TRUE)
  cat("Converted:", shp, "->", geojson_file, "\n")
}

I saw this reference for converting SHP to SVG, but it requires arcGIS and manually relabeling IDs for each .SVG section.

Having a good example .SVG for say the 50 US States would be helpful to look at the formatting to see what code could automate this entire process.

4. Upload any forms or screenshots you can share publicly below.

GIN_adm.zip (936.0 KB) Here is the original .zip file for all the country data downloaded with the R Script from the showcase that could be used to make .

[Tyler_Depke]

From other sources it seems like Enketo doesn’t support select from .geojson file, is that correct? Even if that is the case, I still want to explore this issue/see if there is an easy way to get data for this kind of question with publicly available data. In that regard, I found a separate source for geoJSON files called geoBoundaries but it looks like ODK doesn’t support MultiPolygons, so I see that for the GIN dataset there are about a dozen or so MultiPolygons. My guess is these are along a coast line and include sets of islands. Has there been an easy workaround for this? I’m not sure how an svg would handle this… @ahblake do you know if SVGs can have multiple shapes that have the same id or how this would work?

I’m playing around with this below to convert .shp files with accompanying .dbf and .shx files to create an svg and got it working successfully with the caveat that each polygon is just named:

list_name	name	label
svg_test	patch_1	Patch 1
svg_test	patch_2	Patch 2
svg_test	patch_3	Patch 3

MyGeodata Converter | MyGeodata Cloud

Works in Enketo and Collect (not Web Forms) but by default the boundaries are so faint that they are very difficult to see unless they are highlighted. This was fixed easily by adding

stroke: #000000; stroke-width: 1.5; 

to this fairly close to the top of the SVG file

*{stroke: #000000; stroke-width: 1.5; stroke-linejoin: round; stroke-linecap: butt}

Enketo Before boundaries

Enketo After changing boundaries

Webforms

[ahblake]

@Tyler_Depke

do you know if SVGs can have multiple shapes that have the same id or how this would work?

You can't have duplicate id values for elements in your SVG

But you can group elements! Check out this followup post where I discovered:

@ahblake

If you group path elements with no outline together and use the group ID in your choice list

• these are selectable in Enketo and a blue border appears around each element and the fill doesn't change like it does for selecting single paths.
  • reference to paths and groups being selectable in Enketo feature announcement: As far as we know, any SVG image could be used to make certain “paths” (<path>) or “groups” (<g>) clickable, but there may very well be surprises for some complex images.
• these are selectable in Collect but there is no visual indication, only the 'Selected' text at the bottom.

[chrissyhroberts]

@Tyler_Depke I agree that SVG maps are really useful and it’s great to see your detailed descriptions of this!

A while back I spent a bit of time doing something similar with SVG maps - non-geographical but the concept is nearly the same. There’s a writeup here https://github.com/chrissyhroberts/Balanced_Prioritisation_Tool_For_OneHealth

I think that I hit similar issues with labelling the selected regions properly and in the end stuck the labels in the choices sheet. It’s not an elegant fix, but it does the job and this approach has worked well for us. Doesn’t solve the issue of needing unique labels.

Meanwhile I also came up with a geo-polygon system that I use for geofencing here https://chrissyhroberts.github.io/Code_Book/ODK_geofencing.html. It uses GPS to locate the device and compare to a set of polygon data, ultimately telling you through the cascading select which polygon the device is inside (if any).

It’s a simple but really effective method that decomposes the polygons in to a list that you can feed to ODK forms. You have to trade off the resolution against the surface area you cover, but it works robustly and doesn’t add appreciable load time to data collection after the form is initially loaded on to the device (which can be time consuming).

Hopefully these will be helpful to your thinking.

[Tyler_Depke]

@chrissyhroberts

Meanwhile I also came up with a geo-polygon system that I use for geofencing here https://chrissyhroberts.github.io/Code_Book/ODK_geofencing.html. It uses GPS to locate the device and compare to a set of polygon data, ultimately telling you through the cascading select which polygon the device is inside (if any).

This is exactly what I’m looking to do, but I’m wary of relying fully on GPS points as I’m unsure of how this would go with people using a public form in a place with low tech-literacy in addition to the fact that those who are able to do so may have other privacy concerns. Essentially I need to get info down to a sub-prefecture level, but the select_one with autocomplete appearence I think would be entire

That gameboard in the first link you shared is super impressive, especially when thinking about getting opinions from stakeholders that are likely marginalized/overlooked!

I’m definitely more of a novice when it comes to coding but I used chatGPT and python to get some SVGs. I think I’m nearly done but I’m getting some unexpected behavior in Enketo which is because I’m not sure what the optimal XML format is for highlighting regions as a user hovers over them.

[seewhy]

Have you tried using QGIS and creating a Layout for your map, then using ‘Save as SVG’ - you can use all your ‘shapefiles’ (or other vector layer format) with labels etc and make it look how you want it to appear in ODK.

Depending on the dataset and the extent of the map you are creating, you might need to clean up the output slightly (i.e. open the resulting file in something like Inkscape and edit any shapes that extend beyond the boundaries of your map border). This should then enable you to work with the SVG to create a clickable image… I have not tried the last step and I am unsure what level of ‘metadata’ is included in the SVG output from QGIS.

But might be worth experimenting. And might save you some ChatGPT interactions - no need for python coding either!

[ahblake]

@seewhy

Have you tried using QGIS and creating a Layout for your map, then using ‘Save as SVG’ - you can use all your ‘shapefiles’ (or other vector layer format) with labels etc and make it look how you want it to appear in ODK.

This is something that had been in the back of my mind, but not tested. So thanks for the nudge!

I took a shapefile (source, opened in QGIS, classified according to name, created a simple layout with only this layer and exported to SVG with default options.

Shape file is ~24MB, SVG export is ~8.4MB. After deleting everything from the attribute table except the name, SVG export is ... 8.4MB. (Opening in inkscape and exporting as svgz format is ~1.3MB, svgz support would be great!). Simplifying geometry first (snap to grid, 0.1 degrees) resulted in ~2MB export.

But, what are the id values? Bad news, 'name' was not preserved, and paths were assigned random groups just as when you convert a PDF to SVG, eg Burundi becomes g353002, back to square one.

Or are we, there's a plugin SimpleSvg!
The simplified geometry results in a 164kB export (!), with the QGIS layer name, then a random group name, then a group with the country name, then the path(s).

Reverting to the non simplified geometry and using the plugin spat out a 214kB export.

In my laziness I didn't remove spaces from names, but luckily the export also can't have spaces in the group name .

Testing this export with a form that has the country names as the name values (where we are trying to select groups with ids = country names) didn't work - Turns out you can select a group inside a layer, but not a group inside a group inside a layer, which this export is. Confirmed by naming the g###### group above the country and adding that name to the choice list, that became selectable.

To fix that I split the QGIS layer to one layer per country, named for the country attribute.

I tried to set the stroke-paint to undefined and outline to 5mm to every element, but it didn't seem to take, as selected groups aren't showing in Enketo or Collect. They do select though!
EDIT: I just spotted my error - the SVG layers inherit the lowercased country name from the split, and there is still a group in a group. But my choice list used the lowercased country names which means you can select svg layers as well as groups/paths/objects, but maybe they don't change colour?

Here's a form and SVG to play with and improve upon:
continent_group_SVG.xlsx (579.2 KB)
africa_shp2svg.svg.zip (39.1 KB)