I have come across a tricky little problem and can't find any ODK documentation about it... It potentially has a big impact on predefined field names for managing the style of features on maps (marker-color, marker-symbol etc). This is problematic especially when working with Entities and trying to create or update them with an external tool...
1. What is the issue? Please be detailed.
When I download a form submission or entity via API call or pyODK the field names are changed if they include the dash character -
For example:
Entity property name: marker-color
Exported property: marker_color
Example df.head (using pyODK to get a normalised dataframe):
| QuadratId | PlotLocation | PlotNumber | PlotType | SubQuadNote | Photo1 | Photo2 | Notes | marker_color | marker_symbol | ... | __system.submitterId | __system.submitterName | __system.attachmentsPresent | __system.attachmentsExpected | __system.status | __system.reviewState | __system.deviceId | __system.edits | __system.formVersion | __system.deletedAt |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 31a94d95-4197-4faf-8ea8-25980e89a04a | None | 1 | Woodland | None | None | None | None | #5fd43f | - | ... | 92 | test | 0 | 0 | None | None | collect:daspi7GhFH2P6SKz | 0 | 20250802 | None |
| 1 rows × 75 columns |
This is problematic where I am trying to update an entity as the downloaded table headers / column names do not match the Entity property. In this specific case the name NEEDS to be marker-color if I want to influence the colour of the marker on a select_one_from_file with map appearance...
If I use pyODK UPDATE, based on the csv that I might create from the table table above, it fails:
PyODKError: ('The request to [URL] failed. Status: 400, content: {"message":"The entity is invalid. You specified the dataset property [marker_color] which does not exist.","code":400.28,"details":{"reason":"You specified the dataset property [marker_color] which does not exist."}}', <Response [400]>)
2. What steps can we take to reproduce this issue?
Create a form with a field called marker-color and upload a submission
and/or
Create an Entity List with a property called marker-color and upload / create an entity
run a pyODK script (e.g. https://getodk.github.io/pyodk/examples/basic-analysis-pandas/) to download form data or entity
examine the df.head and you will see that the field named marker-color in Central is downloaded as marker_color
Compare this with the exported CSV file from Central, which retains the dash character (and adds dashes to concat anygroup names)
3. What have you tried to fix the issue?
A simple search / replace in the header names can break things if there are also field names with _ as these are not affected when exported. So it is difficult to automatically match the exported data with Central's form definition or Entity properties.
| Form definition / Entity Property | OData exported name |
|---|---|
| grid_ref | grid_ref |
| marker-color | marker_color |
This also means that my development version of QuODK cannot easily produce layers that accurately reflect the form-definition or Entity List, unless I can work out how/where these changes are made and reverse them! It would require manually changing some column names in the csv file.
Hoping there might be an undocumented setting / parameter to prevent name changes with OData...
I'm not a very good driver of Kappa so haven't come up with anything sensible... So it's time for Actual Intellegence