Central upgrade failing due to cache_all_submission_geo migration

Support
1

Hi All

I'm trying to update ODK central to the latest version but i'm getting a 502 error when trying to sign in once the container is brought up again.

The servers starting at version v2024.3.1, and the problem seems to happen with the v2025.3 update.

The command docker compose logs -f service shows:

odk@odk-central:\~/central$ docker compose logs -f service
service-1  | wait-for-it: waiting 15 seconds for postgres14:5432
service-1  | wait-for-it: postgres14:5432 is available after 0 seconds
service-1  | generating local service configuration..
service-1  | running migrations..
service-1  | Prefilling cache of submission geofields: 44125 fields to process
service-1  | Done:  1000 / Left: 43125
service-1  | Done:  2000 / Left: 42125
service-1  | Done:  3000 / Left: 41125
service-1  | Done:  4000 / Left: 40125
service-1  | migration file "20250928-01-backfill-submission-geocache-doit.js" failed
service-1  | migration failed with error: SELECT cache_all_submission_geo(true, 1000) - could not parse XML document
service-1  | Error: SELECT cache_all_submission_geo(true, 1000) - could not parse XML document (line 1: Start tag expected, '<' not found
service-1  | --78ebf71844ba0bfe7e1a5d1d9795d0d4
service-1  | ^)
service-1 exited with code 0

The failed migration keeps repeating.

Does anyone have some advice on how to resolve this?
Thanks

Alex

2

We believe this means you have some incomplete XML submissions in your database.

If you have direct database access and want to double check this, you can run the following SQL:

select id, xml from submission_defs where
xml_is_well_formed_content(xml) and not
xml_is_well_formed_document(xml);

We are working on a fix that will likely be released next week.

3

Thanks! Looks like there are 4 incomplete XML submissions.

4

We've now released Central v2026.3.1, which should fix this issue. Thank you for the report, @alexcorran!