Upgrade from Central v1.5.2 to latest failed

Support
1

Hey,

Our ODK upgrade on our production machine failed to upgrade the data and looks like we have lost all our data. Just like what is stated in this issue.

After much experiment and deliberation, we attempted to upgrade our ODK from 1.5 to the latest:

https://odk.staging.eastafricacoffee.org/version.txt
versions:
63cfab894cd619a703a1a38a00756f864e58b42b (v1.5.2-3-g63cfab8)
3078965dc8ad86b34944476d0dd84475487d44ab client (v1.5.2)
12568c504151c919b11c9962dd3ef866cebbd0e7 server (v1.5.1)

We ran the upgrade following the steps and looks like we have the same problem. The upgrade was successful, but the data was not migrated.

We made changes in configuration files:

modified: docker-compose.yml

image
image512×504 104 KB

modified: enketo.dockerfile
image

modified: files/service/config.json.template

image

The postgres version migration was successful, but it said 'no old data found' And we lost all our data.

We have a full-backup via /v1/backup end point as a zip file.
We also have the cache files enketo-cache.rdb and enketo-main.rdb

Before we did the migration, we also downloaded all the forms and submissions through the odk UI.

we may have missed out on some configuration stuff or may have added some extra configurations.

Is there any way in which we can run the migration ensuring that the postgres data is migrated.

Any help will be appreciated. Our ODK in production is down.

Thanks

Prabhakar

2

My attempt to migrate from from 1.5.2 to the latest version migrated the postgres, but did not migrate the data.

Is there any way we can re run and migrate the data?

Any help will be useful.

Prabhakar

3

Your URL is to a staging server. Did you try and confirm the upgrade was successful before rolling it out to production?

And both for staging and production, did you make a full system backup as we suggest in the upgrade instructions? What happens if you try to roll back?

As to the problem at hand, my guess is that changing the database volume as you did in the docker-compose.yml is the root cause of this problem. How did you determine that was a safe change to make?

Also, look through your command line history. Any chance you ran the down command with docker compose? You can check by running history | grep down.

If you have run down, or aren't sure, run the following command and report back with what you find.

sudo docker inspect --type container central_postgres_1 -f '{{(index .Mounts 0).Source}};

sudo find /var/lib/docker/volumes/ -name pg_hba.conf -exec sh -c "dirname {}" \; | sudo xargs du -sh;
4

Hi ,

We are successfully able to migrate the old data and successfully upgrade it to the latest version .

Our ODK data is stored outside of the default configuration /app/data/odk
(host to container mapping) .

We did changes in postgres service volume configuration
postgres14:/var/lib/odk/postgresql/14 to /app/data/newOdk/data/:/var/lib/odk/postgresql/14
and also update postgres14 docker service from
postgres14:/var/lib/odk/postgresql/14 to /app/data/newOdk:/var/lib/postgresql/14

By making this changes we are able to store the data outside of the docker and wherever we wanted .

For migration since our ODK postgres 9.6 data is present in /app/data/odk we made changes in the volumes of postgres service

/var/lib/postgresql/data to /app/data/odk/data:/var/lib/postgresql/data

After doing these changes we are able to migrate the data successfully .

Thanks for the support.