Increase data resilience

1. What is the issue? Please be detailed.
I got an error in server hosting odk central and we were scared of data loss. Fortunately, we perform an upgrade and it was all good but it could have not endup this way.

3. What have you tried to fix the issue?
We are using digital ocean droplet and backup which is great but it saves a copy of infrastructure every 7 days.

However, we would like to know if someone can share experience about:

  1. resizing instance. In the case of digital ocean, it is written that the droplet should be off. It is recommended to take a snapshot and there it says is good to power down droplet Droplet before taking a snapshot to ensure data consistency. So my first question is what should I do in terms of odk central, should stop it with: "docker compose stop" , turn off, perform the snapshot, the resize and then turn on and "$ docker compose up -d"?

  2. I would like to learn how to setup odk central to send data to another postgres database in a different droplet. What resource could you recommend to read/see for this task?

Thank you,

It's generally fine to take live snapshots, especially if the DB is not on the same machine, but you'll get the most reliable snapshot by doing what you described: docker compose stop, power off, snapshot, resize, restart, and docker compose up -d.

See using a custom database server for connecting Central to an external database. I'd recommend using a managed database provider (e.g., AWS RDS) so you don't have yet another server to manage.

Installing Central is relatively easy and it gives you 100% control, but it also means you are 100% responsible for ensuring your install stays fast, reliable, and secure. If you are comfortable running Linux servers, it's very doable, but if you are not, there will be a lot of stress when something goes wrong.

That's why we created ODK Cloud. It's not free, but it gives you the most hassle-free ODK experience. We do things like continuous backups that we can restore quickly, automatic resizing of compute and database to match load, monitoring and mitigation of attacks, and so on. We also provide very fast user-facing support.

All that said, we think it's important that Central is self-hostable and we continue to make it easier to self-host, but self-hosting is always going to be a very technical task.

2 Likes