Data persistence when upgrading/downgrading Central

I strongly support this - thanks for mentioning the certificate, it was not that relevant for my application where I use self-signed certificates. But thinking it over, it may even be really helpful exactly for self-signed certificates.

1 Like

Thanks, Florian,

for ease of installation, I prefer having the database in Docker. And is is only half a line of code, once you get rid of the idea that "this is postgresql". This works for me, but it probably did not work for @cartes because he had letsencrypt certificate. Maybe with the certificated mounted externally, all would be fine.

 postgres:                                         
   image: "postgres:9.6"                           
   volumes:                                        
     - /data/postgresql:/var/lib/postgresql/data   
2 Likes

Thank you all for following up and sharing your experiences and suggestions and I'm sorry you ran into similar issues, @cartes. We have been discussing this and will put forth a concrete plan based on your suggestions and experiences.

Please do note that the database is not a stable interface and that this is not a supported way of accessing Central data. You can read more in this post and we will update documentation to make this clearer. As I described above, we are working on an API to request a database backup without needing direct database access. The analysis workflows we currently design for are based around periodic exports or connecting to the OData feed. If those do not satisfy your requirements, it would be good to learn more about your use case. For example, @dmenne has described wanting to look up specific entities he has collected data about rather than doing bulk analysis.

The docs say:

By default, the only things removed are:

  • Containers for services defined in the Compose file
  • Networks defined in the networks section of the Compose file
  • The default network, if one is used

I see now how removing the container would lose the anonymous volume bindings but I think the docker-compose docs could make that clearer. I have made some suggestions on their end. For what it's worth as we figure out what action we will take, the volume should still be there and can be identified and reattached. I fully acknowledge this is very inconvenient.

1 Like

I landed on this thread looking for something else and realize it would be helpful to provide the latest status.

I made the following addition to docker-compose down docs:

Anonymous volumes are not removed by default. However, as they don’t have a stable name, they will not be automatically mounted by a subsequent up.

We've explored approaches to migrating to named volumes but have decided it's too risky for existing installs. At the same time, we have not identified a scenario for Central management that would require a down. If you believe you need to bring containers down instead of stopping them, please describe your use case.

We've added a warning at the start of the Central installation docs to emphasize that down should not be used and have provided instructions for recovering from one.

1 Like