Increasing disk size for Central on AWS (Ubuntu)

Support
1

Hi fellows,

In one of the Central installations on AWS, I am running out of hard disk space. Need recommendations about how to increase the disk size 'safely' without data loss? Central version is 1.5.3

Many thanks,
Saad

2

I did this only yesterday on my central install on AWS and I followed https://medium.com/@m.yunan.helmy/increase-the-size-of-ebs-volume-in-your-ec2-instance-3859e4be6cb7.

Really quick and easy.

3

Hi,

Yes, I have actually followed similar process to get it done. However, he has missed some important 'good-to-have' steps, and also it's not ODK related directly. After I complete my expansion, I intend to write a tutorial including all these steps, so that everyone in ODK community could benefit.

Regards,
Saad

2 Likes
4

Hi @Saad,

This will be very useful, please do. Highly appreciate your initiative to help the ODK community.

Regards,
Joseph

5

Hi,

This is a summary of the problem and the solution:

Problem: Central server ran out of disk space, and started giving errors on GUI login (error 500 - Something went wrong). No user was able to login to the system from GUI. Server was on AWS.

Solution:

  • Create the snapshot of the disk from AWS Console. Ensure to check that SNAPSHOT and AMI both are properly created. Use the NO-REBOOT option during the dialogue.

  • Login from command line (CLI). Go to ODK central directory.

  • Stop the CENTRAL application via command:

docker-compose stop

WARNING: DO NOT USE COMMAND 'docker-compose down' TO STOP CENTRAL. YOU WILL LOSE ALL DATA.

  • Go back to AWS console. Track the disk volume from the instance resources. Click the volume name to go to VOLUMES section. Select the VOLUME, click MODIFY, and change the disk size to newer bigger value (e.g. 500GB to 1000GB). Hit MODIFY. Then wait the console interface to complete the OPTMIZATION of the new disk size. It may take a few hours, depending upon the size.

  • Next, extend the OS filesystem. Use this tutorial for detailed steps (https://medium.com/@m.yunan.helmy/increase-the-size-of-ebs-volume-in-your-ec2-instance-3859e4be6cb7). Here I am pasting only quick commands, to be executed one by one:

$ df -h
$ lsblk
$ blkid
$ growpart /dev/nvme0n1 1
$ resize2fs /dev/nvme0n1p1
$ df -h (to confirm the new disk size)

If you see no error, your disk expansion would be completed successfully.

  • Go to central directory and start central:

$ docker-compose start

  • You should be able to login to the GUI interface comfortably.

Feel free to engage me for any queries or help needed in anything.

Cheers!
Saad

6

Sorry you ran into this issue and I'm glad you got it resolved. Thanks so much for sharing your process.

Starting with v2023.2 (migration to postgres14), this is no longer an issue. For folks who are running older versions, it can be a problem but please note that data is unlinked, not actually lost. More information on how to recover from this in the documentation.

1 Like