1. What is the problem? Be very detailed.
My 60GB disk at Digital Ocean suddenly became 100% continuously full during a scheduled daily backup to Google Docs. Surveys can no longer be uploaded to Central. Because of the full disk, I could not perform Digital Ocean's required console update and log onto the console.
2. What app or server are you using and on what device and operating system? Include version numbers.
ODK Central v1.2 on a Digital Ocean droplet.
3. What you have you tried to fix the problem?
Following guidance from Digital Ocean support, I rebooted the droplet to the Recovery ISO, mounted my files and used chroot to access my file system.
The "cd central" command returned a directory not found error error. "pwd" revealed that the current directory was /. With FileZilla, I found the Central directory at /mnt/root/central.
After a cd to /mnt/root/central, the "docker-compose --version" command confirmed my previous docker upgrade, returning "docker-compose version 1.29.2, build 5becea4c."
I then attempted to upgrade from Central version from v1.2 to v1.3. Previously, running upgrade cleared tmp files and freed disk space (even if the version was current). However, the "git pull" command returned "fatal: Unable to read current working directory: No such file or directory." I reconfirmed with pdw that the working directory was still /mnt/root/central.
Next, I attempted to follow guidance from the ODK Forum page "Errors with sending forms to ODK Central because hard drive is full". The command "docker exec -it service sh" returned the following error: "Cannot connect to the Docker daemon at unix://var/run/docker.sock. Is the docker daemon running?" This is puzzling, because docker worked fine for other commands.
Any help or guidance to solve this disk full problem would be greatly appreciated.
4. What steps can we take to reproduce the problem?
The attached screenshot shows a sharp peak in disk use when the daily backup to Google Docs takes place. Although the backup is compressed, little actual reduction takes place because almost all the data size is the jpg media file in each survey instance. It appears that the problem occurs when the backup attempts to use more than 100% of the droplet disk space. Although in this case the backup failed because Google Docs was full, a failed backup a few months ago, when fewer survey instances had been uploaded, did not cause the problem with a continuously full Digital Ocean disk..
5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.
