Reducing File Sizes of Submission Attachments and Migrating to Central from ONA

1. What is the issue? Please be detailed.

I’m in a very similar situation to @seewhy where I want to reduce the size of my Central db, but instead of deleting the attachments altogether, I would like to replace a few thousand images with compressed versions at a lower resolution. This form/submissions are currently hosted with ONA, but I plan to migrate to my own ODK Central server hosted on DigitalOcean. I’m unsure of the best way and order to do this migration and image reduction. In addition, the project is currently active and still collecting data, although it is typically a low quantity. I could have this down for a couple of days during certain times without any issues, but I need to make sure it doesn’t extend longer than that.

  1. What are the best ways Migrate a form and submissions from ONA to ODK Central?
  2. What is the best way to automate the reduction of these images that are currently on ONA, but will be moved to Central (DigitalOcean)?

I’ve just started playing around with the Central API wondering what the best way to do this would be, but right now I’m unsure if using Upload an attachment using the same filenames that are reduced in size would be sufficient or if I first need to Clear a Submission Attachment, then upload, or perhaps there is a different way to do this altogether?

2. What steps can we take to reproduce this issue?

3. What have you tried to fix the issue?

Originally when I was going to do this a year or so ago, I used ODK Briefcase to get a local backup of my form and submissions, ran a script to reduce all the images, but never figured out how to re-upload them in an automated way. I remember having issues with Briefcase not being able to upload the form/submissions, but I cannot remember why. I was never able to migrate this form/submissions to Central.

I can’t help with much of this, but it might be worth considering using S3 storage for attachments, rather than losing quality of images (unless it doesn’t really matter). This means that going forward your Central Server will keep a smaller database size and you’ll have a full copy of your data on the cloud, including new submissions - which cuts out the ongoing need to download, compress and replace your media attachments…

My guess on clearing submission attachments is that it could be problematic as they are put into trash (and then purged after 30 days), so new files might be classed as duplicates (in the same way that you can’t duplicate entity uuids). But maybe it will work, or you could manually purge the database… but all of those things feel a bit Here-be-Dragons-esque. S3 could be a more efficient avenue?

Hi @Tyler_Depke,

would also be interesting to hear, please, why you moved to ODK from ONA.