Need to clear ODK central projects and data

Is there a way to clean out all the projects and data on a Central install? Or does someone familiar with docker know a command to run? I have a demo/sandbox server and I want it fresh between groups of people I show it to. What's the quickest way to get that?

Hi @danbjoseph!

This isn't possible at the moment using the frontend alone. For example, forms can be deleted in the frontend, but project's can't. (There's been some discussion around project deletion here.)

That said, project deletion is possible using the API. So one option for you might be to delete projects using the API, then retire web users using the frontend or API. In most ways, Central would then look like it does after it is first installed. However, there would be some differences. For example, the server audit logs would continue to show past activity.

Would that be enough for the demo? There might also be a Docker command to reset the database, which @issa would know.

Thanks @Matthew_White!

Ah, I missed the developer documentation for Central and was only looking at the main documentation site.

This looks like what I need?

Is the data cleared and the storage space freed up? Or just the Central server completely forgets about it?

Yes, I think that's right! Using curl, it'd be something like:

# Create a new session. You will use the resulting token in the next request.
curl -X POST -H 'Content-Type: application/json' -d '{ "email": "[YOUR EMAIL]", "password": "[YOUR PASSWORD]" }' https://sandbox.central.opendatakit.org/v1/sessions

# Delete the project. You can find a project's id by navigating to the project
# in the frontend: the id will be the number at the end of the URL (for example,
# https://sandbox.central.opendatakit.org/#/projects/1).
curl -X DELETE -H 'Authorization: Bearer [TOKEN]' https://sandbox.central.opendatakit.org/v1/projects/[PROJECT ID]

At least as the endpoint is implemented right now, it's more like the latter. The project will be soft-deleted and will still exist in the database (along with its forms and submissions). The project should still show up in the server audit logs. However, it will not be accessible using the frontend or the project management endpoints of the API.

yes, it will only be forgotten about. when enough people complain about needing to reclaim storage space, we can add a mechanism to prune old (>60 days?) deleted Things from the database automatically.

there is no direct command to entirely clear the database, though if you do something like docker-compose rm postgres that'll more or less accomplish the same thing (you'll have to docker-compose build postgres again after).

1 Like

Any changes on the ability to hard-delete the deleted project (from database) in recent upgrade to ODK Central v1.2.2

Deletions will likely be in Central v1.4 and that release will likely ship at the end of the year.

Hi,
I would like to bring an important point of reflection:
In the ODK-CENTRAL interface, it seems that it is not possible to physically remove submissions, just disabling the display because the data is simply archived. The only workaround is to delete the data directly from the database, but you have to be a DBMS expert. This is also the case for media (photos, video, and sound recordings) almost impossible to delete because of keys and table relationships.
In practice, with ODK-CENTRAL, an investigating body cannot guarantee the physical deletion of personal data. I think that ODK-CENTRAL can put investigating bodies in a position of infringement with regard to their commitments with the CNIL and with regard to the RGPD ( Article CNIL ).
It would be good if the next update of the application included the function to delete a submission (physical deletion), as soon as possible, so before the CNIL issues a restriction on the use of ODK-Central in France under certain conditions of use.

cordially

Is the hard delete form / form submissions coming in odk Central 1.4?

Hi @chun_hing_yap! In v1.4, it will be possible to hard-delete (purge) a Form. When a Form is purged, all its Submissions will also be purged. We're planning to add the ability to purge individual Submissions in a later release. For more details about what we're planning for v1.4, see the release criteria.

2 Likes

Hi,
to purge everything, here is the trick I do:
In PgAdmin, I type the query :
TRUNCATE
TABLE
"public"."submissions", "public"."submission_defs", "public"."submission_attachments", "public"."blobs"
RESTART IDENTITY -- Restart integrate seq.
CASCADE -- Purge all tables with foreign keys
RESTRICT;

I tested successfully
Good use, cordially

1 Like

The feature to delete a single submission has not been realized yet in ODK release 1.4. May I know whether it will be in release 1.5?

Deleting individual Submissions is on the roadmap, but we're working on other functionality as part of v1.5. For more details about what's planned for v1.5, see the release criteria. We don't currently have a timeline for when we will add individual Submission deletion.

Now that v1.4 is released, it is possible to purge a Form. When a Form is purged, its Submissions are also purged. At the moment, this is the only time when Submissions are deleted.