Quick intro, in my company, we are using ODK for creating and managing surveys, and it’s working great. However, we also have our own database where we import information from the ODK database.
Currently, we check if there were submissions in ODK within a specific time interval. This method is inefficient because we often check even when there are no submissions, resulting in no results 98% of the time or more.
What high-level problem are you trying to solve?
The main goal of this feature is to receive a notification in the form of a webhook when a new submission is created.
Additionally, webhooks could be mapped to other events, such as when a new draft is created. However, this would add significant complexity to the implementation.
Any ideas on how ODK could help you solve it?
We propose that ODK sends a POST request after a submission is successfully created, to a specified destination. This part could be managed here (though not necessarily): ODK Central Backend Code
To implement this feature, we would need to:
Define the location where the POST request should be sent, requiring a new record in the database.
We would need a frontend interface to input the webhook. We suggest creating an additional tab on the system page with a form for managing webhooks.
One last mark, we would use this as global feature, not tied to any specific user.
If you have some familiarity with Python, you can use pyODK to write a script that sends emails on new submissions that meet your criteria.
If you aren't yet comfortable with Python scripting, you can use Zapier's drag and drop user interface to build an integration to send emails on new submissions to ODK. Zapier will likely be free if you aren't getting lots of submissions.
@yanokwa Thank you for the reply. Unfortunately, Zapier is not an option for us as we would surpass the free tier within a week. And for pyODK you would suggest to create a script that checks for new submissions with a time interval and if there are new submissions send a notification? We are already doing that and that does not help us.
Why isn't the script helping you? My assumption is that it checks and only emails if there are no new submissions. Is the problem that it's not correctly reporting when there are new submissions?
checking if there are submissions at a set interval which results in no action most of the time since we are checking every minute
To:
receiving a notification (POST request) from ODK and only doing the action when it's needed
We have implemented the first scenario and it's working. However, we want to set of the action right on the submission and also not be checking for no reason.
@moreno123 Are you still using this method? Is it working for you? I could see that the webhook method would be most appropriate when you have long irregular intervals without any submissions, then short intervals with a lot of submissions. This makes it hard to find a good static interval to check for new submissions w/o making your checking interval small and then having no changes most of the time. This is currently how I'm doing things with Google Apps Script and while there are not a whole lot of changes most of the time, it still does work...
This use case would be a perfect one where having a webhook instead of a script on repeat every minute might be more helpful. Essentially they need to send an email with data to someone to verify what has been filled out.
Thanks Tyler! FYI it has a limitation for the size of the submission that can be sent, due to the way it's implemented (I mostly use the entity update trigger), but might work for your needs
There is a solution to the problem documented in the issues, but I probably won't have time to implement it for quite a few months
