In Improving connections from ODK to other systems (with health focus) in Tentaive Phase 4 (September 2023 on) there is talk of implementing API Keys & webhooks.
Are these still likely or probable and on what timescale?
Many thanks
1 Like
Our roadmap is at https://getodk.org/roadmap. Understanding interoperability needs is work that's coming next. Can you tell me more about your needs and the problem you are trying to solve?
Thanks for the prompt reply.
We are using ODK to do longitudinal surveys of habitats (in general ponds but also associated terrestrial habitats). We are exploring a new final repository for the data and new workflows to make the whole thing work better and make the data more accessible. The final data repository is likely to be a custom MySQL database using a C# or Python API for access. A couple of things we are thinking about:
-
It would be nice to be able to have webhooks push data into our database(on ODK record create or update), though I acknowledge that we could simply poll ODK central for changes but the webhook approach seems nicer. In our case the data transfer is not time critical, and if a webhook fails it would not be a disaster. Being able to force firing of a webhook manually would be important to enable us to sort failed webhooks easily
-
It might be that we provide a way of editing entities via the API. An API key seems a much better approach than a user/password
-
On another matter SSO that can work with Entra ID (Azure AD) or Google would be great
M
2 Likes
These features are quite important to us as well.
We have been using ODK in an industrial setting for several years, and it is the solution to many of our operational problems. We now want to expand our usage to other projects, and this has triggered a security review which raised some concerns.
- Security:
Our internal security team is blocking further production use of ODK. There are two changes that we need to implement to move forward. It would be good to have these on the roadmap, so our security team could be confident that a solution is coming.- Implement SSO for users - currently this disables the API, which is a core part of our workflow.
- Implement API keys - basic auth breaches security policies, we need a more secure option.
Less a need, but valuable from a functionality perspective,
- Webhooks or similar:
Our need is identical to that described by @mnewmana.
We have a database independent of central. We currently poll the API on a recurrent schedule to pull submissions. This works but does not provide the 'live' connection between Collect and our database that we would like.
There have been multiple instances where our users expected submission data to be present in our database when the scheduled activity to get the data had not yet run. We have had to add a manual trigger to allow users to force the GET operation (the alternative is increasing the polling frequency, with the associated load).
A webhook from Central on each new submission would solve two issues- Load on both sides, as API calls would only be made when they were required
- Massively reduce the latency between a Collect submission and the data existing in our database
My first thought to achieve this was to use listen/notify on the database directly PostgreSQL: Documentation: 9.0: NOTIFY, however our ODK instance is hosted on getodk.cloud, so as far as I am aware this sort of direct interaction with the DB isn't possible. Self-hosting is not an option in the short term due to security and people resourcing limitations.
1 Like