We've developed an app using Android's Accessibility API to automate ODK Collect. It checks if ODK Collect is installed, opens a form, and fills the first question's answer. This approach may misuse the Accessibility API. We're seeking a more direct method to pass parameters to ODK Collect, such as project configuration URL, auto-download forms, and specify the field to fill answers.
This feels like a reasonable approach to meet your specific needs. Is there a concrete issue you're experiencing with it?
This is pretty far outside of our current priorities (https://getodk.org/roadmap). We generally would prefer to reduce rather than increase the integration surface area of Collect because it has historically been unapproachable for most users, brittle, and time-consuming to maintain, especially as Android changes.
For something significant like this to be considered for the core, it would need to come with consideration of edge cases (especially around Android life cycle), tests, and documentation. Is that something you would have the capacity to include? We've had bad experiences in the past with community members contributing narrow solutions without tests that then resulted in a significant maintenance burden.
That said, we are excited about an alternative to QR code configuration using deep linking. The idea would be that instead of scanning a QR code, a user could tap on one of these links. The behavior would then be identical (if a project with the same config exists, prompt to switch to it, if it doesn't create it, then request forms from server, etc). That could possibly be a first step that we work on together.
Using an external app to launch a specific form with values filled in does not fit in well with where we are going. We are working towards Collect having entity-centric views, including map-based ones, in addition to the form-centric presentation it has now. To do that, we'll establish a way to launch forms from individual entities and for those forms to access data for the entity/ies it's about. What you describe is similar but with an external app doing the entity listing. If we take that on now, we're likely to end up with two similar but different ways to launch forms with values. Given this information and @Nishon's findings, if you still feel like this should be part of the core, I think a good next step would be for you to do a spike around how it could be structured so there's something concrete to consider.
Deep linking as an alternative to a QR code is an answer to many of our hopes and dreams!
We are very excited about this and grateful that you are considering implementing it. Let us know if we can help!
Indeed, reducing the integration surface makes sense.
I think what we are getting at is not injecting random stuff into a form.
It's launching ODK with an entity targeted. The entity concept, once fully implemented in ODK, pretty much answers all of our needs. We just need a way to target one of the entities, which may be a fairly small integration surface.
I think it makes sense for ODK to have a basic map view that's integrated, in the same way, that makes for Central to have some basic visualization analytics and serve us some CSV. But just as some users would be using OData to access their data in PowerBI, it might make sense for people with really sophisticated mobile map and navigation needs to use something beyond the basic map view that would be a sensible core feature of Collect. Just as it might not make sense to implement PowerBI functionality within Central, it may not be optimal to implement sophisticated map and navigation functionality within Collect; users who want that might be better served by something external that has a nice but limited integration into Collect (i.e. targeting an entity when launching Collect from a deep link or QR Code).
That's great! It's not currently work that we have scheduled. If you want to try to speed it along, some exploration and design would be much appreciated. For example, here are some kinds of questions that would be helpful to answer:
Are there any likely issues with very long urls? A minimal config would be 400 chars which should not be a problem. Long ones could be multiple thousands of chars.
Do all deep links have to be registered with the Play Store?
When we looked at URLs like collect://settings?config=IAogICJnZW5lcmFsIiA6IHsKICAgICJzZXJ2ZXJfdXJsIjogaHR0cDovL2xvY2FsaG9zdDo4OTg5L3YxL2tleS8ybFdhNGMhdzQxYUtURnRSQzFYQVVNYThnc0N2ZUVQdXdhZVJocHdHVUIwanFsb21nbmpJa1hDZVVQNFdrTzNjL3Byb2plY3RzLzEsCiAgICAidXNlcm5hbWUiOiBmb28sCiAgICAicGFzc3dvcmQiOiBiYXJhbmRzb21lbW9yZXN0dWZmeWVhLAogICJtYXRjaF9leGFjdGx5IjogdHJ1ZQoKCn0= in the past, we found that apps like WhatsApp didn't recognize them as links. That might be a reason to use e.g. https://getodk.org URLs. Would those work offline if Collect is installed?
Are there any considerations that have to be made up front if we'd like the same links to eventually be deferred deep links that open the Play Store if collect is not installed? Or maybe supporting these links offline is not important because forms have to be downloaded for any work to be possible anyway.
Yes, the ideal progression would be to first design and build out Collect's way of listing entities and launching forms from them and then making that mechanism available externally.
If you can't wait for that, it currently seems to me like adding the targeted functionality that you need in a fork for now would be ideal. But again, if you have a concrete proposal on approach, that might change that impression.
Have you considered using Enketo? The biggest advantage of Collect is that it helps manage workflows across multiple forms and maintains state when offline for a while. It doesn't sound like these are as relevant in your context if you want to jump directly into a form. I'm guessing you'll say that the geo widgets and offline mode in Enketo are not sufficiently usable. Those are two areas that we would like to work on sooner than later and that we might be able to consider with you on a timeline that works for you.
Apologies for the slow feedback, I don't have notifications for the forum
You have asked some very good questions @LN that we will investigate and suggest possible answers to.
As for immediate goals:
Deep linking to forms (as an alternative to QR code) seems like it would be desirable for both parties. This is the main thing we need. We will prototype possible solutions and create a PR in our fork for feedback / consideration.
Launching with data pre-populated is very much a nice to have, but it would significantly improve our workflow in the field. If a solution to deep linking is agreed upon, then I think we can address this afterwards, using entities.
As for your point around maintenance: ODK Collect is an integral tool for one of HOT's flagship products, FMTM.
The precursor tool to this, Tasking Manager, has been around in various forms for nearly 10yrs now. I imagine there should be bandwidth for supporting this feature within HOT, NAXA, or another team for the foreseeable future. We would also try to include some comprehensive testing within the PR.
We finally got around to prototyping this during a tech team meeting with NAXA. @zestyping was able to pull together a quick working solution for loading entities via an intent.
To answer your specific questions:
Do all deep links have to be registered with the Play Store?
They only have to be registered on the Play Store if they are HTTP links.
If using the URL protocol as collect://xxx, this isn't an issue.
If the requirement is to have the links identified and hyperlinked within other apps (like whatsapp), then we probably need to use http:// as the protocol and register a verified link on the Play Store, but this is more involved.
Are there any considerations that have to be made up front if we'd like the same links to eventually be deferred deep links that open the Play Store if collect is not installed? Or maybe supporting these links offline is not important because forms have to be downloaded for any work to be possible anyway.
I don't think this is an issue if using a custom protocol collect://.
Further research may need to be done around loading the Play Store ODK Collect listing via an intent, if this is required.
The URL example links to an entity, but to better align with the Central API convention, we may want to use a structure like: /project/ID/entity/ID.
Additional Considerations
The PR allows us to build a custom ODK Collect APK and launch directly with an entity pre-loaded.
We could potentially do what we need without using the entities feature, but think it is best to integrate with the existing entity-based solution (I assume this would be useful to a wider range of users).
Discussion
Are other users potentially interested in this feature?
If we fleshed out the PR and added test cases etc, would this be something you are willing to support in Collect?
It sounds like you've decided links for configuring Collect are not your priority but launching with data pre-populated is, right?
@bs has recently also been in touch with interest in setting values from an external app. In his case it would likely be more than one value.
I'll discuss with the Collect team in the next few weeks and share some thoughts.
I know you may be thinking this is a really simple thing that should be a no-brainer to add given how simple @zestyping's proof of concept is. Here are some examples of questions that we'll be considering:
If we introduce links with a custom scheme, we'll then have two URL types if we want to add http links later. HTTP links are not important to your use case because you're linking from an external app but they can enable workflows like tasking over WhatsApp. They also make more sense for links to configure Collect which we're interested in. Again, those would likely be distributed by email, text message, etc.
Is setting a default value by navigating through the form the best approach or should we consider doing something like using passed in values to generate a filled form instance and opening that?
Should setting multiple values be in scope?
What should the user experience be if Collect isn't configured or doesn't have a local copy of the form?
I think at first we didn't realise that launching directly to a loaded form & pre-populating information were different problems!
Attempting to load Entities made that clearer for us.
Although it would be nice to load the form via a link, preselecting an entity / feature is the much more valuable user experience for us.
You raise many important questions!
If HTTP links are used and must be verified, I wonder if this would prevent usage via a sideloaded APK.
