Access to ODK Collect from XML data from other Android applications

Support
1

1. What is the issue? Please be detailed.
I would like to use a CSPro Android applications able provide a complete error report of some inconsistencies from data collected by ODK Collect family (including KoboCollect). The error report will be displayed on the devices once the enumerator finalize a Collect form (or questionnaire), but not yet submitted. It is why I would like to be able to access to XML form data from ODK collect. In CSPro, there is an object, system app, who allow the execution of activities from any Android apps and return the result
:https://www.csprousers.org/help/CSPro/SystemApp.html,
https://www.csprousers.org/help/CSPro/SystemApp_exec_function.html, https://www.csprousers.org/help/CSPro/SystemApp_getResult_function.html,
So, I would like like to know the name of the ODK Collect activity able to provide access to form data (or any other way), for example if there is a workaround we can program in the XLSForm to facilitate such access such as deep link: https://developer.android.com/training/app-links#deep-links.

2

Hi @htuser,
Would you mind to explain more why you need to use CSPro additionally to check ODK form errors? Which specific limits of ODK constraints etc. do you address?

3

As you know ODK questionnaire programming is a no code process. And it is by design. ODK developers wanted to offer, mainly to developing countries, a low code/no code way to collect data. And it is one of the most successful open source project in the world! Thanks again to the team. ODK democratize electronic data collection and have a huge impact. However, it has a lot of limitations because It is writing logic using Excel formulas and other specific ODK trying to wrap a real programming language. By opposite, a CSPro application developer have a true SDK and real compiler and are using several powerful programming languages for questionnaire development.
a) For business logic
Developers can use :

So developers can take full advantage of the JS ecosystem, among others multithreading (Worker), async operations etc.

b) For the UI,
They can use web technologies such as HTML5 and CSS3 if they don't want to use the native CSPro UI.

There is also a complete bidirectional API between CSPro Logic and JS for questionnaire programming including complete data conversion between JS objects and CSPro logic.(https://github.com/csprousers/helps/commit/88bd1aadf1a6b026516e652a7aadd9c3d16805f8)

So, a CSPro apps Developer can program infinite constraints using whatever complicated business logic etc. Your limit is your mind... This is impossible with ODK because of the architecture it uses. And because of ODK limitations (edit a questionnaire is still in beta), I intend to use CSPro to output an error report for each questionnaire finalized by enumerators:

  • Get the ODK questionnaire XML data. I want to use activity and intent of ODK to launch ODK from a CSPro app an get the XML result in CSPro; This is the goal of this topic. :+1:
  • Transform XML data to JSON using existing JS libs;
  • Store the JSON data in CSPro's SQLIte database, CSPro natively read/write JSON data
  • Run a specific logic on questionnaire's data and output an error report in HTML, PDF or Datatables.

So, the enumerator can correct errors before sending data to ODK servers.

Which specific limits of ODK constraints etc. do you address?

I can only tell, it is impossible to program other than basic (maybe low medium) contraints without a real programming language able to interact with collected data and the business logic.
Here 's a basic example :
Cascading sequential numbering - CSPro Users Forum.pdf (515.6 KB)

Cascading sequential numbering
Best

1 Like
5

If you have been using CSPro, you understand very well why I want to use it for "On device" data quality control for data collected with ODK related technologies. From the way I'm working on Censuses/Sureys, mainly near real-time data quality control and integrity verification, Field Check table etc, my request is a general purpose one.

Do you have an idea on how it is possible to access to on device ODK collect data even before enumerators are submitting collected data?
Thanks in advance
PS:

incl. several things easier to implement

Yes, the ODK Developer team is working very hard to improve and add new features. However, if Python and R (far more complicated than CSPro Logic) are dominating data science and data engineering world, it is because it's almost impossible to collect/process other than simple data without strong programming languages... Latest CSPro versions give this power and flexibility that any apps developer like and will always prefer.

7

There is currently no way for another application to get access to ODK Collect's collected data. As you highlighted, ODK is typically aimed at non-developers so integrations with other applications are not a priority at this time.

In this case, it sounds like your best bet would be to use CSPro exclusively. Can you say more about what role ODK plays in your context?

Alternatively, you could explore the feasibility of implementing some of the specific logic you have in mind in ODK forms. I didn't carefully look at the "cascading sequential numbering" example but it looks like something that could be done with a combination of repeats and Entities depending on exactly what the requirements are. It would not look like CSPro's procedural approach but there's a surprising amount that can be done in XPath which is the query language that ODK uses. As a side note, when using Entities, XPath queries are automatically converted to corresponding SQL queries.

1 Like
8

Thanks a lot @LN for your responses. As you know, because of the innovative approach of low code/no code, ODK is very popular and widely used compare to any other existing CAPI/CAWI system.

it sounds like your best bet would be to use CSPro exclusively

Almost all humanitarian/NGO/UN organizations I know are using ODK related technologies, not CSPro. However, when Data Collection apps require strong quality control, ODK technologies require extensive post processing steps. It's why I wanted to implement some kind of "live" and on device quality control.

XPath which is the query language that ODK uses... I didn't carefully look at the "cascading sequential numbering"

As, I mentioned it, it will be very well for ODK, who is switching now to the JS ecosystem, with Web-forms, to give users a way to directly use SQL and JS for constraint programming. In the meantime, can you take a look at the very small and simple problem solved with CSPro logic for the "cascading sequential numbering" and help me to solve it with entities/Xpath?
Thanks in advance,

9

Can you please open a new thread with more details about the workflow? I don't know enough about CSPro to figure out how/where/to what the code in the thread you shared applies. Some of the things that would be helpful to understand:

  • How/when are concessions and households identified? Are they known ahead of time vs. discovered in the field? It looks to me like the work described in the thread you point to is done as part of project setup but I can't tell.
  • Does a single person do all work related to a single concession? A single household? Are all households in one concession identified/visited sequentially? I'm trying to understand whether or not repeats are appropriate in this context.

In general, the more detail you can provide on the desired workflow, the more likely it is I'll be able to know whether there's a good ODK-based approach. You may want to describe what the process would look like if you had to use paper so that it's not tied to any particular system's model.