R package odkapi for REST API

Development
1

I have uploaded a first version of an R package to use the REST API to github.

Work on this package came to a halt because of higher priority COVID-related stuff, so sorry for the lack of documentation.

Overview

This is an almost complete implementation of the ODK Central REST API for R. I currently cannot continue work on it because of other priorities, so I make it available at an early alpha stage.

Features

  • Uses R6 classes to implement token-controlled access to the API
  • Always returns data frames
  • For a slim footprint, it does not use any function of the tidyverse , but only a the absolute necessary subsetjsonlite, httr, R6, glue. I like tidyverse function for short-lived reports, but have been bitten very often its frequent depreciations when maintaining a package on CRAN that uses the tidyverse functions. The tidyverse has well-tested functions to unnest data used in ruODK; for odkapi, it is planned to use XSLT for nested data, but these functions are not yet complete because my XSLT is rusty.
  • Naming is close to the function names of the online API which has excellent documentation that should be used at the current state.
  • I have tried to make the functions orthogonal to those in Florian Mayer's ruODK ; this package is a lower-level complement to ruODK - see below for a comparison.
  • odkapi has a full sets of tests against the online API, and against a mock version of the API using httptest, and against a live server. The testlive functions require a working server; random projects named testXXX and users will be generated and deleted, so these should not interfere with your working project, but better use a dedicated test installation - there is no guarantee that nothing goes wild, and removed projects do accumulate because deleting projects is not supported in the API.
  • It can generate QR codes; you could steal the code for your own use.

When to use and not to use

  • When you want to batch-retrieve and analyze data, use the higher-level implementation of ruODK. Most importantly, it is well tested with geo-data and images - this package was tested with text/numeric fields only. I also do not have tested odkapi for deeply nested data, and the XSLT-base transformation of is not yet working - XSLT specialists are welcome to work on the stubs.
  • ODK is very limited when you want to retrieve subsets of data, it was build mainly for batch processing; see this discussion. I have written this package as an interface to mirror the data in a local warehouse database, so that retrieving individual records with searches can be implemented.
  • My application is a system for managing forms and data entry in a clinic. Since the web interface is very nerdy - even after using it for several months I need half a dozen of mis-clicks to find the place where the QR code is generated - I use the REST API to create a simplified GUI with Shiny that follows the end-user workflow.
  • In theory, for small chunks of data odkapi might be a bit faster than ruODKbecause it uses tokens instead of passwords to connect to the API server. I have not tested this, though, because it was of minor importance for me.

Missing

  • Documentation and Examples. Use the ODK documentation in the API. The test code in tests/testlive shows usage examples and has almost complete coverage. While the API documentation is good, the returned mock data are sometimes incorrect; the live tests are more reliable.
  • Data returned via XSLT transformations (test-FormsApi_Apiary.R). Tests of these functions fail currently.

Thanks to

A first draft of the API was generated by converting the Apiary docu the swagger-codegen project. This gave a basic framework for the interface, but the generated output was unusable and had to be reworked extensively.

@florianm for his great work on ruODK; I use it heavily for batch reporting.

4 Likes
2

Thanks for sharing your work, Dieter!
Love the R6 approach (way above my pay grade but great to learn from).

If I'd ever get to rework the http layer in ruODK I'd probably use rOpenSci/crul and tokenauth too.

Others with similar needs might be in a good position to contribute and extend odkapi. (Maybe worth to mention in readme / contributing guidelines?)
How do you feel about pkgdown/GH actions for autodocs?

3

Package is private on github now. See

https://forum.getodk.org/t/re-r-package-odkapi-for-rest-api/32118

for the reasons.

1 Like
4

That thread is private too, can't access it unfortunately. Sorry to hear odkapi is no longer public, would love to learn why.

5

@Florian_May The name "odkapi" doesn't comply with ODK's brand guidelines. In the private thread, I asked @dmenne to consider renaming.

1 Like
6

In a first message, I wrote: "Menne ODK - An R package for ODK Central" is ok for a title, but it would be an unacceptable name for a github repository or R-package.

Do you have any realistic suggestion? I admit I do not understand the guidlines here.

After there was no response, I wrote:

Coming from R where the trademark R is in many packages even if not really required (stringr, Rcpp, RCurl..) I am a bit surprised about the strange suggestions in your guidelines for package names. I also do not want to put my or my brand name into an open-source package as you suggested; this will turn off other collaborators.

Since I did not get a reply, I have made the repository private to avoid copyrighting problems. If anyone wants to work on it, feel free to mail me for access.

7

So ruODK should be florian_meyer_down_under_for_odk`. Funny.

8

I understand that naming things is a sensitive issue and must adhere to many constraints. We R developers are additionally constrained through the conventions of the R ecosystem.

My choice of ruODK predates my closer knowledge of the ODK branding guidelines and also it was chosen prior to ODK's efforts to enforce them. By the time it came to a discussion, ruODK was already somewhat of an established name, under peer review with rOpenSci, and well underway to become a more widely used package.
I did indeed get a friendly slap on the wrist for clumsily co-opting the ODK logo into ruODK's hexlogo. That is now rectified through a new logo.
The TAB at the time gave me permission to keep the name ruODK but would have preferred if it didn't contain the letters ODK.

The odkapi package is a wonderful complement to the R/ODK ecosystem, as it targets an entirely different use case and is designed in an opposing architecture (Base R, R6 classes, minimal third party dependencies) to ruODK (tidyverse, S3 classes, and many dependencies useful for visualisation and spatial processing).

It would certainly be beneficial to many others if odkapi were openly available.
I don't have a good hand at naming packages, but if pressed I'd suggest something like centralapi, or maybe centralrestapi. Would something along the words of central, API, rest, R be acceptable?