Questions about ODK Central submission downloads: csv, xml, and audit

[jpringle]

1. What is the problem? Be very detailed.

Original, individual XML submissions

I am trying out ODK Central, just having switched over from ODK Aggregate. It looks like the download options from the web interface and from the API are for the aggregated CSV file only, and not the original, individual XML submissions. Is that right? Is there anyway to get the XML files and audit.csv files in individual folders representing the original submissions like from ODK Briefcase and ODK Aggregate?

Individual attachments, e.g. audit.csv

Say I have a form definition that has an audit.

When I download submissions for this form from the web interface or from the API with attachments, I get a folder in the zip called "media". In that folder, there is only one audit.csv file. Is that because all audit.csv files are named the same, so the software only keeps one of them (possibly at random) due to name collisions? If the submissions had photos and the filenames were the timestamp of the photo, then there wouldn't be name collisions (with very high probability) and all pictures would be in the "media" folder, is that right?

Combined audit.csv file

Consider the combined audit file (that is just one CSV) in the zip from downloading all submissions. Is it just all audit.csv files concatenated one after another in some particular order? From the logging docs, all I see is:

Central will export a CSV with audits from all submissions if an export is requested for a form with an audit.

The problem here is that with ODK Aggregate, I had the original, individual submission.xml files in folders alongside the original, individual audit.csv files, so that I could look at them together. This helped me know what submissions were too fast / too long, etc. But it looks like I do not have access to that information with ODK Central, but I hope I am wrong! In the combined audit.csv, I do not know which audit is for which submission.

2. What app or server are you using and on what device and operating system? Include version numbers.

ODK Collect v1.24.4
ODK Central client (v1.1.2) and server (v1.1.1)

[Matthew_White]

Hi @jpringle!

@jpringle

It looks like the download options from the web interface and from the API are for the aggregated CSV file only, and not the original, individual XML submissions. Is that right?

You can also download all submissions as JSON via the OData endpoint, but there's no endpoint to download all submissions as XML. However, it should be possible to use ODK Briefcase with ODK Central to download the XML. It's also possible to use the API to download an individual submission's XML. It'd be great to learn more about how you're using the data and why it'd be useful to download all submissions as XML rather than CSV or JSON.

@jpringle

I get a folder in the zip called "media". In that folder, there is only one audit.csv file. Is that because all audit.csv files are named the same, so the software only keeps one of them (possibly at random) due to name collisions?

Exactly. All the audit.csv files are in there, and in theory, the .zip software could offer a way to extract them all. (That said, I don't actually know of any .zip software that does so.)

@jpringle

If the submissions had photos and the filenames were the timestamp of the photo, then there wouldn't be name collisions (with very high probability) and all pictures would be in the "media" folder, is that right?

Yes, that should be right!

@jpringle

Consider the combined audit file (that is just one CSV) in the zip from downloading all submissions. Is it just all audit.csv files concatenated one after another in some particular order?

@jpringle

In the combined audit.csv, I do not know which audit is for which submission.

The combined audit.csv should contain all the records that the individual audit.csv files do, and the intention is for the combined audit.csv to match the combined file that Briefcase exports. However, currently as of v1.1, there's a bug whereby the combined file doesn't have an instance ID column (see this topic).

[Matthew_White]

As an update, as part of ODK Central v1.2, the combined client audit CSV export will now include the instance ID. Thanks, @jpringle, for reporting this issue!