I have been programming our survey using ODK, using KoboToolbox as a server, and using ODK Collect in the field.
I have been trying to create a valid method of evaluating the length of an interview. The ODK documentation touts the audit log as the most consistent method for evaluating time per screen, total time, etc., but I have run into several issues. The major issue I cannot determine the cause of is described below:
1. What is the issue? Please be detailed.
The times in the
start column in the audit log do not always appear chronologically.
That is, the time stamp for event A (which appears first if we are reading the audit from top to bottom) will be greater than the time stamp for event B (which appears second if we are reading the audit from top to bottom). For another example, the data appear like the following:
event | start | end
event A | 0091 | 0092
event B | 0089 | 0090
This issue has occurred in 8 out of 10 countries my team surveyed. In each of the eight countries, this happened for anywhere from one to nearly 300 interviews.
The "negative" events (i.e., events which happen before the event that precedes them in the audit file) are most commonly "form resume", but "question", "form start", and "location tracking enabled/disabled" also are associated with "negative" times.
2. What steps can we take to reproduce this issue?
I do not know. I'm not sure why this is happening in the first place.
3. What have you tried to fix the issue?
I have looked at the dates the surveys in question took place. They (the surveys where the start timestamps are not chronological) are not clustered by date either within or between countries.
I checked to see when/if we changed the audit log settings. The audit log settings were edited at a point, but these "negative" times were recorded up to several months later.
I have checked to see if the total interview time makes sense regardless, e.g., subtracting the min from the max start time, summing the length of individual questions/question groups, etc. There are instances where the interview time seems reasonable (e.g., 40 minutes) and times when they don't (e.g. 6 minutes).
I have read through the documentation on audit logs several times and searched the forums, but (un)fortunately, I haven't found record of others having this problem.
4. Upload any forms or screenshots you can share publicly below.
I have attached a csv of the "event", "start", and "end" times of an audit log with "wrong" timestamps.
example_audit.csv (19.9 KB)
I am asking why does this happen? If anyone knows how to solve this issue, that would also be greatly appreciated!