Audit Timestamp Not Chronological

Support
1

Hi All,

I have been programming our survey using ODK, using KoboToolbox as a server, and using ODK Collect in the field.

I have been trying to create a valid method of evaluating the length of an interview. The ODK documentation touts the audit log as the most consistent method for evaluating time per screen, total time, etc., but I have run into several issues. The major issue I cannot determine the cause of is described below:

1. What is the issue? Please be detailed.
The times in the start column in the audit log do not always appear chronologically.

That is, the time stamp for event A (which appears first if we are reading the audit from top to bottom) will be greater than the time stamp for event B (which appears second if we are reading the audit from top to bottom). For another example, the data appear like the following:

event | start | end
event A | 0091 | 0092
event B | 0089 | 0090

This issue has occurred in 8 out of 10 countries my team surveyed. In each of the eight countries, this happened for anywhere from one to nearly 300 interviews.

The "negative" events (i.e., events which happen before the event that precedes them in the audit file) are most commonly "form resume", but "question", "form start", and "location tracking enabled/disabled" also are associated with "negative" times.

2. What steps can we take to reproduce this issue?
I do not know. I'm not sure why this is happening in the first place.

3. What have you tried to fix the issue?
I have looked at the dates the surveys in question took place. They (the surveys where the start timestamps are not chronological) are not clustered by date either within or between countries.

I checked to see when/if we changed the audit log settings. The audit log settings were edited at a point, but these "negative" times were recorded up to several months later.

I have checked to see if the total interview time makes sense regardless, e.g., subtracting the min from the max start time, summing the length of individual questions/question groups, etc. There are instances where the interview time seems reasonable (e.g., 40 minutes) and times when they don't (e.g. 6 minutes).

I have read through the documentation on audit logs several times and searched the forums, but (un)fortunately, I haven't found record of others having this problem.

4. Upload any forms or screenshots you can share publicly below.
I have attached a csv of the "event", "start", and "end" times of an audit log with "wrong" timestamps.
example_audit.csv (19.9 KB)

I am asking why does this happen? If anyone knows how to solve this issue, that would also be greatly appreciated!

2

Hey @mcroche :wave:. Welcome to the forum. If you get a moment, pop on over and introduce yourself here.

There's some known problems with the audit log and fixing these is part of upcoming work on Collect. We've seen both inconsistent ordering of events and also missing events in certain scenarios although events appearing in the wrong order chronologically is not something that has been noticed before (at least that the team has been made aware of). Are you able to highlight an example in the attached CSV? That'd help save us the time of searching through the log!

1 Like
3

Thanks for the quick reply! I've attached an Excel version with the specific events that occur "before" their predecessor. The fourth column (excluding row names) is the difference between the successive start values. All the events in the red box happened before the last event outside the red box (i.e., all row_names > 401 have start times before row_name = 400 start time).
example_audit_highlight.xlsx (27.4 KB)

4

Right I see. So a key example here would be in row 401 (row_name 400) where we see a "form resume" that looks like it's happened before the last "form exit"? Are you able to reproduce this kind of problem dependably, and if so are you able to share the steps and the form you're using?

1 Like
5

No, I am unable to reproduce the problem. It seems to potentially be dependent on location. Our organization does surveys in North Africa and the Middle East. In some countries the error does not occur at all, while in others it occurs relatively frequently, although randomly from what I can tell. I do not know what triggers the error and I am not in the field.

6

Do you use standardized devices or let data collectors use their personal devices? Any information you can share about the device types used in the two regions would be helpful. In particular, do they tend to use different Android versions? And/or is there a particular device manufacturer that is favored in the region where it happens a lot?

2 Likes