ODK Collect: Time in Repeat Changes on Submit

Support
1

1. What is the problem? Be very detailed.

We use ODK Collect with a form on a Google Drive. The form has a repeat question that asks for the time of an observation. If the data of a form is edited, specifically, if one of the repeats is deleted prior to sending the form, the recorded time for all the repeats is shifted +1 hour for all the repeats in that form. That is, if we check the Google Sheet that receives the data, as well as on the device in the "Sent Forms" the time is changed. The change only happens upon send, and it does not happen if repeats are not deleted.

2. What app or server are you using and on what device and operating system? Include version numbers.

ODK Collect v1.22.4 and Google Drive

3. What you have you tried to fix the problem?

We have a "test" entry that we can use to not have to delete entries of the repeat.

4. What steps can we take to reproduce the problem?

Build a form with a repeat questions that asks for the time, then delete repeats before sending.

You can also use our form and sheet. In the latter case, you will have to upload the Excel file as a Google Sheet to a Google Drive of your choice. Then adapt the submission URL on the settings sheet accordingly.

5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.

Excel sheet
XForm definition

Many thanks for your help! Also, ODK Collect is great and will streamline data collection greatly, even if the described behavior persists.

2

Hi @Roger welcome to the forum! Please introduce yourself here!

I tried reproducing the issue you're having but it looks like the form needs a .csv file to pull bee-ids from. Do you have an example of that?

3

Thanks for looking into this. I indeed forgot to add this csv file. Here it is:

CSV File with Bee IDs

Note that this resides in a subfolder of the Google Drive folder where the form is stored. The name of the subfolder is bees-neonic-visits-odk-media.

Oh, and I introduced myself in the thread you linked. Many thanks again!

4

@Roger really sorry for the late reply! I'll a look at that and see if there is anything I can understand from it.

5

@Roger sorry I'm having some problems filling in the form. Could you give me some exact steps to get to the problem you encountered (i.e what questions to fill in and what to fill them in with etc)?

6

Hi @seadowg! Thanks for looking into this ... What problem do you encounter when entering the form?

After downloading the form to ODK collect, we go to "Fill Blank Form". There we set the phase of that particular experiment, and the date for the observations that are to follow. Then we swipe from right to left to get to the actual data, which is implemented as a repeat. We need to pick a feeder, an observation time (in my case it is 14:03 now), and any bees that have visited the feeders, let's say "Test Bee". Now, we swipe left again, and we are asked to add another group, and we add a second group. The next time, instead of adding another group, we finalize and save the form. Now I go to "Edit SavedForm" where I select the form that I just filled in. I go to the "Visitation Repeat", select the "2. Visitor Information" (the time here is still correct), and I delete that. Then we save and finalize the form again. Now I go to "Send Finalized Form" and send the form I just edited. Now I go to "View Sent Forms" and select the form I just sent. This brings me to the date and experimental phase screen. I hit the "up" arrow at the top right corner to get to the page where I can select the "Visitation Repeat". Looking at the "1. Visitor Information", I now see that the observation time has shifted by 1 hour (i.e., it now says 15:03), and that is how the form's data are saved to the Google Sheet, too. So, only for edited sheets, upon sending the sheet the time is changed.

These times are essential for our data analysis, so it is lucky that we caught the shift, and we could reconstruct the original times so far. But it would be great if we could understand (and hopefully fix) what the underlying cause is.

P.S.: Is there an easy way to make screen casts of Androids? If so, I could demonstrate our particular case.

7

I ran through this with my Collect pointing at an Aggregate server and couldn't reproduce but I'll have a look at Google Drive once I set that up for myself.

I've also asked if someone can help out with how to do a screencast :grin:

8

@Roger I can't seem to reproduce this on Google Drive either. The times come through fine for me.

I'm interested to see if there's any possibility this could be a timezone issue: what timezone is your phone (the one with Collect installed on) set to?

9

That is odd, we have at least 3 phones that had the issue. We are in the Eastern US, so it is set to that time zone.
We also had more hourly shifts if more than one observation was deleted, so I am not sure how the time zone would play into that. Is your ODK Collect app the same version?

10

Yeah tried on 1.22.4. Was just hunch that it could be a programming error in Collect relating to timezones but I don't think that's right. I can't seem to reproduce it but going to see if anyone else can have a look.

11

@Roger @seadowg
I was able to reproduce the issue so I'll try to figure out what's going on or what kind of sorcery it is.

2 Likes
12

Thanks @Grzesiek2010 and @seadowg for your continued efforts to find the cause of this. In the meantime, we still collect data with the app, and it is awesome to just run it through the prepared scripts and getting live updates! :smiley:

13

Dear @Grzesiek2010, I wondered whether you had found out anything more about the issue? Is there any way I could contribute to the search for the culprit?

14

I created a pull request with a fix but it waits for review so we just have to wait.

15

Thanks for your patience as this went through troubleshooting, @Roger. The problem is that in a location that experiences daylight savings time, there is no consistent way to interpret a time recorded without a date. The behavior you are experiencing is because you are currently in summer time in your location, but when you open the record, the time field is interpreted with a fixed date of January 1, 1970 which is in winter time. When you make modifications to the record, that time which was shifted by an hour is written back out and saved. You can read more about this at https://docs.opendatakit.org/form-question-types/#time-widget

@Grzesiek2010 has suggested using the current date to interpret times on record re-entry. This would appear to work correctly when you're within a single timezone (summer or winter time) but would fail again in the same way around daylight savings time boundaries.

I see you've designed your form so that the date is collected once outside of the repeat and then only time is collected in the repeat. That makes sense but doesn't give Collect any way to link the date and time. I can think of two things you could do:

  • use a date time question type. That way date is always included with your time so there is no ambiguity in the timezone. I know it's inconvenient to enter the date each time but since the date picker defaults to today's, it's only adding two taps. If the date is not always today's date or if those two taps are too much, I think you could use the date collected outside the repeat as a default inside the repeat. I can show you how if interested.
  • if you're always collecting the current time, you could use a calculation and the now() function to capture a date/time without prompting the user. Be sure to read the usage notes and to use something like once(now()).
1 Like
16

Thanks @LN, I see. The reason why we collected the date outside the repeat was because we are observing live animals, so there is a time component to the collection of the data. But then the now() function should probably work pretty well? I am just a beginner with XLSForms/XForms, and I had difficulties getting the functions to work in the short time I had to come up with the forms. The current experiment is over, but I will make an effort to learn more about this for the future. And let me re-iterate how happy I am to have stumbled across this. Thanks for all your work that goes into the development! Should I mark this as the solution for now?

1 Like
17

Congrats on getting your first forms put together! Please don't hesitate to ask for a check over your forms next time you have data to collect. The community here always has great form design suggestions.

Yes, once(now()) will collect both date and time so if you're always collecting data in the present (rather than entering data about a past observation), you should be able to replace asking the data collector for any date or time information. I'd recommend doing an extra check to make sure their device date and time are properly set.

That would be great, thank you!