Responses for maintained data fields reset to blank/NA after form definition update

[lauresp]

Hello,

I am running a pilot using ODK Central and the App Users function where I am regularly updating form definitions and pushing updates to App Users. After my last update, I have had a problem with lost data in fields/columns that were maintained across definitions.

More specifically, the values for a series of columns that exist in both definitions were deleted.

I'm attaching screen shots of a data pull made on 12/03, showing the columns with only one submission row where previously there were about 17. Will attach the screen shot of a previous pull I made on 11/23 showing the same columns with correct submissions in the first comment

I had been regularly updating form definitions before and after the 11/23 pull with no problem until the latest definition. I was pulling data with ruODK and OData, but I don't think this is the problem. The data for those fields truly gone - I can't access it even in the "edit submission" Enketo viewer.

The only thing I can think that might have caused this problem is that I added a few questions at the beginning of my form, so that subsequent questions are shifted down a few positions. It seems that the data for the 3 questions before the additions has been preserved.

Could this addition of questions have caused the data erasure? If not, what else might have? I'm a little worried about rolling out the finalized survey (planned for the next few days) without understanding what caused this loss!

[lauresp]

Screenshot of the 11/23 pull showing the data that was subsequently deleted

[yanokwa]

@lauresp The data isn't lost, it's just hidden. If you delete a field (typically by renaming or by adding/removing it from a group), Central hides that deleted field in export. In the next release of Central, we'll warn you about this behavior and we'll also let you export this missing data as well.

For now, you can add the missing questions back to your form, but set relevant as false(). This allows the data to show up in the export, but those questions are hidden to data collectors. You can learn more about this technique at Can you still send data from old versions of forms to ODK central - #5 by LN.

[lauresp]

Thank you, this worked to retrieve the data! I had not realized that I pulled it out of the group, now I see what the error was.