Collect and Central out of sync because of multiple Entity processing

I think this is different to what @Thalie & @mathieubossaert are seeing, and it's probably not isolated to multiple entity forms, so please split/merge to a new/different topic as needed, I couldn't find a more relevant one after a couple of searches.

I was testing a form that could create for 3 entity lists and create or update 1 list, due to a (my) bad relevance on all the create lists' form groups, they became relevant when they shouldn't have been and as they had no value in the field that fed the label, this caused the failure.

As a result, because entities 1-3 failed this also caused entity 4 not to be processed in Central - this is all expected. However in Collect, opening the form again did show the change to the entity from list 4, so it was now out of sync with Central. Syncing the project would not correct this until I made an edit to the entity in Central, and after the next sync the entity item in Collect matched Central.

Is this because you made an edit to that Entity's values in Central? Because its values depended on the other Entities' values? Something else?

That makes sense, when the remote and offline Entity have the same version, we currently favor the offline one.

My not-very-satisfying response is that we consider this kind of scenario unlikely after the testing phase so we don't currently plan to address it differently in the near-term. If you do come across a situation in which it happens during a real activity, please share so we can reconsider.

I think our best bet in the near-term around this is to improve the form design and testing experiences.

I reverted my form to the 'bad' version and stepped through it to get a clear picture of what happened.

  • set an incorrect relevance on one of the create entity lists (user)
  • synced project
  • submitted from device a 'create' for another list (item), successfully created new item (v1) and also new user
  • submitted from device, an 'update' to the item, but as no value existed for the field that was used as the 'user' label the entity processing on it failed, which caused the the item update v1->v2 to fail.
  • loaded form on device: the change to the updated item's fields are visible, repeated 'update' on a different entity item so I now had 2x failed updates.
  • checked form folder -
    • The last-saved.xml showed the modified values that didn't make it to v2 in Central
    • there were no CSVs in latest definition folder, older definition folder CSVs didn't have new user / new item
  • synced project
  • checked form folder
    • CSVs now present in latest definition folder, item CSV contains v1 items without the changes, user CSV does contain new user (successfully processed with first item 'create')
  • modified item A in Central, --> v2
  • synced project
  • Collect now shows v2 of Central modified item A, but still shows the local modified version of item B
    • however the CSV content matches Central for both item A & B.

Wherever the offline version exists (last saved, somewhere else?) that should be v2 right? But when I updated the item that had failed processing in Central causing it to go from v1 to v2, and then synced, only then was Central-v2 seen in Collect, before that Collect would only show me the unprocessed "Collect-v2" version of that entity.

My form definitely had relevance issues wrt some of the entity lists, so a well designed and tested form should not see this out of sync issue.

Update...

  • Removed the bad relevance, published, synced, Collect still shows the 'Collect-v2' version of the one that was never processed and hasn't yet been edited in Central.
  • Reset form load cache, no change
  • Reset Blank forms, no change
  • Reset Saved forms, no change
  • Reset everything - start new form screen no longer has a 'sync' button.
    • No forms available, unable to sync. Saw a brief blue pulse but no result.
    • Shared QR from this device to another (which already had the project) and it also now has no sync button but the forms were still present.
    • Checked Server settings and URL was now https://demo.odk.org ! [1]
    • Attempting to send from device 2 resulted in an error 'Could not find the resource you were looking for'.
    • It was the 'All settings' reset that caused this, doing it in isolation leaves the forms / saved forms etc but the project is now unusable unless the server URL is restored
  • Reconfigured from QR - sync button restored, forms restored, but still shows 'Collect-v2' version.
  • Deleted last-saved - no change.
  • Deleted project, readded project - fixed, now shows Central-v1 version.

the 'Collect-v2' value must be elsewhere, like entities.db I guess.


  1. this clears up an issue someone reported last year where the project suddenly had the odk URL, but kept my forms, and nothing would send. they must have 'reset all settings' and forgotten what they did. GAH! ↩︎