Downloaded two different forms with the same form ID and version

Hi all,

Just had a question about this error:

You have downloaded two different forms with the same form ID and version. Maybe they were the same form uploaded at different times or different servers. In any case, you should delete one.

I understand why this error appears. However, it only appears after the second form (with the same id and version as another form) is filled in.

Is there a reason why the second form is allowed to be downloaded, filled in, but breaks on attempted finalising? Surely, if ODK knows that a form with the same form ID and version already exists on the phone, it should prompt the user to this knowledge and not allow the download?

Thanks,
JD

I think the reason it doesn't do this preemptively is that it doesn't seem like a common case -- every extra check is more code to maintain and possibly have bugs in! Servers generally wouldn't allow use of the same form id and version for different form definitions in the same project so something unusual would need to happen. Can you please describe the actions that led to this state and what need drove it?

Hi @LN,
Thank you for your prompt reply.
I've attached an explanation of what we do below (copied from a previous post I made):

"We work with water companies internationally to identify leaks on customer supplies. This involves our site team to visit the external stoptap (a hole in the ground with access to the water pipe), where we attach our sensor. It is important for us to record the connection/disconnection date and varies bits of information about the stoptap. However, a stoptap chamber can have multiple supplies (pipes) and a supply can have multiple properties associated with it (i.e. one pipe can feed multiple houses). To get around this many-to-many relationship, we have one set of forms for the property and another set for the pipes (and more forms for when we locate a previously unknown property or pipe). So, we're essentially using the forms as a jobs list.

This process requires multiple visits and multiple operations team members to carry out the work."

We generate the forms on our custom server and then send them down to the forms. However, the data in the instances of these forms changes all the time. So our guys often try to make sure they have the latest data on the forms. For example, we have a single form for retrieving multiple loggers. This form takes in the serial number of the logger the person is retrieving, uses the 'calculate' function to show them all of the information gathered during deployment of that logger (so that they can make a note of anything that has changed in the meantime), and they send the form up to our sever. There are cases where our server will throw a 500 error, i.e. if they are trying to retrieve a logger that has not been deployed, meaning that finalised form (and it's original blank form) will remain on the phone. If they then Get Blank Forms, they will download the new form with the same form id and name, but with different instances. When they then try to fill out either the existing form or new form, the error "Not exactly one blank form matches this jr_form_id. Form has not been saved as finalised" is shown when they try to finalise the form.

There are a few ways we can solve this issue. One would be to include a hash (i.e. md5) of the form in the form id, ensuring that any change to the instance would be seen (by ODK) as a different form. However, this is quite a large job given the number of forms we would have to hash.

I hope this explains the issue we're having. At the moment, we're explaining to our team to delete all blank forms before getting new one, however this does not always happen.

There did turn out to be a slight change in behavior of form deletion in v1.28 that made this error more likely and harder to recover from. We have made further changes that should alleviate this and it's available in beta. We experimented with also preventing downloading a second form with a form_id and version combination already on the device. Unfortunately, we found out that replacing a form with a matching form_id and version has become part of some workflows so we can't do it yet.

Collect has the expectation that a form_id and version combination identifies a unique form. In your case, it sounds like you could generate a random version for each form that you generate? By default, as updates are downloaded, only the latest downloaded version would be seen by your data collectors.

Hi @LN

We just got this week one isolated case on ODK Collect v2022.2.2 for a project set up using an app user configuration code (so exactly match server settings). My initial hypothesis was that this tablet may have been used by my in-country counterpart when testing drafts before publication, as this is where I had previously experienced (rarely) this issue and I think what you had mentioned in another thread on the forum, but when checking the saved blank forms, it seems that the two forms with same ID/version were added at the same time on the device. Maybe a network issue that resulted in a request duplication when retrieving the forms?

The problematic blank form was successfully deleted and collected data properly transferred to the ODK Central server (v1.4.0), so I am documenting here in case it might be helpful somehow. Also let me know if any additional information may be helpful.

3 Likes

Thanks so much for sharing your experience, @Thalie, and for your offer to investigate further. With intermittent issues, hearing about when they happen is really helpful.

As luck would have it, I experienced this last week and @Grzesiek2010 has been able to identify the cause. Under certain conditions, multiple requests for the forms are issued and allowed to complete at exactly the same time. We're working on a fix and will release it as a patch.

2 Likes

That's great news, and well done to @Grzesiek2010 and you for tracking down the cause of this unexpected behaviour :clap:

1 Like