Updating and creating entities with one follow-up form

[dast]

Hi,

I am testing entities for our purposes. To this purpose I have created a form that creates an entity list of the enumerators to replace the .csv file we are using at the moment.

This forms works fine and entities are correctly created.

I have build a second form very similar to the first to modify the entities. Works fine as well.

For simplicity reasons I now want to use the same form to update AND create new entities in the same entity list. This fails.

To find the error I have removed in the entities tap the label column. In this case, the form manages to create new entities but cannot modify existing entries. It seems as if updating and creating new entities is not possible with the same form, although this procedure is descibed in the tutorial:

Creating AND Updating Entities with one Form
You can give both a create_if and an update_if rule for the same Form. If only one of these expression evaluates to true or 1, then only that operation will be carried out. If both rules evaluate to true or 1, the Entity will be created if it does not exist (as identified by the entity_id expression), and updated if it does. Of course, if neither rule evaluates to true or 1, no Entity changes will occur.

I attach the xlsform.

Thanks you for any help.

Best, Daniel
admin_inspecteur.xlsx (13.0 KB)

[LN]

Currently your entity_id expression needs to explicitly generate an id in the create case. You can do this with an expression like if(${inspecteur_code_existant='', uuid(), ${inspecteur_code_existant}).

Another way to express this same idea is using the coalesce function: coalesce(${inspecteur_code_existant}, uuid())

I think you were getting at a similar idea with inspecteur_code but that is a user-provided code, not a system ID. You can have both but each entity must have a UUID as its entity_id.

Here's a similar example you may find helpful: https://docs.google.com/spreadsheets/d/1poe2lx_MUzbN0ySuZlRX8_fuKqM4LdsBMwKORQW1QRk I'll work to improve docs around this case.

[dast]

Thanks LN! I'll try your solution as soon as possible and give a reply.

[Devendra_Patankar]

Thanks Great help for same problem.

[dast]

Thanks @LN for your support.

It worked with the example you provided. I now try to apply this to another long survey but fail with the following error message when uploading the xls form on the server:

The entity sheet looks like this:

When removing the content in the columns update_if, create_if and entity_id the upload works fine.

Any idea?

[ahblake]

Boring stuff

I tried multiple ways to write the entity_id expression, and none have worked for the 'create' case, but all work for the 'update' case. ${item_to_update} is the selected entity list item, ${action} is the question to either create or update

Tested in Enketo (update worked, create didn't)

• coalesce(${item_to_update}, uuid())
• if(${item_to_update}!='',${item_to_update},uuid())
• if(${action}='update',${item_to_update},uuid())

Tested in Collect (update worked, create didn't)

• if(${action}='update',${item_to_update},uuid())

And, then I looked at the example linked in this thread and saw two other columns create_if and update_if! These are in the docs, but aren't in the 'trees' example form linked from the docs and the docs don't indicate that you must have them to allow change/update functionality @LN.

So I added ${action}='update' to update_if and ${action}='add' to create_if and reverted to entity_id = coalesce(${item_to_update}, uuid()), works in Enketo for create & update!

For others - the docs aren't explicit - if you want to either create OR update in a single form, you must have create_if and update_if columns in the entities sheet of your form, with expressions that evaluate for those cases. Then the entity_id column must also have an expression to return the ID of the item being updated or generate a UUID if being created. eg

• create_if
  • ${create_or_update}='create'
• update_if
  • ${create_or_update}='update'
• entity_id
  • coalesce(${existing_item},uuid())

[Joshua_Ayanlakin]

Hello!

I have a similar issue with this form I scripted for immunization campaign to track missed children resolved on previous days from e-tally. I found it difficult to load it on ONA, I dont know if it could only load on ODK Central or something is wrong with the form. Please help me. Below is the error code I get upon trying to load the form on ONA:
ODK Validate Errors: >> Something broke the parser. See above for a hint. Error evaluating field 'baseVersion' (${meta}[1]/entity[1]/@baseVersion): The problem was located in Calculate expression for ${entity}/@baseVersion XPath evaluation: Instance referenced by instance(Trees)/root/item/__version does not exist Caused by: org.javarosa.xpath.XPathMissingInstanceException: The problem was located in Calculate expression for ${entity}/@baseVersion XPath evaluation: Instance referenced by instance(Trees)/root/item/__version does not exist ... 10 more The following files failed validation: ${tmphs8ch1xl} Result: Invalid

Your file should have the .xlsx, .xml file extension because Ona uses XLS Forms to create forms.

Not sure what to do? Read more aboutUsing XLSForms

Thank you.

[dast]

Can you share your form?

[Joshua_Ayanlakin]

Dear Dast,

Thank you for your response. Please see attached the form and follow up.

missed children follow-up.xlsx (79.1 KB)
form.xlsx (389.5 KB)

[MinimalPotato]

Hey @dast & @Joshua_Ayanlakin !

Looks like you’ve run into a tricky little bug in ODK Central (yep, this one existed in ODK Aggregate as well). It’s one of those sneaky issues that’s really hard to catch - BUT, I happen to know the fix! I’ll first explain what’s happening and then share the solution.

The Explanation

A few years ago, when I was just getting started with ODK, I spent a lot of time reading the docs, testing things out, and basically trying (and failing) over and over again because that's the best way to learn, right? (For the record, I failed 300+ times while learning about ODK and 10,000+ times while learning to code in general! )

During my early ODK experiments, I ran into quite a few weird bugs - which I ignored as I managed to fix them with quick workarounds or tweaks. One of those bugs popped up when I was exploring the pulldata() and instance() functions.

Now, for many users - these two functions seem to do the same thing: looking up and pulling data from an external file (like a .csv). At first, I was confused too. I asked myself - "If both functions serve the same purpose, then why have two?". My guess was that the developers designed pulldata() to be the simple, intuitive option, while instance() was meant for more advanced filtering and use cases. So, naturally, I decided to experiment with both. That’s when I stumbled upon the very same issue you’re facing now!

What did I find?..

• When you use pulldata() to lookup and fetch data from an external file, it automatically loads the .csv onto the device.
• But when you use instance(), it does NOT automatically load the .csv - which is totally unexpected behavior! Without loading the file, the form simply won’t work.

So, if you're using instance(), you have to manually force the file to load.

Now, here’s the connection to your issue:

• The "update_if" and "create_if" fields in the entities sheet behave just like the instance() function in this case - which means you’ll need to manually trigger ODK to recognize the external file / entity list. But how? Check the next section!

The Solution

To make this work,

Step 1: Add the following row at the top of your survey sheet:

type	name	label	choice_filter	relevant
select_one_from_file EntityListNameHere.csv	anything_here	anything_here	false()	false()

Just replace "EntityListNameHere" with the exact name of your entity in the entities sheet (under list_name).

Step 2: Manually create the entity list in ODK Central with the same name. No need to add any properties - just creating the entity list will do!

Step 3: Since we’re not using select_one_from_file to get input from the user to update the specific entity record (because if we were, we wouldn’t have run into this issue in the first place ), you’ll need to use something else - maybe a text field, a fixed calculate field, or any other method that fits your form’s logic (like maybe you wish data collectors to scan the barcode for record selection rather than selecting from a list). Just make sure you’re capturing the data in a way that allows you to update the entity record using a specific entity_id.

Step 4: Upload the draft, then publish! Once you do that, your form should start working!

Final Note

@dast - This is why @LN ’s example works. It includes select_one_from_file participants.csv, where "participants" happens to be the dataset/entity list name. Try changing it to "select_one_from_file abcd.csv", and you’ll see the same error pop up again!

@Joshua_Ayanlakin - While this fix will solve your entity-related issue above, I have to say… I’d recommend making a few changes and check the logic in the form for better usability. You might need to add uuid() under the entity_id field and introduce a label field, develop further logic, etc.

Hope this helps! Wishing you a great day!

[Joshua_Ayanlakin]

Many thanks for your response. I will try this out and provide feedback on the outcome.

[LN]

I'm sorry @dast, I see I never got back to you in this thread!

@MinimalPotato is right that the (terrible) error message Instance referenced by instance(Trees)/root/item/__version does not exist implies that the Trees Entity List is not attached to the form. If in your case it's appropriate to show Entities in a select, the approach @minimalpotato described with select_one_from_file would be a good one.

It looks like what you had been trying to do is generate an ID within your form and use that to make an update without having the Entity List attached. That's not currently something you can do. If you can change the structure to use a select, that will be your simplest approach. If it makes more sense in your scenario to look up the right Entity to update based on a calculated ID, you could use csv-external to attach the Entity List without a select. I think that you'll find this documentation helpful.

In order to update an Entity, you need to have its system ID. If you want to use an ID that's calculated or scanned from a barcode, you'll need to look up the system ID based on another property's value. The system ID is accessible through the name property (this is to make it easy to build selects on Entity Lists). You will need a calculate that looks something like

instance("trees")/root/item[barcode_id = ${barcode_id}]/name

You will then reference that value in the entity_id column on the entities sheet.

Here's another thread discussing this: Why is my submission with a barcode id as the Entity id failing to update the appropriate Entity? - #2 by LN

[Joshua_Ayanlakin]

Thank you @ LN. I appreciate your intervention. I am trying to review the form and will post the update here once am done.