New entities feature

[prabha]

1. What is the issue? Please be detailed.

The entities list is a great feature. I have upgraded my odk-central version and I can see the new entities feature. Great feature and I look forward to leveraging it.

ODK Central v2023.4: More visible Entities and Single Sign-On

We are excited to announce a new release of ODK Central, the ODK server. What's new Entities let you share information between forms so you can collect longitudinal data, manage cases over time, and represent other complex workflows. You can now see Entities on the homepage along with the Forms for each project. This will help you see the total Entity landscape on your Central server. New to Entities? Watch this quick overview from @LN. [Introduction to Entities] Add partic…

I have gone through docs and tree registration and tree followup example. It works like a charm and will be very useful.

Managing Entities in Central

2. What steps can we take to reproduce this issue?

I have a few quetions:

1. I already have a farmer registration form that we have been using with the v2023.3.1. This form has been operational for the last few months.
2. I would like to use this farmer registration from as entities in a follow up form. If I update the form with the entities tab in the new version, will I see all the entities already collected automatically?
3. Is it possible to add a csv file as an entity list and then follow up with the farmer registration form?

Finally, I have th followng scenario:
Even after I have the entity list from the farmer registration form, I will have a situation in the field, where I want to register a new farmer and conduct the followup survey on the same device right there. And I will not have internet connection to upload the farmer registration, then refresh the list, then do the followup survey with the farmer.

How can I do this? And workaround?

Eg. would it be pssible to have an Other in the select_one_from_file and have an Others category and have the person fill the name and basic details do the followup servey?

3. What have you tried to fix the issue?

4. Upload any forms or screenshots you can share publicly below.

[Sadiq_Khoja]

Hi Prabha,

Thanks for trying the new Entities feature.

Unfortunately, you can't make Entities out of old Submissions because those Submissions were created using a Form that didn't have entities sheet and save_to details.

Regarding uploading CSV file to EntityList, that feature is in the pipeline. For now, you can use Create Entity API to create entity from the CSV file. If you know python you can use pyodk; If you don't know programming then you can use Postman for consuming APIs. Here is a short tutorial on how to use CSV file in Postman to upload data in bulk.

[prabha]

Let me be specific.

I already have a Farmer Registration Form that collects entities data in an older version of ODK Version2023.2 and currently we have about 2000 farmers registered.

I can modify the form and add the entities tab and save_to; and upload the form. No other change.

My question is, if I edit and upload a new version of the form with entities sheet and save_to details, will the already registered farmers be taken as entities for the farmer survey form? Will it work? Are the entities created at the time of submission? Is there a workaround?

Thanks

[prabha]

One more thought to create entities from an already existing form and submissions.

Can i add a new version of the existing form with Changing Entity List Settings. The new version of the form will have Create Entities when Submissions are marked as Approved and use an API to change the already submitted records to Approved. Will that work?

[Sadiq_Khoja]

No, existing Framers will not become Entities even if you upload a new version of the Form.

Workaround is to create Entities using Create Entity API. Here are the steps:

1. Download the CSV of existing farmers
2. Upload a new version of the Form that defines "Entity List" called "farmers" (with entities sheet and save_to details)
3. For each row in the CSV, call the Create Entity API

[prabha]

Thank you. Got it. Also looked at the API and we will be able to upload to the EntityList.

When I upload a new version of the "Farmer Registration Form" that defines the "Entity List" called "farmers", will the data newly collected be appended to the same table? This is because we are continuously using the "Farmer Registration" for further analysis.

[Sadiq_Khoja]

Yes, it will be added to the same table (Entity List)

[prabha]

Thanks. We are able to use the API to add the older submissions to the entities data.

But we wanted to know if we can add submission metadata fields into the entities data so that we can related the submission to the entities data. I have added a separate issue on this issue. Using the entities feature

[Lal_S]

Hi Sadiq,

Do you know if this is possible to add entities with @Florian_May's ruODK package?

Thanks,
Sham

[prabha]

Dear @Sadiq_Khoja ,

Apologies for mot being clear. I understand that entities cannot be created from older submissions. And I have managed to download older submissions as a csv and use the API to add the older submissions to the entity list.

But my question is; I would like to link the entities created with the submissions. So if I can have the uuid of the submissions in the entities list created that would be useful. Is this possible?

Prabhakar

[Sadiq_Khoja]

Currently there is no way to create link between Entities created via API and existing Submissions. However, you can create a new property like "submissionUUID" in your Entity List and populate its value from the CSV.

@Lal_S I don't think ruODK has Entities functions yet.

[prabha]

Thanks, @Sadiq_Khoja That is a good suggestion.

Is it possible to add the uuid of a submission as a save_to in the form so that newer submissions will have the submissionUUID?

Prabhakar

[Sadiq_Khoja]

yes

[prabha]

Oh, Nice. This is exactly what I wanted! Thanks.

Just one more question. I already have submission and entities deployed in production. And it is working well.

If I make this change and add the submissionUUID to the entity, will it work? Will the newer submissions have the submissionUUID in the entity list?

Thank you. This was exactly what we wanted.

Prabhakar

[Sadiq_Khoja]

yes, newer Submissions will also have submissionUUID in the Entity List.

[prabha]

One more question:

I select one from record from the entity list in a subsequent form. When I select that record in the subsequent from, I want the name of the farmer as well the submissionID (that is available in the entity list) in the subsequent form. How do I do that?

Thanks

[LN]

You can look up any entity property using an instance expression. The last part of the expression is the property name.

Some resources to look at:

• the XLSForm template
• documentation on forms that use Entities

[prabha]

Thanks.

I am using the instance function

instance('buzaaya-farmer-members')/root/item[name=${farmer}]/submissionUUID

But it does not seem to retrieve the submissionUUID available as a property in the entity.

[LN]

Are you sure that submissionUUID has the exact same case as the property name? Are you able to see in Central that the entity you select has a value set for submissionUUID? Are you getting a blank value or something else? Is farmer the name of the select question you're using to choose an entity?

[prabha]

Thanks for engaging...

1. Yes, submissionUUID is exact case
2. Yes, submissionUUID has a value in Central
3. Do not know if I am getting a blank value. Is there a log where I can see. There is no exception or error
4. The farmer is the name.

a) Is the : the issue?
b) Here is the form spreadsheet...
select_one_from_file buzaaya-farmer-members.csv farmer Farmer Member Pick the farmer from the list yes autocomplete collection_centre=${cc} value=farmer_name
c) And the calculate
instance('buzaaya-farmer-members')/root/item[name=${farmer}]/submissionUUID

See screenshot.