Limiting Entity access for App Users and Data Collectors

[norlowski]

Currently, when a form has an Entity List attached, all Entities are always sent to all users that request the Entity List. For some projects, this is a significant barrier to adoption of Entities either because of privacy concerns or because their Entity Lists are so large that it’s not practical to always download them fully. We’ve seen projects do things like split their forms (register_north, register_south) and Entity Lists (participants_north, participants_south) to get around this limitation.

We have been exploring how to approach this need for some months and have identified a number of related issues:

• App Users are often defined as roles (e.g., vaccinator) rather than individuals because the absence of bulk tools in Central makes it hard to manage App Users as individuals.
• Confusion between roles: App Users and Web Users with the Data Collector role feel like they should be very similar but they’re currently represented very differently.
• It’s easy to get lost while managing users: You can configure users and access from multiple pages

This led us to explore ways to fundamentally rework user management in Central. We made some prototypes and got some great feedback from user testing – shoutout to everyone that gave feedback so far!

For now, we have decided to come back to the underlying problem of limiting Entity access for App Users and Data Collectors and have some ideas for making improvements in that area in a phased way without taking on all user management issues at once.

We’re exploring the concept in a phased approach:

Phase 1: Single ownership

In this phase, there are two options for Entity access:

1. Data Collectors and App Users will either be able to access all entities in a project
2. Or only the entities they have created. This should benefit projects with many entities where users only need access to their own.

In this phase, we would enable changing Entity ownership through the API only. Changing Entity ownership is important to support cases where Entities were created through CSV upload and need to be given App User/Data Collector owners or where Entities created by one individual need to be reassigned to another individual.

Phase 2: Single ownership with the ability to change owners

If single ownership is enabled, Project Managers will be able to change entity owners. This should benefit projects with many entities where users only need access to a subset.

Phase 3: Group ownership

Project Managers can create groups and assign users to them, where users can be part of multiple groups. This should benefit projects where users need access to entities associated with roles or locations.

We’d love to hear from you on these approaches:

• How might phase 1 and 2 work within your projects?
• What potential challenges or concerns do you have?
• What questions are top of mind for limiting Entity access?

[ahblake]

Really excited to see this as it's a size (especially when entities get media...!) performance / privacy issue for me and many others I know.

• How might phase 1 and 2 work within your projects?
  • P1 - I would have multiple app users that need to access the same subset of the entity list(s) and they would not create them themselves in most cases. If I could accept a single 'app user' for a region and rely on deviceID &/or entered user info then this would allow using the API to change ownership of each subset of the list to limit access.
    • You described it as access 'all entities OR the entities they have created (i.e. ownership)'. Is there a possibility for 'generic/project wide entities AND entities they have created/own'?
  • P2 - This looks like P1 but with a Central interface for changing ownership, which is ok for small entity lists but impractical for large ones unless there is a filter and select all UI.
  • P3 - this looks like it solves most of my issues!
• What potential challenges or concerns do you have?
  • As above, OR vs AND access and ability to change many through Central
  • If an owner is an app user and access is only to that app user, can the submission be edited in Enketo as that user is not the app user and so wouldn't have access to the entity items?
• What questions are top of mind for limiting Entity access?
  • would CSV upload allow assigning ownership at time of creation?
  • would ownership change with an update to an entity by app user B vs creation of an entity by app user A? (assuming set as all users access all, but ownership is known)
  • only tangentially related, but will CSV upload allow updating existing entities vs only adding new at some point?

[norlowski]

Hi @ahblake, thanks for the questions and feedback!

If an owner is an app user and access is only to that app user, can the submission be edited in Enketo as that user is not the app user and so wouldn't have access to the entity items?

Currently only Project Managers or System Administrators can edit submissions. Entity filtering doesn't apply to them (yet?) so they would be able to see and edit all Entity data when opening an existing submission.

would CSV upload allow assigning ownership at time of creation?

We do want to make it so that a CSV upload would allow assigning ownership at the time of creation. That could be part of P2 with a database id or we may first migrate users to having human-friendly usernames.

would ownership change with an update to an entity by app user B vs creation of an entity by app user A? (assuming set as all users access all, but ownership is known)

We currently don't think that updates will change ownership. We may at some point introduce a new form design action for changing ownership. That would be similar conceptually to the create or update actions.

only tangentially related, but will CSV upload allow updating existing entities vs only adding new at some point?

That's the hope! We haven't taken it on yet because it's complex but you can use the pyodk merge function to achieve that goal if you're comfortable with some scripting.

[chun_hing_yap]

With this 3 phases concept and perhaps implementation later, is there still a limit on how big the entity list can be? What is the expected size of entity list that could work under this setup before hitting a real performance/practical issue?

For trees use case, perhaps half a million is a number that I am looking at. With current entity feature, it will be just too overwhelming. If the entity is with location to show in a map, the limitation will be even more obvious. At the moment, I will use attached csv or json when the dataset is at this scale. When entity features are really essential, the only option is to create numerous entities, which will make data management more cumbersome.

It is a bit difficult to strike a balance between performance and entity management now. The ideal situation is to have both optimized, manage just only one entity with good performance (subset entity by access group).

Phase 3 does seem to address my need if the performance can be improved as well.

[Syed_Muhammad_Qadeer]

This is very useful. I used to call this requirement as Differential Access of entities to users.

In my workstream, such feature is highly needed.

Glad to see the progress.