The changes to the Collect end of form screen are not compatible with my workflow

Support
2

Very sorry to hear that these changes have disrupted your work. Hopefully you were following the beta and were able to anticipate them and react appropriately.

Is the reason that you want to keep forms non-finalized that you have a review workflow for your forms? Do they involve supervisor review or just having the data collectors review their own work in a different environment? If you provide details about your workflow, we can share some ideas on how to adapt at some point in the future. We are also going to eventually publish a guide specifically on defining review workflows. The more detail you provide on how you do things today, the better ideas we can share.

1 Like
3

I share the opinion of krosser.
This change to the collect end of form screen are causing additional workflow steps for us. I regret these changes are made.
I wrot already more about my opinion her:

In fact - the fact that the 2023.1.2 version was causing problems for users lies not in how the functionalities work, but in how they were explained. I always explained the difference between checking or unchecking the "finalized" checkbox as such:

Drafts (=not finalized) could not be sent, finalized forms could – and both can still be edited. It prevented that forms that were really in concept phase, could be accidentally sent to the cloud.

For us that is important, as our forms are too heavy and filled with a lot of calculations, so they cannot be edited online in kobotoolbox (as Enketo is not able to handle our forms).

4

It sounds like one of the underlying issues here is that your form can't be edited in Enketo. Please share the form definition so that we can investigate this. Ideally you can post it publicly here but if it's confidential you can also send me a direct message and I'll make sure it stays within the development team.

5

Thank you for your kind response. Unfortunately, I did not follow the beta and only found out about the issue upon updating the app on my test device. It is difficult to keep up with all developmental processes. Perhaps, a newsletter with a poll for such major changes would be helpful.

Overall, I see the beauty of the new concept for the main menu and end screen.

To answer your questions, yes, this new change would require us to update the presentation materials for the training, which includes screenshots of the app, description and different scenarios. But this can be done and not the main issue for us. The human error is the main concern here - when you teach people in the course of several days that they should carefully observe the last screen, read all notes and not tap on the "Send" button when the survey is not final, and still there are people who tap the button. The checkbox served as an additional step/check before the final submission, which is now absent.

Perhaps, it would be helpful to include later on in the settings to toggle a warning pop-up window when the person taps "Send"...

Moreover, on an unrelated note, the previous version of the app had translations for most elements and texts in various languages, but the new version is missing the translations for those new changes (main menu, exit window, and end screen). This also prevents us from using the app in some non-English speaking countries in the WHO European Region. For example, there is a study in Turkiye that is currently being implemented and we have to use the previous version because of that.

I hope this helps clarify.

Best regards,
Serge

2 Likes
6

I was about to start a thread about the end of the form, slightly different use case, but it is related, so this is probably the best place to post...

I don't have a problem with the language used, and in some ways it is more clear to use save as draft as a button with the other option being to 'finalise' a form (if the project is configured to manually send forms) or 'send' a form if the default QR Code project setting are used. But in my case it can be problematic to save as draft when collecting a large number of data points in a fieldwork session... This requires 'double handling' the data to go through each draft form and finalise. Collecting 100+ points in a day adds to an enumerator's 'admin'.

My particular use case is when recording the status of a point in areas of peatland based on nearby features (we use a 30m buffer to determine the status) - sometimes due to topography it is not possible to see a relevant feature, so previously we could edit the form (either by using the existing records map or finalised forms list). Now that is going to be compromised and we need to save as draft just in case... Although there are not too many times that we actually need to change the status, I'd rather the option to be able to edit finalised forms for when it is necessary.

I understand that ODK may have found that in the majority of cases it is important not to allow enumerators to edit finalised data (although if an enumerator just saves as draft they can edit their forms prior to submission, so maybe that is a false friend).

On other forms with complex data models I have advised people to save as draft or to make sure they have data/ wifi switched off to ensure that the form is not sent before they have a chance to check / amend if necessary.

So I did some thinking to try to solve my issues...

If it is not possible to review the decision to make finalised forms editable after the event by default, would it be possible to add a parameter in the 'form settings' (e.g. in Excel) to 'Allow edit finalised / unsent form' - so that it doesn't disrupt existing workflows, and administrators could just deploy a new version of the form with that setting.

Another option might be 'bulk finalise' in the Drafts list - so that you can tap a form to edit it, but there is a check box (and select all option) to allow multiple forms to be finalised / sent depending on the Project settings. That could significantly reduce the 'burden' of going through each form and changing the status - lots of tapping needed for 100 forms!

I recognise that both of these ideas have big implications for the core team and development, but as highlighted by @krosser and @BartRoelandt2 not everyone was ready for the change - and although I was aware of the change, it only became 'real' yesterday when I was out collecting data and wanted to change an existing record (I managed, heuristically, to do it using the map but Collect did slap my wrist for doing so :slight_smile:).

Thanks. I do like the new UI, honestly.

2 Likes
7

Thank you all for the thoughtful conversation around the updated end of screen flow. We are evaluating different options for understanding what kinds of workflows are impacted, the breadth of the negative impact, and possible adjustments to the new behavior. The more information that those who are negatively impacted can provide, the better solution we are likely to come to.

It does seem we have a communication challenge. Do you follow ODK on Twitter/X, Facebook or LinkedIn? We did not get very good response when we made a request for user interviews on those networks and here. We did get better response when we sent out a survey.

Did you know you can subscribe to specific forum tags and categories by using the bell icon at the top of the screen? That will send you emails if you don't visit the forum often. I'd recommend watching at least the breaking-changes and betas categories. We can and should recommend this in more visible places. If you have recommendations on how to reach you and your colleagues, please share.

We have previously experimented with newsletters but never got as many people subscribing and reading as we do here on the forum. Maybe a community member would want to experiment with putting one together again?

Can you please tell us more about your workflow? In what cases are forms complete but not final? Is there a formal review process? Is it more that there are occasional reasons to make light edits like what @seewhy describes? The more detail you can provide here about what needs to be done and why, the better.

This is something we did consider quite seriously. We decided against it because for many workflows this would provide friction in the most desired path. We also considered making it a setting but more settings add complexity and more chances for bugs. As we learn more about workflows that are impacted, we will reconsider these options and others.

This is very related because it ties into community communication and how we release bigger changes. We spend quite a bit of time planning which text in the app we update when to try and get as many translations as possible. With this release, we started early with many different ways to reach out to community translators. We asked for translations on this forum, through all social media channels, and through the Transifex translation platform. We periodically look at newly-available translations and update Collect accordingly. Is there someone in your network that you could ask about contributing translations for the text you need? Our latest call to action is here.

It can be very hard to think through the implications of a feature or concept when considering it in the abstract! That's why we initially try to gather as much information about what users' end goals are and why those goals exist and we are certainly open to course-correcting as we get new information. Thanks for describing the "just in case an edit is needed" workflow. We heard from a number of projects that they want to prevent the temptation of adding a detail later that may be a fabrication or misremembered. But I can see how in your case it might be real new information and that you are not concerned at all about fabrication. Please note that for now, you can continue editing finalized forms from "Ready to send" by tapping on their name.

For all of you wanting to continue editing finalized forms: is there any difference between finalized and sent forms in your workflows? In other words, would it be acceptable to allow edits for both forms that are finalized and ready to send on the device and forms that have already been sent? If we were to introduce something like that, we would likely make it so that Central would show the difference between source and edit for both of those cases. This would be functionality that form designers would need to opt into.

2 Likes
8

Thank you for pointing to social media sources. I have subscribed on LinkedIn and will try to keep up with the news.

Regarding the translation, I have checked and looks like the current version of the app has been translated into several languages almost completely. The missing languages are indeed country-specific. But I guess this work is community-based and we cannot expect a vast coverage. I will try to contribute as much as I can and will ask my colleagues who speak the missing languages if they can help out. But as I understood, all translations go through beta process and it can take some time to have them in the final app. How long does it approximately take to role out a translated app version after the language was added in Transifex?

Our workflow for a cross-sectional population-based survey (STEPS):
data collector visits a household (HH) (can visit 3 times if people are away and needs to save and reopen the draft form marking the visits) > opens a blank form starting with location and IDs > enters HH members of a specific age range > does a random selection of the participant from HH members > proceeds with the interview (STEP1) > proceeds with the physical measurements (STEP2) > records GPS coordinates > finalizes and sends the survey (the latter is done automatically by ODK Collect)

When the data collector cannot get a signal from satellites to record GPS coordinates inside the HH, he/she saves the form, gets outside and tries to get a better signal there.
Thus, there are two main cases when the data collector needs to save the form as a draft and reopen: 1) during multiple visits to the HH, 2) when GPS coordinates cannot be immediately recorded. The third case (rarely happens) is when the survey is paused for whatever reason, saved as a draft and continued at a later time.

From our experience, the issue of tapping the wrong button to accidentally submit the survey was mostly done in the second case - GPS coordinates. The data collector could accidentally finalize the survey and send it before recording the coordinates. The GPS question is not mandatory, since it was often very difficult to get a signal indoors or in a remote area. Therefore, data collectors had an option to either try several times outside the HH or submit some a their completed surveys without GPS coordinates when even outside the GPS did not work.

However, this year, we have introduced a change to the forms, upon switching to a new version of ODK Collect, which allows to "warm up" the GPS when opening a blank form (using start-geopoint). We also adjusted the accuracy of the geopoint. So, these changes have drastically improved the recording of GPS coordinates inside the HH, which could potentially allow us to make the question mandatory. Although it needs to be tested on several surveys to see the rate of recorded coordinates.

Please note that only specific app menus / elements are visible to the data collectors (the rest is password protected by the admin). Thus, they cannot edit finalized and sent forms.

I would like to add here that we would really like to see the updates to ODK Central to allow us remove test submissions, which are done during the data collectors training. Currently, we have to mark them all as "rejected" and filter out during data cleaning, which complicates the process.
The second feature that is really needed is the possibility to remove any old or test projects with forms as well as individual forms. The latter are currently can be put in trash, but it is not automatically emptied. So, emptying it manually should be allowed. Of course, all removal should be done only by admins.
Just FYI, we self-host the UAT and PROD instances of ODK Central on our servers.

9

For us there is certainly a difference between finalised and sent forms. Finalized forms are still on our mobile device and sent forms are (in our case) on the cloud-platform Kobotoolbox. As our forms cannot be edited on Kobo (a lot goes wrong), our enumerators were told that they needed to be very sure a "finalized" form did not contain any mistakes, as they cannot be corrected once sent.
in the former version of ODK the different statutuses meant:

  • finalized-box not checked: cannot be accidentaly sent to cloud
  • finalized: was in fact a draft because could still be edited on mobile device
  • sent: was in reality finalized because unpossible to edit any more on mobile device (but unfortunaltely either on Kobo)
10

I am very much in favour of seewhy's proposal of a 'bulk finalise' in the Draft list. I was explaining what happened with the changes to the new ODK-version to a colleague and he came immediately with the same idea!

Of course the other option adding a parameter in the form settings to 'allow edit finalised' is also usefull, but could probably lead to misuse by enumerators which are not allowed to make changes to finalized forms.

11

Hi @LN
Thanks as always for your considered view of your end users changing needs and our feedback - just when you think you've solved a problem...

I recognise the difficulties of pleasing all the people all the time and finding a 'right' answer.

In my case, of collecting and managing data from a relatively small number of enumerators, I have always worked on the basis of "once it's sent, it is no longer possible to edit", but until that point, you can do what you need to. In general I have not differentiated between finalised and draft except as an advisory where the form needs to be updated as part of the data collection process (i.e. all answers cannot be given in a single session such as location at the end, or the distance travelled etc) - usually it is easier to find the draft submission than a finalised form. To date I have been able to leave that to the enumerator to decide, with the caveat that they need to turn off data to avoid early submission.

Personally, I cannot see a helpful situation where an enumerator can edit SENT data as this has plenty of opportunity for conflict - I generally analyse data after downloading it from Central (incrementally), so that would introduce an additional risk if there is a possibility that the existing records could change and no control over when that might happen. And then there is the issue of 'delete after send' (opt in). That situation would mean that enumerators would not have the opportunity to edit sent data because it would no longer be on the device...

For clarity I have no 'authenticity of data' issues to deal with and for some surveys it is actually beneficial to review the data before sending (when it has been snowing and the notes are very cryptic, for example) or as I highlighted when additional information comes to light. I understand that others may have reasons to prevent editing of 'finalised' forms - but presumably they would also need to have a way of understanding what, where and when questions have been edited after the original survey session.

The 'introduced complexity' of managing offline data and entities adds whole dimensions!

1 Like
12

Thank you for looking at our form. I could not send a form without an accompanying explanation. Since the explanation is quite extensive, I have made a showcase of it. you can find the showcase and one of our 78 forms here:

I am interested to know if you can make Enketo open a finalized form correctly.
When I tried it (on Kobotoolobox), a lot went wrong, starting with not handling the cascade choicelists...

13

Thank you so much! :heart_eyes:

It really depends. Since 2023.2 came out, we have been actively watching translations come in and have scheduled releases just to get more translations in because we knew the new and changed strings were disruptive. Typically we only update translations when we already have a release planned or when a community member specifically requests a release. There are times where this would be hard to accommodate but so far we have always been able to honor those requests.

This is really great to hear, thank you for that feedback. Another option that you could consider is to leave the location question as non-required. If it is left blank, you could ask a question like "The location was left blank. If you need to send anyway, explain why." This text field could be required. Then the user could fill in an explanation if something unusual happened like their phone wasn't getting a location at all. If they don't supply an explanation, for example because they just need to step outside and try again, the form would not be finalize-able so they would have to save as draft. There are lots of variations on this approach including using one or many select questions instead of an open text box.

I see, that makes a lot of sense and explains why staying on 2023.1 is a better option for you at this time. We will consider that as we think of longer-term options.

Thank you, these are on our radar. We don't have a timeline for them yet but we know that they're important.

Yes, this has potential! There are interesting questions around what the experience should be when finalization fails and around how bulk select should be accessed.

Thank you so much, this provides so much valuable detail!

I see that you make use of the search() appearance function. This is indeed not supported by Enketo and also not included in ODK docs (though it's in xlsform.org and still maintained for now). The alternative functionality that works in both clients is select_one_from_file.

The only downside is that the performance for select_one_from_file hasn't fully been optimized yet and it can become slow for very large attached datasets or complex queries. I see you have 10k items in one CSV and 35k in the other. I haven't tried your specific form yet but we generally test with 60k+ items with no performance impact on modern devices. That could be an option to consider if you do want to try server-based edits. We also maintain Enketo and have made several performance improvements recently which you can try out at https://getodk.org/xlsform/ (Kobo and others that use ODK clients often run older versions).

It sounds like your scenario is quite similar to @seewhy's. Your data collectors are motivated and trusted, the data they are collecting is largely objective, and the thing that you're collecting data about is largely unchanging in a short time frame (as opposed to e.g. someone's health). It is possible to make honest mistakes and then it is desired to revisit the environment and fix those errors.

14

I would like to try this out.
Can you point me to a help page were this is explained?
I can not find it on xlsform.org

15

The docs I linked to above are https://docs.getodk.org/form-datasets/#building-selects-from-csv-files. The XLSForm docs are https://xlsform.org/en/#multiple-choice-from-file

I realized that unfortunately you won't be able to use https://getodk.org/xlsform/ to verify the very latest speed improvements because it doesn't support attachments. But it will be worth trying with select_one_from_file on older versions too.

16

Thanks again to everyone who has taken the time to describe their workflows and needs.

We have done some iteration and are currently planning to add bulk finalization of drafts as suggested above and a check for errors at the end of the form filling experience.

Here is what we have in mind for bulk finalization:

Screen Shot 2023-08-28 at 12.42.23 PM
Screen Shot 2023-08-28 at 12.42.23 PM1386×572 151 KB

We've made three significant decisions here:

  • It's an all or nothing action: you can't apply it to a subset of drafts. This is because we don't want to add complexity or visual clutter for most users. If there are forms that aren't complete and definitely need to be edited, the form design should include a way to indicate that so that finalization can be blocked. Any forms that can't be finalized will stay in drafts and will be indicated as having errors.
  • The functionality is in the overflow menu rather than directly on the app bar. This is because we expect it to be a relatively infrequent action, even for a project in which it's a key part of the workflow.
  • Bulk finalization will show a progress dialog and block the user from using Collect rather than working in the background. We don't think there's anything useful to do in Collect in parallel and doing this in the background would increase complexity and risk.

Another thing we will do is introduce an automatic check for errors at the end of the form filling experience. Currently, this only happens when a user attempts to Finalize/Send. This can be problematic if a user saves every form as draft and later bulk finalizes. If they accidentally skipped some questions and save as draft, they wouldn't find out about the missing answers until bulk finalizing when they might not have access to whatever they were collecting data about. So we think that to make this flow complete we also need to provide earlier feedback about form errors.

In the new Collect version with the features above, we would recommend that people who used to always finalize and then make edits as needed migrate to always saving as draft and then bulk finalizing as needed. They can hide the "Finalize/Send" button at the end of form filling to guarantee going through bulk finalization.

Please let us know if you have any feedback.

2 Likes
17

Hi LN,
Why did you choose for a bulk finalize which is a all-or-nothing choise?

Is it not possible to give the user select/deselect boxes for every draft form and buttons to select/deselect all? and a button "finalize all selected drafts"?

This exists now in the ODK-collect 2023-1 version under the "send finalized forms":
There one can choose which form to send/ or select-deselect all and then bulk-send all selected forms.

Why do I suggest this: In our process it happens that we would have a few forms that we want to keep in draft stage because we need to go back to the vegetation plot and determine species. In such case we would not be able to send any form because we only can go through the bulk-finalize and send process if ALL forms are really finished.
So we would have the risk of keeping a lot of data on our mobile device, just because of some remaining draft-forms.

18

Please see my first point above:

If the species is required and not filled in, those submissions would not be marked as finalized. Only forms that are complete and have no validation errors are eligible for finalization. What we recommend in a case like that if there isn't already a question that is required and left blank would be to add an extra question at the end that would be something like "Have you fully verified every answer?" and if left blank or marked as no, finalization would be blocked.

To be explicit, if you need to selectively make sure that certain filled forms aren't finalized, you could open each of those, edit them such that they can't be finalized (e.g. answering 'no' to "Have you fully verified every answer?"), and then bulk finalize. It is higher-effort than selecting all and unselecting the forms you want to skip but we think it's the right tradeoff given that this kind of workflow is less common than others.

1 Like
19

Dear @LN,
here another workflow which unfortunately will no more work now:
We used the finalised status to mark the cases for local review with/from a supervisor before sending them. The supervisor will fill few (mandatory) fields QA status, her/his id & secret password, optional remark - before the form can be sent by an enumerator (or supervisor).

So, our requirement is to have a status (from the enumerator) which marks a "finalised" case for review by/with a supervisor before a form is / can be sent. (A bit similar to the validation status in Enketo.) If the QA finds problems, the enumerator could even go back to the household to fix the data. Finalised status can be removed (and reset).

Our experience and recommendation is to allow quality assurance directly in the field, still during data collection, esp. in remote areas and with teams moving around (and with limited workflow support for editing/sending back individual submissions.)

Another use case for the (old) finalisation was that an enumerator or supervisor could at any time press "finalisation" to run all checks (using move to the end). This was even helpful to position to a last edit point (e.g. mandatory question) when an interview was interrupted and reopened.

As far as we understood your "non-edit" change was mainly caused by the new "entity" concept. I am not sure if this was the best idea. As several postings indicate, "finalisation" seems another, separate concept and an established workflow element.

Additionally, we want to keep the Collect settings option (also for Central), please: set auto-send off/on, set automatic finalisation off/on.

20

Thanks for adding to the conversation, @wroos. Workflows with in-field review were a significant motivator for reworking the way finalization worked. Previously, because finalization was not actually final, if a supervisor reviewed and signed off on a submission, unless they could send right away, they had no way of ensuring that a data collector didn't go back and make further edits. This is one of the many problems our user research identified with allowing submissions to go back and forth between draft and finalized states.

Here is how we expect workflows with a supervisory step to work:

  • The form has questions intended for a supervisor (as you described)
  • When a data collector does their work, they can't finalize because they end up with missing required questions for the supervisory process
  • If there's a need to quickly differentiate between drafts that need some data collector action vs. drafts that just need sign off (e.g. the workflow takes place over multiple days), then instance name can be used for that purpose. For example, a prefix such as [DAY 2 NEEDED] or Supervisor-ready or even :white_check_mark: can be added based on form answers.
  • If a supervisor finds the need for more work to be done, they don't sign off, so the form still can't be finalized. Again, the instance name can be used to indicate that a revisit is needed.
  • Once a supervisor signs off, the intent is for that submission to be sent without any further edits made. At that point they finalize and either autosend runs or the data collector sends when they are ready.

It sounds like you have a different workflow in mind. Can you please give us all the steps and exactly where having a truly finalized state is problematic? If you can share an example form that encodes your workflow, that would help make things more concrete.

You mention wanting to know whether draft forms have errors and wanting to jump to the first missing question which makes a lot of sense. First, note that if you have required supervisory questions that the data collector knows are blank, they can continue to use the "finalize" button for this purpose. We've also added in-form validation from ⋮ > Check for errors. This will do what you describe: it will jump to the first error found.

In the next release, we will add feedback about whether a form has errors even when saving as draft and we will also indicate whether or not a draft has any errors in the drafts list (see the "Saved with errors" prefix on the rightmost mockup in my post above).

2 Likes
21

Thanks for your quick and detailed reply.

Sorry, I don't like the idea of "overloading" and dynamically changing now the instanceName to mix-in a status/workflow concept.

We would prefer a status attribute/variable (maybe a metadata via setting sheet) which allows to clearly separate forms, even in different UI "folders"/menu items", still in progress at enumerator level (e.g. take a lunch break or meet household again next days) and forms declared as "finalised" by the enumerator, i.e. ready for local QA by/with supervisor.

We would also like that new concepts are similar in Collect and Enketo, very often we use both in the same survey. E.g. Enketo/KoboToolbox has a "validation_status" and a "Validate" option in preview.