1. What is the issue? Please be detailed.
I have a large ODK form that is being filled out by people in multiple locations on their own laptops/desktops . We’ve sent the “online web only” version of the survey link.
I have had several users in several parts of the world report that they cannot submit their forms - when they click submit they just see the rectangle progress bar and the blue slider just keeps going back and forth within the rectangle but never finishes. There are no validation error messages or submission error messages, it’s just that it doesn’t get submitted.
Please Help, Where should I look to resolve this asap.
2. What steps can we take to reproduce this issue?
Anyone aware of this, what steps should I take to resolve this problem we have a very tight deadline to resolve this.
3. What have you tried to fix the issue?
Users have tried to update and change web browsers Explorer, Mozilla and Chrome
4. Upload any forms or screenshots you can share publicly below.
So, you use a private server installation (ODK Central).
Normally, data should not get lost, even with submission failed, but stay in the browser cache.
A robust workaround might be to save the draft before submitting it. See also https://blog.enketo.org/auto-save/.
Update ... web browsers may result in data/cache loss.
@ wroos kindly help elaborate on saving data in the browser cache and saving the draft before submiting it. Is there something I need to enable for this kind of saving to work so that when the hang happens, user can refresh the page without losing there captured data and keep attempting to submit with refreshes. How can this be achieved, normally once you refresh the entered data is lost. Please help. Kindly bare in mind we are strict using the laptops web online version and not mobile.
They might first "save as draft" and then (later) send. This is a standard feature, at least in KoboToolbox, and might be used until the problem is solved.
As LN asked in another posting:
Can you tell us more about the web form and how it has been distributed to the user? Is it a Public Access Link or is the user a Central Web User? Did you add /x/ to make it offlineable or is it a standard online-only form?
1. What is the issue? Please be detailed.
I have a very large form that uses lots of begin repeat counts that loops through multiple countries say more than 20 countries, inside there are multiple loops that collect different demographic and aggregate information and many other data. Also, I have the form divided into sections collecting studies, trainings, policy data and many more. The form has to be completed by over 100 users spread across the global. Users reported loops not opening up to enter data under those sections, they also reported being able to capture data for first 3 countries and the system starts hanging and failing to refresh loops. Also, users have reported data entered on one loop or country being moved to another loop or country in the selection.
2. What steps can we take to reproduce this issue?
To better understand the situation, I recreated their scenarios by selecting as many as 15 countries and recapturing the data. Capturing data on each and every loop, I have discovered that both the online web and online web offline (with an X) they slow down as you are capturing and looping through the many forms begin repeat count loops until eventually the system becomes so slow to refresh selection to a point of it being unusable.
3. What have you tried to fix the issue?
I tried to use the online web cache (x) offline. Can someone point to me whats happening? If this is refresh/slow speed of ODK, what the solution to overcome this behavior.
This is on odk on digital ocean.
Please help.
4. Upload any forms or screenshots you can share publicly below.
Thanks for providing extra information about your scenario. It sounds like you are using a web form and that your form design includes nested repeats, is that right? It sounds like you also have calculations across those repeats and you are expecting multiple instances of those repeats.
Some high-level ideas you could explore, most of which are in the documentation:
Make sure you are on the latest version of Central (use the question mark menu in the upper right corner of the screen). We've made many performance improvements to web forms recently. If you're running an older version, you may want to try your form at https://getodk.org/xlsform/ to see whether upgrading would help
Reduce the calculations you do in the form. Can you do some in analysis instead? Can you conditionally run certain calculations (e.g. use an if to only calculate a sum when all values are entered)
Unroll some of the repeats (make them into groups with fixed contents)
Split the form into multiple forms. With web forms, you could create links to the subforms and specify some default values for them. There is not much documentation around this because we'd like to improve the implementation but you can see how to do it today here.
If respondents have Android phones, try Collect and see if it performs better.
It would be helpful if you're able to share your form so we can make more specific recommendations and also use it as a target for performance improvements. If you can't share it publicly you could message it directly.
On the above attached on the cycled left when you enter a value to loop through and open sets of questions for each number of trainings, the right cycled errors also flag.: no context element found /data/brTraining/bgTrainingSection/bgTrainingSection1a/brDistrictTraining 1
Sometime when you enter a 1 for the 1 number of trainings the systems would hang and wont open, then enter a huge value like a 10 and switch to 1, form loads and at times produces the errors.
When you succeed on the above and try to submit the form, the below hang happens and errors on the browser inspect console runs as cycled on the right.
These and many more hangs happen. any advise will help to address these specifically.
Also on the test upgraded ODK platform, does it allow testing with preloaded attachments that support the form? I wont like to test and see how it will behave on the latest updated odk before running an upgrade??
Quick question, on my select choices I only have have 4 items users can select but then it shows 14 selected and yet I have only selected 2 from the 4. Please note this is common behavior for most of my dropdown selections?
Thanks for taking the time to try to narrow down on the problems and for sending me the form. I can reproduce the behavior you have described but am unable to isolate the problem case. It is reproducible on the latest Enketo version unfortunately.
I no longer think that it's performance related. It seems more likely that either a form design issue, an Enketo bug, or some combination of the two is leading to the crashes and then hang. Is it the case that some form submissions are going through ok? Is there only the one path that you pointed to which causes crashes?
If you can identify the paths that are leading to crashes, the next step for us to be able to look further into it would be to create a small form that isolates the problem.
If someone else has seen behavior like this before, it would be very helpful if they could share their form as well.
I did not see that when I tried the form out. My guess is that once a crash like you pointed to above occurs, calculations no longer happen correctly.
Thanks @LN for looking into this and for the update. I will be doing data analysis looking at the whole dataset of what's been submitted so far. This Friday we will close the data collection window and open data analysis, validation and cleaning on Monday, I will keep you and the team here posted on the findings around the current submissions. We will open the form for data editing first 2 weeks of January and I had hoped we would have found a solution and at this point I feel even the upgrade we have planned for will not resolve the issue.
One thing I have noticed using both the online and offline versions of this web form is, if you are capturing data say for 1 or 2 countries, you will have very little problems in form refreshing fields and the begin repeat loops opens up and works without much problem. Complaints and issues get worse as more countries or more loops are added/opened to capture various needed section of the data which are all critical. So the huge the dataset that needs to be entered on the ODK form, the more the issues increase.
I really do not know what can be done to the design form to overcome this if the design has issues, the current form implementation answers all the data collection and process scenarios needed, how to optimize this at this point really have no idea.
Does Enketo have limitation in terms of how much begin repeat loops it can accommodate? Are you able to privately share the form with someone from ENKETO to also take a look, if there are not here and get there views from Enketo end?
That suggests there is also a performance-related issue, then. The hanging at submit time is definitely a crash, though.
This is surprisingly hard to give a definite answer to. It depends heavily on the types of computations the form has, level of nesting, and device performance. We know there are bugs and performance issues with nested repeats in particular and are actively working to resolve them.
