I am watching a weird submission entry in my aggregate. It has no META INSTANCE ID, no START TIME (END TIME is there), no DEVICE ID and SUBSCRIBERID as well. Rest of the data seems OK (also includes 2 images, which are fine). The missing META INSTANCE ID is a seriously concerning thing. Please note that i did not move this form from INCOMPLETE SUBMISSIONS to main. It landed as COMPLETED form in the main aggregate window.
What can be the possible issue? Has this happened to anyone else? What can be the possible issue where META INSTANCE ID goes missing?
I have never seen this. Was it sent from Collect? Are you able to find out what version of Collect and what device? The more information you can provide the more likely it is someone will have an idea.
Just when I thought it might be a very rare issue and might not appear again, I came across this yet again!
For the first instance of the issue, here is the environment:
Aggregate version 2.0.5
Collect version 1.27.1
Device is Samsung Galaxy S5 (Android v6)
Aggregate server is significantly power for both CPU and RAM, and load is also negligible (project is not launched yet).
Problem: A form submission on the aggregate does not have a META INSTANCE ID assigned to it (It shows blank). I compared the other data fields, they seem to be OK, except that the faulty submission does not have a START timestamp on it (metadata). The faulty submission instance also has images with it which are correctly uploaded to the server (no issue in that too).
What other logs may I share with you for your help?
I checked the _CORE table for the form in aggregate database. The META INSTANCE ID field for that specific entry is NULL, however, the _URI field of the same submission does carry a value.
I downloaded the data through briefcase. The briefcase downloaded that submission with the URI identifier as folder name. Inside the folder, the submission.xml file still has the META INSTANCE ID field missing.
Unfortunately I have to delete the submission, since my custom dashboard indexes the data on META INSTANCE ID value (which I never thought could go missing until 2 days ago!), and it is crashing for not having found that value on that form. But I have the briefcase downloaded data for that instance, if you need it for investigation.
The only thing I can think of is that somehow the preloads aren't being run. The fact that the start timestamp is also missing strongly supports that hypothesis. Do you have any other metadata fields defined in this form (e.g. username)? If so, are those also blank?
Do you have any way of knowing whether it was that device both times? Has that same device with Collect v1.27.1 been able to send submissions with start timestamp and instance ID?
No, not the same device both times. The one I mentioned (S5) is one, and the recent one is some other device.
I have a lot of metadata fields in the form: start, end, deviceID, subscriberID, simserial. For this instance, START is missing, END is available, while all others are missing. The last 3 being missing did not raise an alarm for me because I see a lot of submission entries missing these values. However, START and END are always there, except with this abnormal one.
I traced one more submission with slightly abnormal data. It has the START metadata value but END value is missing. The form does have META INSTANCE ID, but 80% of the form data is missing. It is not possible to finish and submit the form with that missing questions, so I am not sure how or why this form submission is here, and also got marked as COMPLETED as well. Anyhow, I think it's a separate issue than the one at hand.
Since the device ID and other identification fields are missing, I have no way to know if the same device sent another successful submission or not.
Thanks for sending me the form definition and more information about the devices, @Saad. I've done some analysis and I doubt very much that it has anything to do with the form definition or with Aggregate.
That both the instance id and the start time which are set by the same process are both missing strongly points to something going wrong in Collect. The hypothesis I have right now is that it's related to a crash we're seeing in our analytics but that we can't reproduce. We added some code to initialize Mapbox maps for use with offline layers and that seems to be crashing on some devices. If that crash happened right as a form was loading, it could possibly lead to the behavior you're experiencing.
What I find unsatisfying about this hypothesis is that the crash affects a relatively small percentage of our users so it's surprising that you would have seen it multiple times. How many data collectors do you have total? Have you seen it again? This initialization only happens once per device so if this were the cause, you wouldn't see it more than once per data collector. Do you have reason to believe it has happened more than once for a data collector?
For those familiar with the Collect implementation, all preloading happens in FormLoaderTask.initializeForm. Changing all the calls to formDef.initialize to have a false value for their first parameter leads to instances with no preloads. I've thoroughly looked at FormDef.initialize and I don't think that there's any way for it to crash or somehow not preload the main instance if the first parameter is true. However, the Mapbox crash is deep in native code so it could interrupt that process. But then the only way I can see for Collect to go down the branch where FormDef.initialize has a false first parameter is if a savepoint file was saved. I can't figure out how a savepoint file could be saved before the FormLoaderTask completes. So it feels quite inconclusive.
Unfortunately I don't think I can do a whole lot more here. If anyone else has experienced this it would be really helpful to know it.
