I had a great conversation with @Batkinson about this and a number of topics of common interest. After Wednesday's TSC call, the plan is to wait at least until Collect v.1.27 before releasing an API so that we can learn more about the needs of real users with integrations. @Batkinson found that reasonable. In the mean time, he and I will discuss any tentative plans we may have for the replacement API and try to make sure that the solution will be reusable for all.
Sorry for not replying, it seems my notifications are off & I'm late to the changes to getodk
Ona is about to start adapting some of our apps, WHO Steps app, to using scoped storage & ensuring that the integrations to ODK Collect are updated. I found some code that generates a CSV used for the pulldata function in the form. We were solving the CSV media file hackily by adding it to the ODK form media folder. It seems that this is the only feature that does not use the ContentProvider or the activity hooks.
Do you have any ideas off the top of your head? If not, I can get back with several options later & we can see what would be good.
I don't have this in my brain anymore so definitely take my thoughts here with a grain of salt.
I think the general approach would be for external apps to hand Collect URI(s) to form definition and/or media files with permissions through an intent extra. I vaguely remember giving some thought to where this should go and I think my conclusion was that it should be a new, dedicated activity that takes whatever URI(s), copies the files into the Collect directory, registers them in the database and generally gets them ready for use.
Even though you're only looking to register media files (in this case a CSV) with Collect, I think the solution design should also include registering a form definition. Those are tightly related and we'd want to make sure that they're handled coherently. Additionally, whatever we come up with, we should then later be able to add something symmetrical for handing submissions to Collect.
That means if you do still want to add support for external applications handing form definitions and media files to Collect, there should be a proposed approach by mid-August at the latest (in time for v1.29 planning). A PR in mid-September would be ideal so there's plenty of time to go through quality assurance. The last chance to get a PR in would probably be the last week of September.
@LN Thanks for the update. I'm working on getting resources allocated so that I can start on this. I think that the September deadline is considerate enough. The August timeline for proposing an approach also sounds good. Sorry for the silence on my end. I'll communicate in due time. Thanks again for the update
Hi @Agumash
I can assure you it's not related. Maybe there is something wrong with your server. Please create a separate topic if you can't solve your problem by yourself.
@LN I couldn't get the resources to work on it now. We'll reach out when the situation changes & I get a go-ahead
We also discovered that the files are accessible for some android versions inside the /data folder. However, they are private on Android 9. The support team & some users tend to copy over the failed submissions for sending with ODK Briefcast on a computer
Thanks for that. I can access the submissions now both through the file manager and ADB. It seems that I had forgotten to perform the migration on the Android 9 device while testing on the devices. Sorry for that . Thanks again
There are two changes coming up as we fully finalize this transition:
In v1.30, planned to be released around 2/22, new installations will no longer perform migrations from /sdcard/odk. If you upgrade from an older Collect installation still using /sdcard/odk, your data will be migrated. If you had previously used an older version of Collect, then uninstalled it, then install Collect v1.30, anything in the /scdard/odk folder will be ignored.
In v1.31, likely out in May, migrations will not be performed at all. If you have data in /sdcard/odk that you want to use, you'll need to either manually move the folders with adb or first install v1.29 or under and then upgrade to v1.31 after a successful migration.
This should have no impact on most users.
For organizations that choose to stay on a well-known Collect version for some time, this information could help inform the timeline for their next upgrade.
As described in the prior post, migrations from /sdcard/odk/ will no longer be performed starting with the next Collect version, now called v2021.2 (versioning change discussion).
Additionally, we plan to remove some update and delete functionality currently provided by FormsProvider and InstanceProvider. This is only relevant to applications that deeply integrate with Collect (e.g. OpenHDS, ODK Clinic). As far as we know, after the storage migration, there are no meaningful update or delete actions to take. If you access the ContentProviders for Collect v1.30.1 and use update or delete, please reply and describe what your app does.
We are making this change because as far as we know, there is nothing useful to do with update and delete without also being able to read and write corresponding form definition and form instance files. Additionally, as @seadowg started writing tests for the update and delete behavior, he identified several bugs and inexplicable behavior. We have to change the implementation of the ContentProviders for multiple project support and it would be best if we could avoid carrying forward undocumented behavior that we don't fully understand.
That cannot be done currently. You could propose a specification and code design for it based on some of the ideas I shared here and the ClipData solution we used for requesting binary files from external apps.