Populating ODK form from earlier data, filled form XML files or CSV or any other alternatives

Hi ODK Devs,

I am trying to find an clean solution to populate a form based on an ID
(e.g. for a person already registered, it should populate whole form), one
of the possible solutions seems to be (correct me if i am wrong) external
itemsets example: http://opendatakit.org/help/form-design/external-itemsets/ but
i think i would have to put a lot of manual effort to prepare CSV in
correct format like in external itemsets example to query it (populating
from it would be yet another task).

The 2nd and more clean approach could be (i guess) to use ODK Collect's
filled form loading functionality, lets say we save all filled forms .XML
files and put it on device (each form saved using instanceName i.e. Unique
ID). We search all forms based on ID (i.e. form file name) if it already
exists, it opens that particular form for further editing. I am more
inclined towards this approach.

I understand it may involve customizing ODK collect in some way, any
pointers towards that or other alternatives to achieve similar
functionality would be really appreciated.

Best Regards,
Ayub

Ayub,

The simplest option is to use SurveyCTO (http://surveycto.com). It's
based on ODK and has very easy to use pre-filling.

Your alternative of saving all filed forms would work, but it'd
require you to insert pointers to those instances into the instances
database. This is required because Collect doesn't automatically
process instances it didn't create that show up on the SD card.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Thu, Dec 19, 2013 at 9:50 AM, Ayub nrspaggregate@gmail.com wrote:

Hi ODK Devs,

I am trying to find an clean solution to populate a form based on an ID
(e.g. for a person already registered, it should populate whole form), one
of the possible solutions seems to be (correct me if i am wrong) external
itemsets example: http://opendatakit.org/help/form-design/external-itemsets/
but i think i would have to put a lot of manual effort to prepare CSV in
correct format like in external itemsets example to query it (populating
from it would be yet another task).

The 2nd and more clean approach could be (i guess) to use ODK Collect's
filled form loading functionality, lets say we save all filled forms .XML
files and put it on device (each form saved using instanceName i.e. Unique
ID). We search all forms based on ID (i.e. form file name) if it already
exists, it opens that particular form for further editing. I am more
inclined towards this approach.

I understand it may involve customizing ODK collect in some way, any
pointers towards that or other alternatives to achieve similar functionality
would be really appreciated.

Best Regards,
Ayub

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Ayub,

Can you give me any info on the "ODK Collect's filled form loading functionality" you mentioned in this post? I would like to upload partially loaded form instances so the field can validate and retrieve additional data......is this possible on ODK COLLECT 1.x? you have sparked my interest......

John Harper

··· On Thursday, December 19, 2013 7:50:36 AM UTC-7, Ayub wrote: > Hi ODK Devs, > > > I am trying to find an clean solution to populate a form based on an ID (e.g. for a person already registered, it should populate whole form), one of the possible solutions seems to be (correct me if i am wrong) external itemsets example: http://opendatakit.org/help/form-design/external-itemsets/ but i think i would have to put a lot of manual effort to prepare CSV in correct format like in external itemsets example to query it (populating from it would be yet another task). > > > The 2nd and more clean approach could be (i guess) to use ODK Collect's filled form loading functionality, lets say we save all filled forms .XML files and put it on device (each form saved using instanceName i.e. Unique ID). We search all forms based on ID (i.e. form file name) if it already exists, it opens that particular form for further editing. I am more inclined towards this approach. > > > I understand it may involve customizing ODK collect in some way, any pointers towards that or other alternatives to achieve similar functionality would be really appreciated. > > > Best Regards, > Ayub

Hi Yaw,

Found the instances.db database in SDCARD/odk/metadata. It shows unique
displayName of my form (... perfect). Now if we:

  1. Copy all form folders under odk/instances from all devices.
  2. Insert/Merge rows in instances.db from all devices.

At this point ODK should be able to see all the forms (manually editable
under 'Edit Saved Form'). Now what changes in source files (and where if
possible in ODK collect as it has huge source code) would i need to make in
ODK collect to:

  • Query instances.db displayName column against an input question (a string
    ID). If that matches, close the new form and load existing form SD card for
    editing.

I have basic familiarity with ODK collect's source code, mainly from
collect WIKI.

Unfortunately we cannot use SurveyCTO at the moment due to its cost.

Best Regards,
Ayub

John,

You can certainly programmatically (or manually) generate
partially-filled forms and insert them into the /instances folder (and
the instances database).

If you are looking for a turn-key solution, try SurveyCTO. It allows
you to pre-load existing data from a baseline, listing, or other data
source and use that data in your surveys.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Mon, Jan 27, 2014 at 11:03 AM, jharper1986@gmail.com wrote:

Ayub,

Can you give me any info on the "ODK Collect's filled form loading functionality" you mentioned in this post? I would like to upload partially loaded form instances so the field can validate and retrieve additional data......is this possible on ODK COLLECT 1.x? you have sparked my interest......

John Harper

On Thursday, December 19, 2013 7:50:36 AM UTC-7, Ayub wrote:

Hi ODK Devs,

I am trying to find an clean solution to populate a form based on an ID (e.g. for a person already registered, it should populate whole form), one of the possible solutions seems to be (correct me if i am wrong) external itemsets example: http://opendatakit.org/help/form-design/external-itemsets/ but i think i would have to put a lot of manual effort to prepare CSV in correct format like in external itemsets example to query it (populating from it would be yet another task).

The 2nd and more clean approach could be (i guess) to use ODK Collect's filled form loading functionality, lets say we save all filled forms .XML files and put it on device (each form saved using instanceName i.e. Unique ID). We search all forms based on ID (i.e. form file name) if it already exists, it opens that particular form for further editing. I am more inclined towards this approach.

I understand it may involve customizing ODK collect in some way, any pointers towards that or other alternatives to achieve similar functionality would be really appreciated.

Best Regards,
Ayub

--
You received this message because you are subscribed to the Google Groups "ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi John,

As Yaw told me earlier and right now, you would need to put all forms
filled XML files into /instances folder, as well as update instances.db
sqlite database file (open it with some sqlite viewer to understand how it
stores information about forms, change/update "Is Complete" column property
of form to false and you'll see it as editable inside ODK collect without
any coding, or you can make your own custom activity in android that
completely ignores this flag and starts an activity with
Intent.ACTION_EDIT, this intent is handled by FormEntryActivity if i
remember it correctly). This approach would require to maintain a master
instances.db database for all forms each and update each device with filled
forms xml files + master database.

This is certainly one option to pre-load forms data based on some Unique
ID. If you need any further details let me know.

Ayub

Ayub,

Take a look at the ODK Tables source to see how it loads pre-filled
ODK Collect instances.

I don't know how much your time is worth, but $100/month for SurveyCTO
is going to be cheaper than coding this up.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Fri, Dec 20, 2013 at 8:54 AM, Ayub nrspaggregate@gmail.com wrote:

Hi Yaw,

Found the instances.db database in SDCARD/odk/metadata. It shows unique
displayName of my form (... perfect). Now if we:

  1. Copy all form folders under odk/instances from all devices.
  2. Insert/Merge rows in instances.db from all devices.

At this point ODK should be able to see all the forms (manually editable
under 'Edit Saved Form'). Now what changes in source files (and where if
possible in ODK collect as it has huge source code) would i need to make in
ODK collect to:

  • Query instances.db displayName column against an input question (a string
    ID). If that matches, close the new form and load existing form SD card for
    editing.

I have basic familiarity with ODK collect's source code, mainly from collect
WIKI.

Unfortunately we cannot use SurveyCTO at the moment due to its cost.

Best Regards,
Ayub

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Yaw,

I am able to query forms based on their displayName id (which is unique in
my case), it loads up form for editing. (so far everything seems to be
working correctly).

Since ODK name folders for a form instances such as {Form Name} + {Date
Time} i think given number of devices used for form filling, two or more
forms could have same folder (instanceFilePath) name (and could cause
trouble when many forms are placed in same folder). Just to keep folder
naming unique, it should be {Form Name}+ { Some UUID } + {Date Time}. So
far i couldn't found the right place in source files where ODK Collect does
instance folder naming while performing Save. It would be really helpful if
you could guide me that.

Best Regards,
Ayub

After a long search found following code in FormEntryActivity.java

String path = Collect.INSTANCES_PATH + File.separator + file + "_"

  • time + "SOME UUID";

if (FileUtils.createFolder(path)) {
formController.setInstancePath(new File(path + File.separator+ file + "_" +
time +"SOME UUID"+".xml"));
}

I guess that would solve uniqueness problems with folder and file naming.
Will keep posted on the rest of the issues i would have to deal with in
process :).

Ayub