Odkmeta import do file invalid syntax importing attributes as characteristics

Kate_Schneider · August 8, 2017, 2:02am

I used Kobotoolbox and am now trying to get my data into STATA 14.2 using odkmeta. I’ve gotten pretty far, but now I’m getting an error when I run the import do file odkmeta created when it gets to my first variable with a text response (the first “please specify other”).

Here is the section of the do file that precedes the error:

Import ODK attributes as characteristics.

insheet using "mydata.csv", comma nonames clear
local fields
foreach var of varlist _all {
local field = trim(var'[1]) assert :list sizeof field' == 1
assert !`:list field in fields'
local fields : list fields | field
}

insheet using "mydata.csv", comma names case clear
unab all : _all

This runs fine, as do all the first several variables with numeric code responses. But then when I get here:

caregroup_promoter_other
Duplicate variable name with o_prefill_codes-caregroup_promoter
local pos : list posof "o_prefill_codes-caregroup_promoter_other" in fields
local var : word pos' of all'

I get invalid syntax after this last command.

Any help figuring this out would be much appreciated!

-Kate

Matthew_White · August 23, 2017, 8:28pm

Hi Kate,

Stata will issue that error message if `pos' is 0, which could happen if o_prefill_codes-caregroup_promoter_other is not among the column headers of the CSV file. Does that column appear in the CSV file?

How are you exporting from KoBo to odkmeta?

odkmeta was built with ODK Briefcase in mind. Other servers, such as KoBo, sometimes add custom export functionality, but any differences from Briefcase can cause issues for odkmeta. Sometimes this custom export functionality has options to create a CSV file formatted similarly to Briefcase, but that's not always available.

For example, Briefcase exports CSV files with column headers that include the field's group name, separated from the field name with a hyphen. Some servers don't include the group name in the column header, or separate it from the field name with a character other than hyphen. KoBo's legacy CSV export seems to have this issue, as it supports / and . as group separator characters, but not -.

KoBo's newer CSV export allows you to specify a hyphen as the group separator if you select "XML values and headers" as the value and header format and you check "include groups in headers." However, when I tried this, the resulting file seemed to be missing data. For example, I don't see columns that contain the "other" responses to "select or other" fields. That is, for a single "select or other" field, I expect two columns in the resulting CSV file: one for the initial selection and one for the free-form text other response if "other" is initially selected. Does KoBo not export the free-form text other responses? I also don't see repeat groups in this file, so maybe I'm doing something wrong...

An alternative might be to forego KoBo's custom export functionality and use Briefcase instead. KoBo indicates here and here that it supports Briefcase pulls. However, it looks like that functionality hasn't been working since at least earlier this year:

If KoBo restored Briefcase support or supplemented its custom export functionality to generate Briefcase-like exports, odkmeta would be easily compatible with KoBo. Otherwise, we would need to make changes to odkmeta.

I will mention one other odkmeta option that may be helpful. Usually, when odkmeta does not find a field in the CSV file that it expects, the do-file results in an error. However, if you specify option relax, odkmeta does not issue that error message. See the help file for more information. This could resolve the issue you're seeing with "select or other" fields, but based on what I'm seeing with the KoBo export, I suspect that you may still encounter issues with repeat groups. There's also currently a bug with option relax that can result in an error when the CSV file contains longer field names, as yours seems to.

Hope that helps! Let me know if you have any questions.