Increasing string length beyond 255 for long geotraces

Hi, @Eddy_Rellum!

Aggregate doesn't support sending XLSForms. You should transform it to an XForm file and use that with Aggregate.

Hi, ggalmazor, is there a tool to transfer XLS files to XForms? I am an end-user, not a programmer. The issue that you are mentioning is completely new to me. I never worked with XForms.

Hi @Eddy_Rellum

Which file did you upload in the Aggregate?

Check this link https://docs.opendatakit.org/xlsform/ it has information on how to convert the form

You can use the offline or the online option.

No need to be a programmer :slight_smile: The these tools are used by most of the community here :slight_smile:

Just for convenience, this the link to an online converter: http://opendatakit.org/xlsform/

Hi both, yes, I tranfered the xls file with XLSForms. Although I am an end-user, I am using ODK for several years now. So I know how the conversion process and uploading to aggregate works. However, I have a problem with the increasing string length. For one way or another, it does not work. Some (older) forum discussion mention that you have to change things in the xml header and some say you only have to create a column (bind::odk:lenght) and the binding odk:length="xxxx" (xxxx = e.g. 1000). Then convert it to xml with XLSForms. However, this does not work. So my question is: Is there something wrong with my xls form or with my Aggregate version or do I still have to change something in the xml header after I changes my xls to xml.?

Oh, I see now :slight_smile:

Alright, I've converted your XSLForm to XForm and it's definitively not producing the right odk:length attribute on the binding:

<bind nodeset="/ODK_form_test_geotrace_lenght/baseline_area_ident_gps_trace" odk:length="odk:length=&quot;1000&quot;" type="geotrace"/>

This should be:

<bind nodeset="/ODK_form_test_geotrace_lenght/baseline_area_ident_gps_trace" odk:length="1000" type="geotrace"/>

I don't have experience with XSLForms, but it seems that you would probably want to change the value of the bind::odk:lenght column to just 1000.

2 Likes

Hi @Eddy_Rellum,

There is way of doing that in xls. This instruction were available in the previous site

Datastore String Length¶
In ODK Aggregate 1.x, you can supply an attribute to the to specify the maximum number of characters used to store a string field; by default, the datastore layer limits strings to 255 characters or less. If a submission has data beyond this length, it is silently truncated upon submission.

Specifying a value less than 255 can significantly improve the storage efficiency of the database layer. You can also specify a value greater than 255, up to a maximum of about 16000 UTF-8 characters. The higher you set this value, the worse the storage efficiency within the datastore.

To set the string length:

Step 1: In the header of the form definition, add a xmlns alias for odk (shown added as the last xmlnsalias at the bottom of the <h:html> tag):

<?xml version="1.0"?>

<h:html xmlns="http://www.w3.org/2002/xforms"
xmlns:h="http://www.w3.org/1999/xhtml"
xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:jr="http://openrosa.org/javarosa"
xmlns:odk="http://www.opendatakit.org/xforms"
>
<h:head>
...
Then, in the bind, you can define an odk:length attribute to specify the length of a string field. Length can be specified for barcode, select1 and string fields. Multi-selects (select) fields have a 255 character maximum length for each selection value (and selection values must not contain whitespace); there are no limits to the number of choices that may be selected within a multi-select.

<bind nodeset="/nm/repeat_observation/notes" type="string" odk:length="600"/>

Hope they are clear ...

Hi ggalmazor, that was the trick! It worked. So what I did was:

  1. create a column with bind::odk:length
  2. add to the respective node for 'geotrace': odk:length="1000"
  3. convert this to xml with XLSForms
  4. open the xml with a text editor and remove the 'garbage' (it said something like: odk:length="blabla%1000" produced and cnahge this to odk:length="1000".
  5. Save and close and upload it to aggregate.

Worked perfectly! Many thanks for the help! Also the other respondents for their effort. Thanks!

1 Like

Hi @Eddy_Rellum, see this as well

Step 1: Create a column in xls with "bind::odk-length" as a title
Step 2: For those text inputs specify the length ie 15999
Step 3: Convert the xls to xml
Step 4: Open the xml with a text editor such as notepad and change odk-length="15999" to odk:length="15999"
Step 5: Paste this to the form definition

<?xml version="1.0"?>

<h:html xmlns="http://www.w3.org/2002/xforms"
xmlns:h="http://www.w3.org/1999/xhtml"
xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:jr="http://openrosa.org/javarosa"
xmlns:odk="http://www.opendatakit.org/xforms"
>
<h:head>
...

This is how i used to do it, and have used it successfully for text inputs, have never used for geotrace
ODK_form_test_geotrace_lenght.xlsx (22.0 KB)

Hi Samwel, many thanks for your help. Good for others to know this and that there are several ways to do it. This also clarify the differences in approach. I did read about your solution too in previous discussions (working with bind::odk-length), but was not aware that in that case you must enter just the number of characters (not odk:length="xxx" as I did). For end-users this may be confusing.

1 Like

Great to see everyone jumping in to help here!

It looks like there is some old documentation around. As @ggalmazor has described, it should not be necessary to modify XML for this feature to work.

I have updated the documentation on increasing the Aggregate database field length with the current recommended approach.

The XLSForm should look like:

type name label bind::odk:length
select_multiple opt_abc multi Select multi 500
geoshape shape Select an area 1000
5 Likes

Thanks so much @LN. I was looking for this option. Though I can do that on XML file too; however, this is more convenient.

2 Likes

Hi LN,

What you are posting about the way the XLS form should look like is correct. However, for clarity: I still needed to edit my xml after converting the XLS to XML. Without xml edit, it did not work in my case. As I described, it worked for me if I removed the 'blabla-code' in odk:length (the converted xml contained a lot of code such as: odk:length="blabla%8%1000"). I replaced this 'blabla-code' with a text editor to odk:length="1000" (to have for 1000 characters). I don't know if this has to do with an older version of Aggregate (which I am using), or you have to carry out an xml-edit also for the latest Aggregate...

By the way: I just did a search in my xml on "odk:length" to find all the entrances that need to be changed.

Did you only include the value (e.g. 1000) in the bind::odk:length column? Can you share exactly what showed up in the XML? If you can share your new form and new XML, that would be great. What converter are you using to go from XLSForm to XML (e.g. XLSForm online or offline)?

1 Like

Hi LN,

I found out that the off-line XLS converter that I purchased does not work (version 1.4.0) with bind:odk:length. So only the online converter works (https://opendatakit.org/xlsform/)

When I convert the XLS with this online converter and open the xml with a text editor, the following code appears in the xml form:ODK_form_geoshape.xml (60 Bytes)

For other readers: This code can appear multiple times in your xml, depending on the number of entrances that you entered bind:odk:length in your xls. You have to replace them all.

So I removed all this code and replaced it by odk:length="15999". Then saved it and it works.

2 posts were split to a new topic: Suggestions for odk:length parameter

@Eddy_Rellum That output makes it look like you're specifying odk:length="15999" in your XLSForm when you should be specifying just 15999. Can you please double check that? This post shows exactly what the column should look like.

If not, please paste the row in your XLSForm that specifies the length so we can try to reproduce.

Hi LN,

That is correct. In the question row I put: odk:length="15999". See the attached excel that I used for the conversion to xml.ODK_form_test_geoshape.xlsx (20.1 KB)

Ok! Make sure you use just the number in that column and you should not need to do any XML modification.

Hi LN,

Oke. Thanks! if I don't write back, in case your sollution has worked.

1 Like