Regex Constraint Syntax Confusion - validates online but not in form

esobel · October 2, 2018, 2:32pm

Hi everyone! I've been searching here and elsewhere for a solution to my problem but all my attempts at resolving it myself have been fruitless. I have a form that collects many measurements and want to force users to enter them the same way each time and across users.
I would like all measurements to be in feet and inches. Example: 1'-9" or 12'-11" or 100'-2"
I have tried regex but all my attempts throw a parse error when I get to Nafundi's xlsform Offline. I have found code on stackoverflow that validates in regex101 but will not pass the validate. Please help!

I started with ODK Build (v0.3.3?) and exported to xls and edited there (on Win7 x64)
We use Collect v1.16.2 on Android 7 tablets.

I have kept logs of all I have tried with regard to regex. Here are my attempts:
--regex log--
first attempt - failed parse: regex(., '[0-9]{1,2}['][-][0-9]{1,2}["]')
code from stack: ^(?!$|.'[^\x22]+$)(?:([0-9]+)')?(?:([0-9]+)\x22?)?$ - didn't include a hyphen separator
edited for xlsform: regex(^(?!$|.'[^\x22]+$)(?:([0-9]+)')-) - failed parse
attempt 2: regex(., '(?!$|.'[^\x22]+$)(?:([0-9]+)')-') - failed parse
attempt 3: ^(?!$|.'[^\x22]+$)(?:([0-9]+)')[-](?:([0-9]{1,2}\x22?)$
attempt 4: regex(, '^(?!$|.*'[^\x22]+$)(?:([0-9]+)')-

link to original edited regex - https://regex101.com/r/mL8fD7/26

I don't see a way to attach my xls or screenshots.
Thank you in advance for the help!!

yanokwa · October 2, 2018, 6:20pm

I haven't tried it in the form, but a simple regex would be [0-9]+'-[0-9]+". But you might run into problems with quotes in XML so that has to be escaped and maybe your devices will use smart quotes, etc. And even if you get that working, it seems like putting the units would make it hard to analyze the data.

I would recommend a different approach. Use two fields in a group and display it on one screen and ask the user to enter data for feet and data for inches. Then put a constraint on inches so it can't go over 12.

esobel · October 2, 2018, 7:05pm

Thanks, I thought about doing 2-4 fields and concatenating for end users. The challenge I am facing is that data is collected in one way in the field and used by designers and analysts in different ways in the office.
Perhaps I could do a decimal field and ask them to convert their measurements.

esobel · October 2, 2018, 7:23pm

Is there somewhere that spells out the syntax of regex required for xls forms? It seems to require "regex(., 'expression here') in lieu of ^(expression$)
I'm new to regex so I'm still learning which parts do what but it seems like the excel form needs to be told the constraint is regex? When exported from Build, it just shows as (. >4) so it's kind of confusing.

yanokwa · October 2, 2018, 8:21pm

https://docs.opendatakit.org/form-regex has a good primer!

esobel · October 3, 2018, 5:11pm

As an update, after much testing (and failing), I managed to get " to work by using &quot; in the xml file.
I am still unable to get the apostrophe to work - I have tried &apos; in the xls but I got a "not machine readable" error and the xml did not validate 5' in the form....only 5
Using &apos; in the xml throws a form parse error on Collect and won't open the form.
I would like to get this resolved without using additional fields. If I could have one string instead of 2 with a calculate, it would be ideal.
For anyone searching and coming here after me, I found this blog to be helpful in explaining what xls is looking for with regex and providing examples:
http://blog.cartong.org/2018/03/15/all-about-regex-in-xls-form-when-how-and-examples-in-the-humanitarian-and-development-fields/#comment-64170
Through that blog, I changed my regex to include a carrot ^ and it seemed to help.
These are the expressions I tried since my OP:

  		attempt 5:	regex(., '^([0-9]{1,2}\'[-][0-9]{1,2}\")$') 
  		attempt 6:	regex(., '^([0-9]{1,2}[\'][-][0-9]{1,2}[\"])$') 
  		attempt 7:	regex(., '^([0-9]{1,2}['][-][0-9]{1,2}["])$') 
  		attempt 8:	regex(.,'^(.([0-9]{1,3})['- .)]*[0-9]{1,2}["]$')
  		attempt 9: regex(., '^([0-9]{1,2}[\'][-][0-9]{1,2}[\x22])$') 
  		attempt 10:	regex(., '^([0-9]{1,2}[\'])*([-][0-9]{1,2}[\x22])$') 
  	for quotes - &quot;	works when edited in xml but &apos; does not
  		attempt 11:	regex(., '[0-9 ().',%+-]{1,150}')
  		attempt 12: 	regex(., '^([0-9 ().',%+-]{1,150})$')
  		attempt 13:	regex(., '^([0-9]{1,3})*([',%+-])$')
  		attempt 14:	regex(., '^([0-9]{1,3})*\"'"$')
  		attempt 15:	regex(., '^([0-9'-]{1,3})$')
  		attempt 16:	regex(., '^([0-9'\-]{1,3})$')
  		attempt 17:	regex(., '^([0-9]+[']{1,3})+$')
  		attempt 18:	regex(., '^([0-9]*[']{1,3})*$')
  		attempt 19:	regex(., '^([0-9\'\-]{1,3})*$')
  		attempt 20: regex(., '^([0-9 \'']{1,3})$') - failed with bad node
  		attempt 21:	regex(., '^([0-9] [\']{1,3})$')
  		attempt 22:	regex(., '^([0-9]*[\' ']{1,3})$')
  		attempt 23:	regex(., '^([0-9]*[\' ']{1,3})*$')
  		attempt 24:	regex(., '^([0-9]*[']{1,3})*$')
  		attempt 25:	regex(., '^([0-9]*\'{1,3})*$')
  		attempt 26:	regex(., '^([0-9]+\'{1,3})+$')
  		attempt 27:	regex(., '^([0-9]+('&apos;'){1,3})+$')
  		attempt 28:	regex(., '^([0-9]*('&apos;'){1,3})*$')
  		attempt 29:	regex(., '^([0-9]*(&apos;){1,3})*$') - warning machine readable language
  			produced a xml that would not validate x' in the Collect App.  only numeric values
  		attempt 30:	regex(., '^([0-9]*(\'){1,3})*$')
  		attempt 31:	regex(., '^([0-9]*(\'\){1,3})*$')
  		attempt 32:	regex(., '^([0-9]*''{1,3})*$')
  		attempt 33:	regex(., '^([0-9 \' ]{1,3})$')
  		attempt 34:	regex(., '^([0-9 \' \-]{1,3})$')

I am committed to figuring this out.

Xiphware · October 4, 2018, 10:03am

Try this:

regex(.,'^[0-9]+[\x27][-][0-9]+["]$')

I tested it and appears to work as desired under KoboToolbox (as the Validation Criteria -> Validation Code); eg it will accept the likes of

1'-9"
12'-11"
100'-2"

but YMMV depending if you try to push this thru other tools, which may have other special characters that must be escaped appropriately... Also, obviously, its not checking that your inches are <=12; that would require a far more sophisticated regex [exercise left to the reader...].

I found the trick was having to escape the feet's single quote in the regex itself (!), using its ASCI code (the single quote ' is ASCI hex 27). Otherwise, trying to escape it within the XPath expression, or XML, etc just messed everything up.

Note: Kobo translates the " to " for you when you export the form. So the actual XML binding looks like:

<bind constraint="regex(.,'^[0-9]+[\x27][-][0-9]+["]$')" nodeset="/data/enter_feet_and_or_inches" required="false()" type="string"/>

Good luck - these sorta brain-teasers keep me up at night too...

esobel · October 4, 2018, 1:31pm

Oh my gosh! It worked!! I feel like I tried everything but I never thought to use the character code for the single tick mark. Ha, thank you SO SO much!

Xiphware · October 4, 2018, 8:54pm

By the looks of it, I'd say you had probably tried about every (other) conceivable permutation! Your perseverance is truly admirable, so I'm looking forward to seeing your regex for checking inches...

BTW, I might suggest a slightly more permissive regex:

regex(.,'^[0-9]+[\x27] *[-,]{0,1} *[0-9]+[\x22]$')

which will accept such variations as

14' 3"
14',3"
14' - 3"
14'3"

[as well as using the hex to escape the double quote too, for consistency and 'cleanliness']

esobel · October 5, 2018, 3:21pm

Thanks again! Not a bad idea to use hex as a cleaner resolution. This way, I'll remember that those characters need to be treated special. If I leave one, I might not remember why the other isn't escaped.