Forms with French accent chars

Have a project in Mali and am having major issue with forms that use
accent characters in french. Using collect on Android the forms work,
however, the accent characters appear as question marks with black
backgrounds. We also are using JavaRosa on Nokia phones. Same forms
crash the app with an xform parsing exception. However, since the
JavaRosa app allows one to specify a different fetch server, I have
taken the same xml files and put them on the file system and used a
simple php script to put up the formList and send the phone the xml
file from disk storage. These work fine and the accent marks appear
properly.

Was wondering if anybody has a handle on this, with Aggregate, why
when the same xml is sent to the phone it does not work. Could it be
the fact the the form is stored in a BLOB and that the Java string
type (UTF-16). I have been able to verify that my workaround test and
Aggregate are sending the same header.

Thoughts??

··· -- John B. Murphy jbmurphyUSA@gmail.com

Which version of Aggregate are you using? If you are using a 0.9.x
version then a fix contained in in the unreleased Aggregate 1.x is to
modify the response type variable to be

RESP_TYPE_XML = "text/xml";

to

RESP_TYPE_XML = "text/xml; charset=utf-8";

Waylon

··· On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: > Have a project in Mali and am having major issue with forms that use > accent characters in french. Using collect on Android the forms work, > however, the accent characters appear as question marks with black > backgrounds. We also are using JavaRosa on Nokia phones. Same forms > crash the app with an xform parsing exception. However, since the > JavaRosa app allows one to specify a different fetch server, I have > taken the same xml files and put them on the file system and used a > simple php script to put up the formList and send the phone the xml > file from disk storage. These work fine and the accent marks appear > properly. > > Was wondering if anybody has a handle on this, with Aggregate, why > when the same xml is sent to the phone it does not work. Could it be > the fact the the form is stored in a BLOB and that the Java string > type (UTF-16). I have been able to verify that my workaround test and > Aggregate are sending the same header. > > Thoughts?? > > -- > John B. Murphy > jbmurphyUSA@gmail.com >

To keep the thread up to date an issue was filed:

http://code.google.com/p/opendatakit/issues/detail?id=215&start=100

Thanks John for finding this issue!

Waylon

··· On Wed, May 18, 2011 at 5:27 PM, W. Brunette wrote: > Which version of Aggregate are you using? If you are using a 0.9.x > version then a fix contained in in the unreleased Aggregate 1.x is to > modify the response type variable to be > > RESP_TYPE_XML = "text/xml"; > > to > > RESP_TYPE_XML = "text/xml; charset=utf-8"; > > Waylon > > On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: >> Have a project in Mali and am having major issue with forms that use >> accent characters in french. Using collect on Android the forms work, >> however, the accent characters appear as question marks with black >> backgrounds. We also are using JavaRosa on Nokia phones. Same forms >> crash the app with an xform parsing exception. However, since the >> JavaRosa app allows one to specify a different fetch server, I have >> taken the same xml files and put them on the file system and used a >> simple php script to put up the formList and send the phone the xml >> file from disk storage. These work fine and the accent marks appear >> properly. >> >> Was wondering if anybody has a handle on this, with Aggregate, why >> when the same xml is sent to the phone it does not work. Could it be >> the fact the the form is stored in a BLOB and that the Java string >> type (UTF-16). I have been able to verify that my workaround test and >> Aggregate are sending the same header. >> >> Thoughts?? >> >> -- >> John B. Murphy >> jbmurphyUSA@gmail.com >> >

Waylon,

We have a project in Mali starting next week. Requires french language
forms. Is there anyway we can get the build scripts for Alpha6 with MySQL
back-end so we could apply that fix?

Or get a war with the fix in it?

No joy building uiexperiment.

Best
John

John B. Murphy
(617) 504-1531
jbmurphyUSA@gmail.com

··· On May 18, 2011 8:27 PM, "W. Brunette" wrote: > Which version of Aggregate are you using? If you are using a 0.9.x > version then a fix contained in in the unreleased Aggregate 1.x is to > modify the response type variable to be > > RESP_TYPE_XML = "text/xml"; > > to > > RESP_TYPE_XML = "text/xml; charset=utf-8"; > > Waylon > > On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: >> Have a project in Mali and am having major issue with forms that use >> accent characters in french. Using collect on Android the forms work, >> however, the accent characters appear as question marks with black >> backgrounds. We also are using JavaRosa on Nokia phones. Same forms >> crash the app with an xform parsing exception. However, since the >> JavaRosa app allows one to specify a different fetch server, I have >> taken the same xml files and put them on the file system and used a >> simple php script to put up the formList and send the phone the xml >> file from disk storage. These work fine and the accent marks appear >> properly. >> >> Was wondering if anybody has a handle on this, with Aggregate, why >> when the same xml is sent to the phone it does not work. Could it be >> the fact the the form is stored in a BLOB and that the Java string >> type (UTF-16). I have been able to verify that my workaround test and >> Aggregate are sending the same header. >> >> Thoughts?? >> >> -- >> John B. Murphy >> jbmurphyUSA@gmail.com >>

Waylon,

Was just looking at the code,

note that

    } else {
      resp.setContentType(HtmlConsts.RESP_TYPE_XML);

in src/org/opendatakit/aggregate/servlet/FormXmlServlet.java

Will have no effect because it is below the call to PrinteWriter.

Just FYI

-john

··· On Fri, May 20, 2011 at 4:38 PM, W. Brunette wrote: > To keep the thread up to date an issue was filed: > > http://code.google.com/p/opendatakit/issues/detail?id=215&start=100 > > Thanks John for finding this issue! > > Waylon > > On Wed, May 18, 2011 at 5:27 PM, W. Brunette wrote: >> Which version of Aggregate are you using? If you are using a 0.9.x >> version then a fix contained in in the unreleased Aggregate 1.x is to >> modify the response type variable to be >> >> RESP_TYPE_XML = "text/xml"; >> >> to >> >> RESP_TYPE_XML = "text/xml; charset=utf-8"; >> >> Waylon >> >> On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: >>> Have a project in Mali and am having major issue with forms that use >>> accent characters in french. Using collect on Android the forms work, >>> however, the accent characters appear as question marks with black >>> backgrounds. We also are using JavaRosa on Nokia phones. Same forms >>> crash the app with an xform parsing exception. However, since the >>> JavaRosa app allows one to specify a different fetch server, I have >>> taken the same xml files and put them on the file system and used a >>> simple php script to put up the formList and send the phone the xml >>> file from disk storage. These work fine and the accent marks appear >>> properly. >>> >>> Was wondering if anybody has a handle on this, with Aggregate, why >>> when the same xml is sent to the phone it does not work. Could it be >>> the fact the the form is stored in a BLOB and that the Java string >>> type (UTF-16). I have been able to verify that my workaround test and >>> Aggregate are sending the same header. >>> >>> Thoughts?? >>> >>> -- >>> John B. Murphy >>> jbmurphyUSA@gmail.com >>> >> >

--
John B. Murphy
(617) 504-1531
jbmurphyUSA@gmail.com

John,

Mitch is working on building the new tree right now as he is switching
things to maven and there are some glitches still. Feel free to start
an email exchange with him directly msundt@cs.washington.edu (i
checked with Mitch and he said that would be best).

Waylon

··· On Tue, May 24, 2011 at 12:00 PM, John Murphy wrote: > Waylon, > > We have a project in Mali starting next week. Requires french language > forms. Is there anyway we can get the build scripts for Alpha6 with MySQL > back-end so we could apply that fix? > > Or get a war with the fix in it? > > No joy building uiexperiment. > > Best > John > > John B. Murphy > (617) 504-1531 > jbmurphyUSA@gmail.com > > On May 18, 2011 8:27 PM, "W. Brunette" wrote: >> Which version of Aggregate are you using? If you are using a 0.9.x >> version then a fix contained in in the unreleased Aggregate 1.x is to >> modify the response type variable to be >> >> RESP_TYPE_XML = "text/xml"; >> >> to >> >> RESP_TYPE_XML = "text/xml; charset=utf-8"; >> >> Waylon >> >> On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: >>> Have a project in Mali and am having major issue with forms that use >>> accent characters in french. Using collect on Android the forms work, >>> however, the accent characters appear as question marks with black >>> backgrounds. We also are using JavaRosa on Nokia phones. Same forms >>> crash the app with an xform parsing exception. However, since the >>> JavaRosa app allows one to specify a different fetch server, I have >>> taken the same xml files and put them on the file system and used a >>> simple php script to put up the formList and send the phone the xml >>> file from disk storage. These work fine and the accent marks appear >>> properly. >>> >>> Was wondering if anybody has a handle on this, with Aggregate, why >>> when the same xml is sent to the phone it does not work. Could it be >>> the fact the the form is stored in a BLOB and that the Java string >>> type (UTF-16). I have been able to verify that my workaround test and >>> Aggregate are sending the same header. >>> >>> Thoughts?? >>> >>> -- >>> John B. Murphy >>> jbmurphyUSA@gmail.com >>> >

Okay, cool.

Thanks
John

John B. Murphy
(617) 504-1531
jbmurphyUSA@gmail.com

··· On May 24, 2011 3:05 PM, "W. Brunette" wrote: > John, > > Mitch is working on building the new tree right now as he is switching > things to maven and there are some glitches still. Feel free to start > an email exchange with him directly msundt@cs.washington.edu (i > checked with Mitch and he said that would be best). > > Waylon > > On Tue, May 24, 2011 at 12:00 PM, John Murphy wrote: >> Waylon, >> >> We have a project in Mali starting next week. Requires french language >> forms. Is there anyway we can get the build scripts for Alpha6 with MySQL >> back-end so we could apply that fix? >> >> Or get a war with the fix in it? >> >> No joy building uiexperiment. >> >> Best >> John >> >> John B. Murphy >> (617) 504-1531 >> jbmurphyUSA@gmail.com >> >> On May 18, 2011 8:27 PM, "W. Brunette" wrote: >>> Which version of Aggregate are you using? If you are using a 0.9.x >>> version then a fix contained in in the unreleased Aggregate 1.x is to >>> modify the response type variable to be >>> >>> RESP_TYPE_XML = "text/xml"; >>> >>> to >>> >>> RESP_TYPE_XML = "text/xml; charset=utf-8"; >>> >>> Waylon >>> >>> On Wed, May 18, 2011 at 12:58 PM, John Murphy wrote: >>>> Have a project in Mali and am having major issue with forms that use >>>> accent characters in french. Using collect on Android the forms work, >>>> however, the accent characters appear as question marks with black >>>> backgrounds. We also are using JavaRosa on Nokia phones. Same forms >>>> crash the app with an xform parsing exception. However, since the >>>> JavaRosa app allows one to specify a different fetch server, I have >>>> taken the same xml files and put them on the file system and used a >>>> simple php script to put up the formList and send the phone the xml >>>> file from disk storage. These work fine and the accent marks appear >>>> properly. >>>> >>>> Was wondering if anybody has a handle on this, with Aggregate, why >>>> when the same xml is sent to the phone it does not work. Could it be >>>> the fact the the form is stored in a BLOB and that the Java string >>>> type (UTF-16). I have been able to verify that my workaround test and >>>> Aggregate are sending the same header. >>>> >>>> Thoughts?? >>>> >>>> -- >>>> John B. Murphy >>>> jbmurphyUSA@gmail.com >>>> >>

Hello,

I had a similar issue with the form in displaying scripts in Hindi (One of
the Indian language) and we could overcome by tweaking the aggregate code:

src/odk/aggregate/servlet/FormXmlServlet.java:

Current :

xmlString = xmlString.toString();
resp.setCharacterEncoding(ServletConsts.ENCODE_SCHEME);
PrintWriter out = resp.getWriter();

Earlier/prior to above edit:

PrintWriter out = resp.getWriter();
resp.setCharacterEncoding(“utf-8”)

Also, I tried to test by downloading the "test_accents.xml" from
http://code.google.com/p/opendatakit/issues/detail?id=215&start=100 and it
is worked for me.

Thanks,
Nayak

··· On Wed, May 25, 2011 at 12:37 AM, John Murphy wrote:

Okay, cool.

Thanks
John

John B. Murphy
(617) 504-1531
jbmurphyUSA@gmail.com
On May 24, 2011 3:05 PM, "W. Brunette" wbrunette@gmail.com wrote:

John,

Mitch is working on building the new tree right now as he is switching
things to maven and there are some glitches still. Feel free to start
an email exchange with him directly msundt@cs.washington.edu (i
checked with Mitch and he said that would be best).

Waylon

On Tue, May 24, 2011 at 12:00 PM, John Murphy jbmurphyusa@gmail.com wrote:

Waylon,

We have a project in Mali starting next week. Requires french language
forms. Is there anyway we can get the build scripts for Alpha6 with
MySQL
back-end so we could apply that fix?

Or get a war with the fix in it?

No joy building uiexperiment.

Best
John

John B. Murphy
(617) 504-1531
jbmurphyUSA@gmail.com

On May 18, 2011 8:27 PM, "W. Brunette" wbrunette@gmail.com wrote:

Which version of Aggregate are you using? If you are using a 0.9.x
version then a fix contained in in the unreleased Aggregate 1.x is to
modify the response type variable to be

RESP_TYPE_XML = "text/xml";

to

RESP_TYPE_XML = "text/xml; charset=utf-8";

Waylon

On Wed, May 18, 2011 at 12:58 PM, John Murphy jbmurphyusa@gmail.com wrote:

Have a project in Mali and am having major issue with forms that use
accent characters in french. Using collect on Android the forms work,
however, the accent characters appear as question marks with black
backgrounds. We also are using JavaRosa on Nokia phones. Same forms
crash the app with an xform parsing exception. However, since the
JavaRosa app allows one to specify a different fetch server, I have
taken the same xml files and put them on the file system and used a
simple php script to put up the formList and send the phone the xml
file from disk storage. These work fine and the accent marks appear
properly.

Was wondering if anybody has a handle on this, with Aggregate, why
when the same xml is sent to the phone it does not work. Could it be
the fact the the form is stored in a BLOB and that the Java string
type (UTF-16). I have been able to verify that my workaround test and
Aggregate are sending the same header.

Thoughts??

--
John B. Murphy
jbmurphyUSA@gmail.com