CSV Support File - Not showing special characters on Android devices

What is the problem? Please be detailed.

Hi, my name is Diego, and i'm from Argentina. My english is so bad but i will try to do my best.
The problem i have is with a CSV Support File. When i see the form and the data-preloading on my Android smartphone / tablet, special characters (coming from the CSV file) not showing correctly. For example, an accent is displayed like a black diamond with an interrogant symbol inside.

What ODK tool and version are you using? And on what device and operating system version?

I'm using the v1.11.1 ODK version, in two devices: Samsung Galaxy J2 (Smarthphone, Android 5.1.1) and a Noblex T7A6 (Tablet, Android 6.0).

What steps can we take to reproduce the problem?

I create a simple XML form and CSV support file with Excel (XLSForm), and later i pushed the two files with the ODK Aggregate function (Im using a Google Cloud server on appspot). Before upload, i save the CSV file on UTF-8 format, but nothing happen, the special characters still showing wrong.

What you have you tried to fix the problem?

I googled A LOT of time. More than two days... and still. I tryed with all the ODK software (XLSForm Offline, Validate, Briefcase, etc.) and too with changing the CSV format to another: ANSII, ISO, Unicode, etc. Nothing.

Anything else we should know or have? If you have a test form or screenshots or logs, attach here.

I use the pulldata() function in the form, to get the data from the CSV file. I read about the search() function but i dont understand too much. Anyway, I don't think this matters.
Screenshot: https://i.imgur.com/mHWwJg8.png

I really appreciate your help. Thank you very much in advance. Regards from Argentina, Diego.

Hi Diego!

So if I understand correctly, you have a problem with characters not showing correctly in the csv. However, it sounds like this is a problem is in the csv itself when you say that before uploading you saved the csv but the characters were still wrong. Do you have a version of the data without the bad formatting that you can go back to? UTF-8 should be the right format for the csv, but it will have a higher chance of success if you start with a file where the data is correct and then go straight to the UTF-8 csv.

1 Like

Hello, regarding THIS (it's in 2016) there was a problem of encoding, I don't know if it's fixes.
Hi, @LN could you look here, pleas?

Diego, your English is much better than my Spanish. :blush:

The screenshot you shared looks like it's from a select_one question. Did you make sure to save the form XML as UTF-8?

I made a quick test form here and tried that from Aggregate. All the special characters came through fine so I don't think there's anything wrong with the software.

If you'd like to try my form, you can use accents.xml (1.4 KB) and test.csv (48 Bytes)

If you could share your form that would make it easier to see what might be happening.

2 Likes

Hi Diego, I've been having some struggles with CSV support files and special characters as well. I use the search() function but I don't think that matters.

The problem I found is that when you have saved an xlsx file as CSV UTF-8, but then make some changes in the CSV in MS Excel and simply click "save", Microsoft for some reason does not save it as UTF-8 anymore. Each and every time you save your CSV you have to specifically indicate that it has to be UTF-8. Forget it once and your CSV is ruined.

For this reason I work with an xlsx file as a basis and save it as csv from there. Whenever I need to make changes, I make the changes in the xlsx file and save it as CSV UTF-8 again.

I'm not sure whether this is the cause of your issue as well, but perhaps it helps :slight_smile:

1 Like

Hi people! ¿How are you?

elmps2018, u understand wrong. The special characters in the CSV are showing great, but on devices they not.

abdoul, thanks for the linked topic and for calling to LN haha

@LN thanks for ur great reply. I downloaded the files and pushed to my server. Your forms are great, showing all the special characters correctly! But i cant find the difference with my forms. At first i thinked could be the name of the label's column. In your survey the column's name is called 'label', but in my form it's called 'label::English'. I removed the ::English part but nothing happened. Other thing that i see different in your files, is the way you show the special characters. I get the CSV data for passing like choice options, not like text like you. Anyway, i tryed show special characters only like text, but still the error persists. Finally, replying ur question, the XML's file dont give me problems with the special characters, if u see my screenshot in detail, u will notice the word "deseará" showing correctly. That word comes from the XML, not CSV, so the problem i think around in another place, not in the XML. Anyway, i leave u my files. I dont understand why ur forms showing great on my devices haha... thanks for all! :slight_smile:
Edit Because i'm a new user, cant upload files. I upload the files on another server, like a .rar file. Take the link: https://goo.gl/78PVUd

Lieke, i take that advice like a good practice. Thanks!

2 Likes

Hi again! I dont know if double-post is allowed in the forum, but i need get up the thread.

Anybody could with this? Thanks!

I was able to reproduce the problem when I loaded the file you included in the .rar above. In my text editor (Sublime Text), I opened the CSV and saved it with encoding UTF-8. I re-uploaded the form and CSV to Aggregate and then accents showed as expected. I would encourage you to save the CSV using a text editor rather than Excel.

Please take a moment to Introduce yourself here! so we can learn about you and the projects you work on!

3 Likes

@LN, u are awesome! I use Sublime Text too. I tried the steps u described, and the CSV shows great! U save me a lot, thanks, really. All is about UTF-8 encoding, but Excel save with errors. Sublime does it good.

Well, i will introduce myself ! Thanks again, and regards from Argentina. We still in contact!

Can anyone close the thread? :slight_smile:

2 Likes