ODK Briefcase v1.16 Beta

ggalmazor · June 25, 2019, 4:11pm

We need your help to ship great software, so please download this beta, try it, and report the issues you find. The release will be delayed until all reported issues with the beta are fixed.

For everyone who uses Briefcase, the important changes to verify are:

Replaced HTTP communication and parallel job execution layers for pull and push operations.

Briefcase will try to use all available local resources i.e. pull and push operations should feel snappier, and users can limit the number of HTTP connections thanks to new configuration settings.
Users can set a maximum number of simultaneous HTTP connections in the Settings tab of the UI or using the new -mhc CLI arg
Other things that would be great to get feedback about:
- Cancellation of jobs on the UI
- Tracking message language.

Download beta

ODK-Briefcase-v1.16.0-beta.0.jar

rassam · June 30, 2019, 9:04am

When ever I try to use ODK-Briefcase-v1.16.0-beta.0.jar i get an exception, java.sql.SQLException: No suitable driver found for jdbc:hsqldb. while ODK-Briefcase-v1.15.0 is working normally.

Many thanks.

dicksonsamwel · June 30, 2019, 9:47am

Hi @rassam,

Thanks for trying out the beta. Your inputs are highly appreciated.

Can you check for the log file and attach it here. You can find it in the same folder where the briefcase file is located. That file will provide better insights of what is happening.

Thanks.

rassam · June 30, 2019, 12:45pm

Hi @dicksonsamwel

Thanks for your rapid response,

attached is the log file.

Many thanks

briefcase.2019-06-28.log.txt (2.4 KB)

dicksonsamwel · July 1, 2019, 6:44am

Hi @rassam,

Is this the complete log file or you deleted some lines?

If so please, could you attach complete file here or send an email with the file to ggalmazor@gmail.com.

Thank you being a beta tester, your inputs are highly appreciated.

ggalmazor · July 1, 2019, 8:13am

Hi, @rassam! Sorry for taking so much time to answer to your question. I was off during the weekend.

It looks like you're using an OpenJDK Java release, which is perfectly fine in general, but it won't work with the beta. This beta-0 requires Oracle's Java8 release.

We've already solved that but the fix won't be available until the official v1.16 release.

rassam · July 1, 2019, 4:29pm

Hi @ggalmazor

Thanks for your clear response, Yes the issue was in the jdk version, i was using JDK-12. I've installed java 8 and every thing runs smoothly.

Thanks a lot

ggalmazor · July 1, 2019, 4:54pm

Thank you for testing the beta, @rassam!

rassam · July 5, 2019, 7:05pm

Hi @ggalmazor

one more thing, I have an issue in Arabic characters in the exported csv file, I've tried it in both beta and release of ODK-Briefcase v 1.16.0.

Note: I changed character encoding to 65001 Unicode (UTF-8), but its not working.

Thanks

yanokwa · July 5, 2019, 8:08pm

To import CSVs into Excel, you cannot download and open in one step; nor can you double-click on the CSV. You must open Excel and choose Import. If you are asked, the file origin or encoding is UTF-8. Can you try that and see if that works?

I've sent in a pull request to add this to the docs. https://github.com/opendatakit/docs/pull/1067

rassam · July 5, 2019, 8:39pm

Thanks @yanokwa for your rapid response.

I always did the same steps as you explained and it works for all releases of Briefcase prior to v 1.16. But in ODK-Briefcase v1.16 it doesn't work.

Many thanks

ggalmazor · July 8, 2019, 1:54pm

Hi, @rassam! Could you send me a form and some submissions to test it?

rassam · July 8, 2019, 3:33pm

Hi, @ggalmazor Attached is the form and ODK Briefcase Storage containing submissions.

Many Thanks.

KII.zip (957.8 KB)

ggalmazor · July 8, 2019, 3:55pm

Thanks! I'll try to reproduce the issue

ggalmazor · July 8, 2019, 4:11pm

Hi, @rassam!

I would like to confirm what I'm seeing in the submissions you've sent me. Take the submission with UID uuid:029be172-443e-4c5f-8c42-744a4f14de44, for example. I have this XML file:

<KII_C5 id="KII_C5" instanceID="uuid:029be172-443e-4c5f-8c42-744a4f14de44" version="1906251040" submissionDate="2019-07-01T19:49:09.956Z" isComplete="true" markedAsCompleteDate="2019-07-01T19:49:09.956Z" xmlns="http://opendatakit.org/submissions">
  <start>2019-06-30T18:33:24.934Z</start>
  <end>2019-07-01T19:48:06.678Z</end>
  <deviceid>353754070200048</deviceid>
  <subscriberid/>
  <simserial/>
  <username>YE1046</username>
  <KII_note/>
  <A>
    <KII_Name>&#216;&#174;&#216;&#167;&#217;&#8222;&#216;&#175; &#217;&#8230;&#216;&#173;&#217;&#8230;&#216;&#175; &#216;&#185;&#217;&#8222;&#217;&#352; &#216;&#167;&#217;&#8222;&#217;&#8230;&#216;&#177;&#217;&#352;</KII_Name>
    <Position>&#216;&#185;&#216;&#167;&#217;&#8218;&#217;&#8222; &#216;&#173;&#216;&#167;&#216;&#177;&#217;&#8225;</Position>
    <Phone_no>777208029</Phone_no>
    <Governorate>Aden</Governorate>
    <District>Al_Buraiqeh</District>
    <Ozla/>
    <Village/>
    <Area>Urban</Area>
    <Date_of_Interview>2019-07-01</Date_of_Interview>
    <Time_of_Interview>21:33:24</Time_of_Interview>
    <U_think_UR_eligible>No</U_think_UR_eligible>
  </A>
  <interview>
    <B>
      <Age>54</Age>
      <Gender>Male</Gender>
      <Educational_Status>Secondary</Educational_Status>
      <Years_of_education>9.0</Years_of_education>
      <KII_Profession>&#216;&#185;&#217;&#8230;&#217;&#8222; &#216;&#174;&#216;&#167;&#216;&#181;</KII_Profession>
    </B>
    <C>
      <C.Q1>Saw_the_poster</C.Q1>
      <C.Q1.1/>
      <C.Q2>More_than_one_month</C.Q2>
      <C.Q3>
        <generated_table_list_label_35/>
        <C.Q3.1>&#217;&#8222;&#216;&#167;&#217;&#352;&#217;&#710;&#216;&#172;&#216;&#175;</C.Q3.1>
        <C.Q3.2>&#217;&#8222;&#216;&#167;&#217;&#352;&#217;&#710;&#216;&#172;&#216;&#175;</C.Q3.2>
      </C.Q3>
      <C.Q4>Help_the_people_in_Yemen</C.Q4>
      <C.Q4.1/>
      <C.Q5>Former_SWF_beneficiaries</C.Q5>
      <C.Q5.1/>
      <C.Q6>This_is_a_new_UNICEF_project</C.Q6>
      <C.Q6.1/>
      <C.Q7>UNICEF</C.Q7>
      <C.Q7.1/>
      <C.Q8>To_buy_food To_pay_for_health_costs</C.Q8>
      <C.Q8.1/>
      <C.Q9>They_are_happy_to_receive_cash</C.Q9>
      <C.Q9.1/>
      <C.Q10>No</C.Q10>
      <C.Q10.1/>
      <C.Q10.1.1/>
      <C.Q11>No</C.Q11>
      <C.Q11.1/>
      <C.Q11.1.1/>
      <C.Q12>I_dont_know_enough_about_the_payment_process_to_allow_myself_any_judgment_on_it</C.Q12>
      <C.Q13>No</C.Q13>
      <C.Q13.1/>
      <C.Q13.1.1/>
      <C.Q14>Yes</C.Q14>
      <C.Q15>SMS</C.Q15>
      <C.Q15.1/>
    </C>
    <D>
      <D.Q16/>
      <D.Q16.1>No</D.Q16.1>
      <D.Q16.1.1/>
      <D.Q16.2>No</D.Q16.2>
      <D.Q16.2.1/>
      <D.Q16.3>No</D.Q16.3>
      <D.Q16.3.1/>
      <D.Q16.4>No</D.Q16.4>
      <D.Q16.4.1/>
      <D.Q16.5>No</D.Q16.5>
      <D.Q16.5.1/>
      <D.Q16.6>No</D.Q16.6>
      <D.Q16.6.1/>
      <D.Q16.7>No</D.Q16.7>
      <D.Q16.7.1/>
      <D.Q17>&#217;&#8222;&#216;&#167;&#217;&#352;&#217;&#710;&#216;&#172;&#216;&#175;</D.Q17>
      <D.Q18>&#217;&#8222;&#216;&#167;&#217;&#352;&#217;&#710;&#216;&#172;&#216;&#175;</D.Q18>
    </D>
    <E>&#217;&#8222;&#216;&#167;&#217;&#352;&#217;&#710;&#216;&#172;&#216;&#175;</E>
  </interview>
  <Supervisor_name>Asel_Ali_Mohammed_Soliman</Supervisor_name>
  <Enumerator_ID>YE1046</Enumerator_ID>
  <n0:meta xmlns:n0="http://openrosa.org/xforms">
    <n0:audit>audit.csv</n0:audit>
    <n0:instanceID>uuid:029be172-443e-4c5f-8c42-744a4f14de44</n0:instanceID>
  </n0:meta>
</KII_C5>

Take a closer look to the answer for the KII_Name field: <KII_Name>Ø®Ø§Ù„Ø¯ Ù…ØÙ…Ø¯ Ø¹Ù„ÙŠ Ø§Ù„Ù…Ø±ÙŠ</KII_Name>

As you can see, all the characters are encoded as their escaped UTF-8 entity codes. Using an online decoder, I can translate that list of characters to this: Ø®Ø§Ù„Ø¯ Ù…ØÙ…Ø¯ Ø¹Ù„ÙŠ Ø§Ù„Ù…Ø±ÙŠ. This matches what Briefcase v1.16 exports in my case. I've verified that Briefcase v1.15 produces the same file.

Taking this results into account, my guess at this moment is that there's something wrong in the source submissions. @rassam, could you double check the ZIP file you've sent me? Maybe you could push the form and submissions to the sandbox Aggregate server at https://sandbox.aggregate.opendatakit.org so that I can pull it with older versions of Briefcase to see if I get different submission contents.

rassam · July 8, 2019, 5:06pm

Hi, @ggalmazor

I guess the issue is in the pull process using ODK-Briefcase v1.16.0, when I pulled the same instance using ODK-Briefcase v1.15.0 it produces a a submission file with correct encoding.
Attached is the same submission with UID uuid:029be172-443e-4c5f-8c42-744a4f14de44 that I pulled using ODK-Briefcase v1.15.0.

Many thanks.submission.xml (2.6 KB)

rassam · July 8, 2019, 5:13pm

Hi, @ggalmazor I,ve uploaded the form and two submissions https://sandbox.aggregate.opendatakit.org. so that you can test it.

Many thanks.

ggalmazor · July 8, 2019, 6:27pm

That would make sense.

Testing it, thanks!

ggalmazor · July 8, 2019, 6:33pm

OK, this is what I've done (all from a clean state: no prefs, empty storage directory):

First, pull the form with v1.15 and then export it with v1.16 works
Second, pull and export the form with v1.16

Both scenarios worked OK. My computer uses es_ES.UTF-8 locale

I'm guessing that your default encoding wasn't UTF-8 when you pulled the form with v1.16. Could you confirm this?

rassam · July 8, 2019, 8:22pm

Hi, @ggalmazor Thanks for your valuable support.

I think that the issue is in my computer, windows OS, I've tried it in Ubuntu and it worked correctly.

Thanks!