Question about briefcase and duplicate data

Support
1

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the data
    still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent as a
    batch (we did 15 submissions) and it says sending 1 of 15 etc - if this
    fails at 10 of 15 due to poor internet, if you re action the send request
    it sends all the submissions again and then creates duplicates in the
    appspot.

Is there any way to get around these problems or is there a way to manage
the duplicates better in appspot.

Thanks

Jon

2

Briefcase DOES NOT delete any data off your phone. It is up to you, after
using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form definition.

<h:html xmlns="http://www.w3.org/2002/xforms"
xmlns:h="http://www.w3.org/1999/xhtml"
xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:jr="http://openrosa.org/javarosa"
xmlns:orx="http://openrosa.org/xforms/"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the OpenRosa
metadata standard (
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ).
Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or a
blank (default) namespace and looks for the first occurance of
meta/instanceID in the form and uses that value as the unique identifier
for the submission. If a filled-in form has a unique identifier, Aggregate
can merge the multiple uploads and prevent duplications.

Mitch

··· On Mon, May 14, 2012 at 9:23 AM, Jon Parsons wrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the data
    still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent as a
    batch (we did 15 submissions) and it says sending 1 of 15 etc - if this
    fails at 10 of 15 due to poor internet, if you re action the send request
    it sends all the submissions again and then creates duplicates in the
    appspot.

Is there any way to get around these problems or is there a way to manage
the duplicates better in appspot.

Thanks

Jon

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

3

Hey Mitch,
Thanks for the tip. Jon, I guess we should add that field into the forms
soon as possible, just as precaution to prevent a similar incident which
Ricky (upload over what was already uploaded).

··· On Mon, May 14, 2012 at 1:36 PM, Mitch S wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you, after
using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form definition.

<h:html xmlns="http://www.w3.org/2002/xforms"
xmlns:h="http://www.w3.org/1999/xhtml"
xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:jr="http://openrosa.org/javarosa"
xmlns:orx="http://openrosa.org/xforms/"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the OpenRosa
metadata standard (
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ).
Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or a
blank (default) namespace and looks for the first occurance of
meta/instanceID in the form and uses that value as the unique identifier
for the submission. If a filled-in form has a unique identifier, Aggregate
can merge the multiple uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons j.parsons@globalcanopy.orgwrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the data
    still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent as
    a batch (we did 15 submissions) and it says sending 1 of 15 etc - if this
    fails at 10 of 15 due to poor internet, if you re action the send request
    it sends all the submissions again and then creates duplicates in the
    appspot.

Is there any way to get around these problems or is there a way to manage
the duplicates better in appspot.

Thanks

Jon

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss of
enthusiasm"
---------------------------------------------------------------------------------------------------------->
OutSourceRox*
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

4

Mitch, I just saw this and I know it might be a dumb question, but I
already have an section in my xml file. Do I add another
so I put what you mentioned below or do I add that block to the
current section? Also, for , does the label need to be similar to the title "My Data" in my form
which has
<Price_Tracker id="Price_Tracker">??
I'm just not sure...if you could clarify. I can see myself messing up
somewhere. :slight_smile: Thanks.

DataMax

··· On Monday, May 14, 2012 8:36:21 PM UTC+3, Mitch wrote: > > Briefcase DOES NOT delete any data off your phone. It is up to you, after > using briefcase, to select the forms on the phone and delete them. > Additionally, Briefcase does not attempt to find only the finalized forms > on your phone -- it will grab everything. So you should first finalize the > forms on the phone; if that isn't possible, you can go into Briefcase's > directory tree and delete the incomplete submissions from there, if needed > (they will generally have the same directory name at this point). > > As for duplicates, you should define an instanceID in your form definition. > > xmlns:h="http://www.w3.org/1999/xhtml" > xmlns:ev="http://www.w3.org/2001/xml-events" > xmlns:xsd="http://www.w3.org/2001/XMLSchema" > xmlns:jr="http://openrosa.org/javarosa" > xmlns:orx="http://openrosa.org/xforms/" > > > > My Data > > > > > > > ... > > readonly="true()" calculate="concat('uuid:', uuid())"/> > ... > > You can omit the orx: namespace prefix, but the above follows the OpenRosa > metadata standard ( > https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ). > Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or a > blank (default) namespace and looks for the first occurance of > meta/instanceID in the form and uses that value as the unique identifier > for the submission. If a filled-in form has a unique identifier, Aggregate > can merge the multiple uploads and prevent duplications. > > Mitch > > On Mon, May 14, 2012 at 9:23 AM, Jon Parsons <j.pa...@globalcanopy.org wrote: > >> Hi >> >> We have used briefcase in the field in Guyana, to get around internet >> connection problems, as the phones just would not send data direct through >> wifi and gave us all sorts of error and connection problems. BTW We only >> had a 512kbs down and 256kbs up data rate through a satelite that was >> giving us an actual 200kbs down and 50kbs up rate. >> >> But I have two questions about briefcase and its use >> - We have noticed that when briefcase pulls data from the phone, the data >> still there in the send finalised form area, so if the phone is then >> finally connects to good internet, these forms will be submitted and create >> duplicates. >> - Also when the data is collected together in briefcase and then sent as >> a batch (we did 15 submissions) and it says sending 1 of 15 etc - if this >> fails at 10 of 15 due to poor internet, if you re action the send request >> it sends all the submissions again and then creates duplicates in the >> appspot. >> >> Is there any way to get around these problems or is there a way to manage >> the duplicates better in appspot. >> >> Thanks >> >> Jon >> >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >
5

I am creating new forms using XLSForm http://opendatakit.org/use/xlsform/ and
would like to incorporate the code you suggest to help with duplicates, but
I am not sure how to incorporate this code into the excel document

could you advise

Thanks

··· On Wednesday, 3 October 2012 11:59:17 UTC+1, Jon Parsons wrote: > > Hi Mitch > > apologies - I previously posted a reply to this post while logged in with > another google account (g.lea) could you ignore/delete these posts > > Jon > > On Monday, 14 May 2012 18:36:21 UTC+1, Mitch wrote: >> >> Briefcase DOES NOT delete any data off your phone. It is up to you, >> after using briefcase, to select the forms on the phone and delete them. >> Additionally, Briefcase does not attempt to find only the finalized forms >> on your phone -- it will grab everything. So you should first finalize the >> forms on the phone; if that isn't possible, you can go into Briefcase's >> directory tree and delete the incomplete submissions from there, if needed >> (they will generally have the same directory name at this point). >> >> As for duplicates, you should define an instanceID in your form >> definition. >> >> > xmlns:h="http://www.w3.org/1999/xhtml" >> xmlns:ev="http://www.w3.org/2001/xml-events" >> xmlns:xsd="http://www.w3.org/2001/XMLSchema" >> xmlns:jr="http://openrosa.org/javarosa" >> xmlns:orx="http://openrosa.org/xforms/" >> > >> >> My Data >> >> >> >> >> >> >> ... >> >> > readonly="true()" calculate="concat('uuid:', uuid())"/> >> ... >> >> You can omit the orx: namespace prefix, but the above follows the >> OpenRosa metadata standard ( >> https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ). >> Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or >> a blank (default) namespace and looks for the first occurance of >> meta/instanceID in the form and uses that value as the unique identifier >> for the submission. If a filled-in form has a unique identifier, Aggregate >> can merge the multiple uploads and prevent duplications. >> >> Mitch >> >> On Mon, May 14, 2012 at 9:23 AM, Jon Parsons wrote: >> >>> Hi >>> >>> We have used briefcase in the field in Guyana, to get around internet >>> connection problems, as the phones just would not send data direct through >>> wifi and gave us all sorts of error and connection problems. BTW We only >>> had a 512kbs down and 256kbs up data rate through a satelite that was >>> giving us an actual 200kbs down and 50kbs up rate. >>> >>> But I have two questions about briefcase and its use >>> - We have noticed that when briefcase pulls data from the phone, the >>> data still there in the send finalised form area, so if the phone is then >>> finally connects to good internet, these forms will be submitted and create >>> duplicates. >>> - Also when the data is collected together in briefcase and then sent as >>> a batch (we did 15 submissions) and it says sending 1 of 15 etc - if this >>> fails at 10 of 15 due to poor internet, if you re action the send request >>> it sends all the submissions again and then creates duplicates in the >>> appspot. >>> >>> Is there any way to get around these problems or is there a way to >>> manage the duplicates better in appspot. >>> >>> Thanks >>> >>> Jon >>> >>> -- >>> Post: opend...@googlegroups.com >>> Unsubscribe: opendatakit...@googlegroups.com >>> Options: http://groups.google.com/group/opendatakit?hl=en >>> >> >> >> >> -- >> Mitch Sundt >> Software Engineer >> University of Washington >> mitche...@gmail.com >> >
6

Hello all,
Anyone for triplicates? I am getting a little lost here, and am not sure if
my duplicates from briefcase are happening in the same context. I created a
form with a repeating group. I use that repeat group each time I want to
save an event"s data. For example, the form starts with the person's name,
etc, but the repeating group gets his/her blood pressure and weight. I save
the form with the person's name. A month later, I can open that form --
because I named it with the person's name, jump to the repeating group and
add another event (blood pressure and weight). The problem is, when I do a
pull and export to csv, the first time it is OK, records look great, but
the second time I do a pull & export (in the meantime I added more events
to a person's form), I see duplicates (and even triplicates and
quadruplets) in both the main and child csv files. I know that repeat
groups were not designed as a 1-n DB child table, but I would think I could
use it for that (Imagine getting three samples per day of something from 10
people over a period of 10 days) -- the repeat groups would act as the
sample event data capture. So what am I doing wrong to get duplicates and
triplicates?
Curtis

··· Le lundi 14 mai 2012 19:52:57 UTC+2, OutSourceRox a écrit : > > Hey Mitch, > Thanks for the tip. Jon, I guess we should add that field into the forms > soon as possible, just as precaution to prevent a similar incident which > Ricky (upload over what was already uploaded). > > On Mon, May 14, 2012 at 1:36 PM, Mitch S <mitche...@gmail.com wrote: > >> Briefcase DOES NOT delete any data off your phone. It is up to you, >> after using briefcase, to select the forms on the phone and delete them. >> Additionally, Briefcase does not attempt to find only the finalized forms >> on your phone -- it will grab everything. So you should first finalize the >> forms on the phone; if that isn't possible, you can go into Briefcase's >> directory tree and delete the incomplete submissions from there, if needed >> (they will generally have the same directory name at this point). >> >> As for duplicates, you should define an instanceID in your form >> definition. >> >> > xmlns:h="http://www.w3.org/1999/xhtml" >> xmlns:ev="http://www.w3.org/2001/xml-events" >> xmlns:xsd="http://www.w3.org/2001/XMLSchema" >> xmlns:jr="http://openrosa.org/javarosa" >> xmlns:orx="http://openrosa.org/xforms/" >> > >> >> My Data >> >> >> >> >> >> >> ... >> >> > readonly="true()" calculate="concat('uuid:', uuid())"/> >> ... >> >> You can omit the orx: namespace prefix, but the above follows the >> OpenRosa metadata standard ( >> https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ). >> Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or >> a blank (default) namespace and looks for the first occurance of >> meta/instanceID in the form and uses that value as the unique identifier >> for the submission. If a filled-in form has a unique identifier, Aggregate >> can merge the multiple uploads and prevent duplications. >> >> Mitch >> >> >> On Mon, May 14, 2012 at 9:23 AM, Jon Parsons <j.pa...@globalcanopy.org wrote: >> >>> Hi >>> >>> We have used briefcase in the field in Guyana, to get around internet >>> connection problems, as the phones just would not send data direct through >>> wifi and gave us all sorts of error and connection problems. BTW We only >>> had a 512kbs down and 256kbs up data rate through a satelite that was >>> giving us an actual 200kbs down and 50kbs up rate. >>> >>> But I have two questions about briefcase and its use >>> - We have noticed that when briefcase pulls data from the phone, the >>> data still there in the send finalised form area, so if the phone is then >>> finally connects to good internet, these forms will be submitted and create >>> duplicates. >>> - Also when the data is collected together in briefcase and then sent as >>> a batch (we did 15 submissions) and it says sending 1 of 15 etc - if this >>> fails at 10 of 15 due to poor internet, if you re action the send request >>> it sends all the submissions again and then creates duplicates in the >>> appspot. >>> >>> Is there any way to get around these problems or is there a way to >>> manage the duplicates better in appspot. >>> >>> Thanks >>> >>> Jon >>> >>> -- >>> Post: opend...@googlegroups.com >>> Unsubscribe: opendatakit...@googlegroups.com >>> Options: http://groups.google.com/group/opendatakit?hl=en >>> >> >> >> >> -- >> Mitch Sundt >> Software Engineer >> University of Washington >> mitche...@gmail.com >> >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> > > > > -- > *Play it forward and Have a good one!* > --- > "Like a camera I use the negative to develop..uDig" > "Success is the ability to go from one failure to another with no loss of > enthusiasm" > > ----------------------------------------------------------------------------------------------------------> > *OutSourceRox** > * > *Roxroy K. Bollers* > GIS and IT Consultant > tel. 592.685.2737 > *.........................................................* > *Follow Us On* > *Facebook * > Google+ > *Web * > >
7

It is like as shown here: http://opendatakit.org/help/form-design/ These
fields are placed within the first block in your file, which
holds the data that is transmitted up to ODK Aggregate.

Adding this to one of your existing forms would create an incompatible
change and you would need to revise the form id (the id="mydata" attribute
on the top-level element of the first in your file). This form
id does not need to look anything like the title. Build, for example, uses
something like id="build_Untitled-Form_1341338353" for this id.

··· On Fri, Aug 17, 2012 at 5:10 PM, DataMax wrote:

Mitch, I just saw this and I know it might be a dumb question, but I
already have an section in my xml file. Do I add another
so I put what you mentioned below or do I add that block to the
current section? Also, for , does the label need to be similar to the title "My Data" in my form
which has
<Price_Tracker id="Price_Tracker">??
I'm just not sure...if you could clarify. I can see myself messing up
somewhere. :slight_smile: Thanks.

DataMax

On Monday, May 14, 2012 8:36:21 PM UTC+3, Mitch wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you,
after using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form
definition.

<h:html xmlns="http://www.w3.org/2002/**xformshttp://www.w3.org/2002/xforms
"
xmlns:h="http://www.w3.org/**1999/xhtmlhttp://www.w3.org/1999/xhtml
"
xmlns:ev="http://www.w3.org/**2001/xml-eventshttp://www.w3.org/2001/xml-events
"
xmlns:xsd="http://www.w3.org/**2001/XMLSchemahttp://www.w3.org/2001/XMLSchema
"
xmlns:jr="http://openrosa.org/**javarosahttp://openrosa.org/javarosa
"
xmlns:orx="http://openrosa.**org/xforms/http://openrosa.org/xforms/
"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the
OpenRosa metadata standard ( https://bitbucket.org/**
javarosa/javarosa/wiki/**OpenRosaMetaDataSchemahttps://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema). Aggregate recognizes the
http://openrosa.org/xforms/ (orx) namespace or a blank (default)
namespace and looks for the first occurance of meta/instanceID in the form
and uses that value as the unique identifier for the submission. If a
filled-in form has a unique identifier, Aggregate can merge the multiple
uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons j.pa...@globalcanopy.orgwrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the
    data still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent as
    a batch (we did 15 submissions) and it says sending 1 of 15 etc - if this
    fails at 10 of 15 due to poor internet, if you re action the send request
    it sends all the submissions again and then creates duplicates in the
    appspot.

Is there any way to get around these problems or is there a way to
manage the duplicates better in appspot.

Thanks

Jon

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

8

Hi Jon,
The meta element and instanceId is automatically included in all xforms
generated by the ODK XLSForm. You can specify an id attribute (e.g. ) by adding a form_id column to the settings sheet of the
XLSForm.
Regards,
-Nathan

··· On Wednesday, October 3, 2012 4:02:43 AM UTC-7, Jon Parsons wrote: > > I am creating new forms using XLSForm and > would like to incorporate the code you suggest to help with duplicates, but > I am not sure how to incorporate this code into the excel document > > could you advise > > Thanks > > > > On Wednesday, 3 October 2012 11:59:17 UTC+1, Jon Parsons wrote: >> >> Hi Mitch >> >> apologies - I previously posted a reply to this post while logged in with >> another google account (g.lea) could you ignore/delete these posts >> >> Jon >> >> On Monday, 14 May 2012 18:36:21 UTC+1, Mitch wrote: >>> >>> Briefcase DOES NOT delete any data off your phone. It is up to you, >>> after using briefcase, to select the forms on the phone and delete them. >>> Additionally, Briefcase does not attempt to find only the finalized forms >>> on your phone -- it will grab everything. So you should first finalize the >>> forms on the phone; if that isn't possible, you can go into Briefcase's >>> directory tree and delete the incomplete submissions from there, if needed >>> (they will generally have the same directory name at this point). >>> >>> As for duplicates, you should define an instanceID in your form >>> definition. >>> >>> >> xmlns:h="http://www.w3.org/1999/xhtml" >>> xmlns:ev="http://www.w3.org/2001/xml-events" >>> xmlns:xsd="http://www.w3.org/2001/XMLSchema" >>> xmlns:jr="http://openrosa.org/javarosa" >>> xmlns:orx="http://openrosa.org/xforms/" >>> > >>> >>> My Data >>> >>> >>> >>> >>> >>> >>> ... >>> >>> >> readonly="true()" calculate="concat('uuid:', uuid())"/> >>> ... >>> >>> You can omit the orx: namespace prefix, but the above follows the >>> OpenRosa metadata standard ( >>> https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema ). >>> Aggregate recognizes the http://openrosa.org/xforms/ (orx) namespace or >>> a blank (default) namespace and looks for the first occurance of >>> meta/instanceID in the form and uses that value as the unique identifier >>> for the submission. If a filled-in form has a unique identifier, Aggregate >>> can merge the multiple uploads and prevent duplications. >>> >>> Mitch >>> >>> On Mon, May 14, 2012 at 9:23 AM, Jon Parsons wrote: >>> >>>> Hi >>>> >>>> We have used briefcase in the field in Guyana, to get around internet >>>> connection problems, as the phones just would not send data direct through >>>> wifi and gave us all sorts of error and connection problems. BTW We only >>>> had a 512kbs down and 256kbs up data rate through a satelite that was >>>> giving us an actual 200kbs down and 50kbs up rate. >>>> >>>> But I have two questions about briefcase and its use >>>> - We have noticed that when briefcase pulls data from the phone, the >>>> data still there in the send finalised form area, so if the phone is then >>>> finally connects to good internet, these forms will be submitted and create >>>> duplicates. >>>> - Also when the data is collected together in briefcase and then sent >>>> as a batch (we did 15 submissions) and it says sending 1 of 15 etc - if >>>> this fails at 10 of 15 due to poor internet, if you re action the send >>>> request it sends all the submissions again and then creates duplicates in >>>> the appspot. >>>> >>>> Is there any way to get around these problems or is there a way to >>>> manage the duplicates better in appspot. >>>> >>>> Thanks >>>> >>>> Jon >>>> >>>> -- >>>> Post: opend...@googlegroups.com >>>> Unsubscribe: opendatakit...@googlegroups.com >>>> Options: http://groups.google.com/group/opendatakit?hl=en >>>> >>> >>> >>> >>> -- >>> Mitch Sundt >>> Software Engineer >>> University of Washington >>> mitche...@gmail.com >>> >>
9

It is likely that you have multiple copies of the original form in the ODK
Briefcase storage area. Try downloading the data to a new ODK Briefcase
storage location, rather than reusing an existing location. Do you still
see duplicates?

If you do still see duplicates, is the PARENT_KEY field the same for these
rows, or is it different. If it is different, this would indicate that you
have two different submissions contributing to the apparent duplicates.

Note that after pulling data from ODK Collect, the data remains on the ODK
Collect device, and you need to manually delete it via the Delete Saved
Forms screen. If you have entries with different PARENT_KEY values, it is
likely because you never deleted the earlier submissions.

Note that the instanceID does not prevent duplicate forms from co-existing
in ODK Briefcase. ODK Briefcase does not do the de-duplication processing
that ODK Aggregate does. De-duplication is currently handled in ODK
Aggregate only.

ODK Briefcase pulls data from data sources, and the last data source it
pulls from either "wins" (for ODK Aggregate) or "coexists" (for ODK
Collect). Once the data is pulled from ODK Collect, it is important that
you then go to the Delete Saved Forms screen and delete them from the
phone, otherwise, on the next pull, you will get a second copy of those
forms, plus an initial copy of any new form.

Additional inspection and comparison code would have to be added to enable
ODK Briefcase to decide whether two copies of a submission copied from ODK
Collect are identical. If two or more filled-in forms collide, they will
coexist in the Briefcase storage location as
.../instances/collect-directory-name/...
.../instances/collect-directory-name-2/...
etc.

You would need to decide whether these are actual duplicates or simply due
to two or more phones happening to pick the same collect-directory-name for
their different submissions. This is where the additional inspection and
comparision code would need to be written. In this case, since the
filled-in form reflects updated values, I would expect that this inspection
and comparison code would identify them as different and would still decide
to keep both.

··· ----

Mitch

On Fri, Aug 17, 2012 at 5:36 AM, Curtis Broderick <c.broderick@epiconcept.fr wrote:

Hello all,
Anyone for triplicates? I am getting a little lost here, and am not sure
if my duplicates from briefcase are happening in the same context. I
created a form with a repeating group. I use that repeat group each time I
want to save an event"s data. For example, the form starts with the
person's name, etc, but the repeating group gets his/her blood pressure and
weight. I save the form with the person's name. A month later, I can open
that form -- because I named it with the person's name, jump to the
repeating group and add another event (blood pressure and weight). The
problem is, when I do a pull and export to csv, the first time it is OK,
records look great, but the second time I do a pull & export (in the
meantime I added more events to a person's form), I see duplicates (and
even triplicates and quadruplets) in both the main and child csv files. I
know that repeat groups were not designed as a 1-n DB child table, but I
would think I could use it for that (Imagine getting three samples per day
of something from 10 people over a period of 10 days) -- the repeat groups
would act as the sample event data capture. So what am I doing wrong to get
duplicates and triplicates?
Curtis

Le lundi 14 mai 2012 19:52:57 UTC+2, OutSourceRox a écrit :

Hey Mitch,
Thanks for the tip. Jon, I guess we should add that field into the forms
soon as possible, just as precaution to prevent a similar incident which
Ricky (upload over what was already uploaded).

On Mon, May 14, 2012 at 1:36 PM, Mitch S mitche...@gmail.com wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you,
after using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form
definition.

<h:html xmlns="http://www.w3.org/2002/**xformshttp://www.w3.org/2002/xforms
"
xmlns:h="http://www.w3.org/**1999/xhtmlhttp://www.w3.org/1999/xhtml
"
xmlns:ev="http://www.w3.org/**2001/xml-eventshttp://www.w3.org/2001/xml-events
"
xmlns:xsd="http://www.w3.org/**2001/XMLSchemahttp://www.w3.org/2001/XMLSchema
"
xmlns:jr="http://openrosa.org/**javarosahttp://openrosa.org/javarosa
"
xmlns:orx="http://openrosa.**org/xforms/http://openrosa.org/xforms/
"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the
OpenRosa metadata standard ( https://bitbucket.org/**
javarosa/javarosa/wiki/**OpenRosaMetaDataSchemahttps://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema). Aggregate recognizes the
http://openrosa.org/xforms/ (orx) namespace or a blank (default)
namespace and looks for the first occurance of meta/instanceID in the form
and uses that value as the unique identifier for the submission. If a
filled-in form has a unique identifier, Aggregate can merge the multiple
uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons j.pa...@globalcanopy.orgwrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the
    data still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent
    as a batch (we did 15 submissions) and it says sending 1 of 15 etc - if
    this fails at 10 of 15 due to poor internet, if you re action the send
    request it sends all the submissions again and then creates duplicates in
    the appspot.

Is there any way to get around these problems or is there a way to
manage the duplicates better in appspot.

Thanks

Jon

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss of
enthusiasm"
------------------------------------------------------------
------------------------------*---------------->
OutSourceRox
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

10

Ah yeah, makes sense. Thanks!

Try downloading the data to a new ODK Briefcase storage location, rather
than reusing an existing location. Do you still see duplicates?

Nope ; )

··· 2012/8/17 Mitch S
1 Like
11

Mitch,
Thanks again for your response. However, you did not comment, and I
wouldn't mind hearing from Gaetano either, about using repeating groups as
a sort of 1-N table where the questionnaire is the 1 and the repeating
group is the N table. It seems like a good idea and a bad idea at the same
time. A way to use ODK for research where you go back to the "thing" n
number of times to take measurements. Your thoughts?
Curtis

··· 2012/8/17 Mitch S

It is likely that you have multiple copies of the original form in the ODK
Briefcase storage area. Try downloading the data to a new ODK Briefcase
storage location, rather than reusing an existing location. Do you still
see duplicates?

If you do still see duplicates, is the PARENT_KEY field the same for these
rows, or is it different. If it is different, this would indicate that you
have two different submissions contributing to the apparent duplicates.

Note that after pulling data from ODK Collect, the data remains on the ODK
Collect device, and you need to manually delete it via the Delete Saved
Forms screen. If you have entries with different PARENT_KEY values, it is
likely because you never deleted the earlier submissions.

Note that the instanceID does not prevent duplicate forms from co-existing
in ODK Briefcase. ODK Briefcase does not do the de-duplication processing
that ODK Aggregate does. De-duplication is currently handled in ODK
Aggregate only.

ODK Briefcase pulls data from data sources, and the last data source it
pulls from either "wins" (for ODK Aggregate) or "coexists" (for ODK
Collect). Once the data is pulled from ODK Collect, it is important that
you then go to the Delete Saved Forms screen and delete them from the
phone, otherwise, on the next pull, you will get a second copy of those
forms, plus an initial copy of any new form.

Additional inspection and comparison code would have to be added to enable
ODK Briefcase to decide whether two copies of a submission copied from ODK
Collect are identical. If two or more filled-in forms collide, they will
coexist in the Briefcase storage location as
.../instances/collect-directory-name/...
.../instances/collect-directory-name-2/...
etc.

You would need to decide whether these are actual duplicates or simply due
to two or more phones happening to pick the same collect-directory-name for
their different submissions. This is where the additional inspection and
comparision code would need to be written. In this case, since the
filled-in form reflects updated values, I would expect that this inspection
and comparison code would identify them as different and would still decide
to keep both.

Mitch

On Fri, Aug 17, 2012 at 5:36 AM, Curtis Broderick < c.broderick@epiconcept.fr> wrote:

Hello all,
Anyone for triplicates? I am getting a little lost here, and am not sure
if my duplicates from briefcase are happening in the same context. I
created a form with a repeating group. I use that repeat group each time I
want to save an event"s data. For example, the form starts with the
person's name, etc, but the repeating group gets his/her blood pressure and
weight. I save the form with the person's name. A month later, I can open
that form -- because I named it with the person's name, jump to the
repeating group and add another event (blood pressure and weight). The
problem is, when I do a pull and export to csv, the first time it is OK,
records look great, but the second time I do a pull & export (in the
meantime I added more events to a person's form), I see duplicates (and
even triplicates and quadruplets) in both the main and child csv files. I
know that repeat groups were not designed as a 1-n DB child table, but I
would think I could use it for that (Imagine getting three samples per day
of something from 10 people over a period of 10 days) -- the repeat groups
would act as the sample event data capture. So what am I doing wrong to get
duplicates and triplicates?
Curtis

Le lundi 14 mai 2012 19:52:57 UTC+2, OutSourceRox a écrit :

Hey Mitch,
Thanks for the tip. Jon, I guess we should add that field into the forms
soon as possible, just as precaution to prevent a similar incident which
Ricky (upload over what was already uploaded).

On Mon, May 14, 2012 at 1:36 PM, Mitch S mitche...@gmail.com wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you,
after using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form
definition.

<h:html xmlns="http://www.w3.org/2002/**xformshttp://www.w3.org/2002/xforms
"
xmlns:h="http://www.w3.org/**1999/xhtmlhttp://www.w3.org/1999/xhtml
"
xmlns:ev="http://www.w3.org/**2001/xml-eventshttp://www.w3.org/2001/xml-events
"
xmlns:xsd="http://www.w3.org/**2001/XMLSchemahttp://www.w3.org/2001/XMLSchema
"
xmlns:jr="http://openrosa.org/**javarosahttp://openrosa.org/javarosa
"
xmlns:orx="http://openrosa.**org/xforms/http://openrosa.org/xforms/
"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the
OpenRosa metadata standard ( https://bitbucket.org/**
javarosa/javarosa/wiki/**OpenRosaMetaDataSchemahttps://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema). Aggregate recognizes the
http://openrosa.org/xforms/ (orx) namespace or a blank (default)
namespace and looks for the first occurance of meta/instanceID in the form
and uses that value as the unique identifier for the submission. If a
filled-in form has a unique identifier, Aggregate can merge the multiple
uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons j.pa...@globalcanopy.orgwrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the
    data still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent
    as a batch (we did 15 submissions) and it says sending 1 of 15 etc - if
    this fails at 10 of 15 due to poor internet, if you re action the send
    request it sends all the submissions again and then creates duplicates in
    the appspot.

Is there any way to get around these problems or is there a way to
manage the duplicates better in appspot.

Thanks

Jon

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss
of enthusiasm"
------------------------------------------------------------
------------------------------*---------------->
OutSourceRox
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--

Curtis BRODERICK
Head of training and user support
Tél : + 33(0)1.53.02.40.61 06.15.33.57.37

---------------------------EpiConcept---------------------------
Systèmes d'Information en Santé
Health Information System
47, rue de Charenton - 75012 PARIS
Tel : +33 (0)1 53 02 40 60 Fax : +33 (0)1 53 02 40 62
-----------------------www.epiconcept.fr------------------------

Ce message et toutes les pièces jointes sont établis à l'intention
exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur, merci de le détruire et d'en
avertir l'expéditeur.

The information contained in the Email and any attachments is
confidential and intended solely and for the attention and use of the
named address(es). It may not be disclosed to any other person without
the express authority of EpiConcept, or the intended recipient, or
both.
If you are not the intended recipient, you must not disclose, copy,
distribute or retain this message or any part of it.

12

We are working on such a capability for ODK Collect 2.0. The next major
release - a couple of months away from alpha. The idea would be that when
you reach a repeated section, it would be a completely separate subform and
you would have the opportunity to start a new instance of that subform or
edit/delete a previously completed instance. It would have a UI that would
present a list of previously entered subform instances and "add a new
instance". We are planning on eliminating the concept of a "repeat group"
as it exists in javarosa. You may want to be in our alpha testers?

··· On Wed, Aug 22, 2012 at 12:57 AM, Curtis Broderick < c.broderick@epiconcept.fr> wrote:

Mitch,
Thanks again for your response. However, you did not comment, and I
wouldn't mind hearing from Gaetano either, about using repeating groups as
a sort of 1-N table where the questionnaire is the 1 and the repeating
group is the N table. It seems like a good idea and a bad idea at the same
time. A way to use ODK for research where you go back to the "thing" n
number of times to take measurements. Your thoughts?
Curtis

2012/8/17 Mitch S mitchellsundt@gmail.com

It is likely that you have multiple copies of the original form in the
ODK Briefcase storage area. Try downloading the data to a new ODK Briefcase
storage location, rather than reusing an existing location. Do you still
see duplicates?

If you do still see duplicates, is the PARENT_KEY field the same for
these rows, or is it different. If it is different, this would indicate
that you have two different submissions contributing to the apparent
duplicates.

Note that after pulling data from ODK Collect, the data remains on the
ODK Collect device, and you need to manually delete it via the Delete Saved
Forms screen. If you have entries with different PARENT_KEY values, it is
likely because you never deleted the earlier submissions.

Note that the instanceID does not prevent duplicate forms from
co-existing in ODK Briefcase. ODK Briefcase does not do the de-duplication
processing that ODK Aggregate does. De-duplication is currently handled in
ODK Aggregate only.

ODK Briefcase pulls data from data sources, and the last data source it
pulls from either "wins" (for ODK Aggregate) or "coexists" (for ODK
Collect). Once the data is pulled from ODK Collect, it is important that
you then go to the Delete Saved Forms screen and delete them from the
phone, otherwise, on the next pull, you will get a second copy of those
forms, plus an initial copy of any new form.

Additional inspection and comparison code would have to be added to
enable ODK Briefcase to decide whether two copies of a submission copied
from ODK Collect are identical. If two or more filled-in forms collide,
they will coexist in the Briefcase storage location as
.../instances/collect-directory-name/...
.../instances/collect-directory-name-2/...
etc.

You would need to decide whether these are actual duplicates or simply
due to two or more phones happening to pick the same collect-directory-name
for their different submissions. This is where the additional inspection
and comparision code would need to be written. In this case, since the
filled-in form reflects updated values, I would expect that this inspection
and comparison code would identify them as different and would still decide
to keep both.

Mitch

On Fri, Aug 17, 2012 at 5:36 AM, Curtis Broderick < c.broderick@epiconcept.fr> wrote:

Hello all,
Anyone for triplicates? I am getting a little lost here, and am not sure
if my duplicates from briefcase are happening in the same context. I
created a form with a repeating group. I use that repeat group each time I
want to save an event"s data. For example, the form starts with the
person's name, etc, but the repeating group gets his/her blood pressure and
weight. I save the form with the person's name. A month later, I can open
that form -- because I named it with the person's name, jump to the
repeating group and add another event (blood pressure and weight). The
problem is, when I do a pull and export to csv, the first time it is OK,
records look great, but the second time I do a pull & export (in the
meantime I added more events to a person's form), I see duplicates (and
even triplicates and quadruplets) in both the main and child csv files. I
know that repeat groups were not designed as a 1-n DB child table, but I
would think I could use it for that (Imagine getting three samples per day
of something from 10 people over a period of 10 days) -- the repeat groups
would act as the sample event data capture. So what am I doing wrong to get
duplicates and triplicates?
Curtis

Le lundi 14 mai 2012 19:52:57 UTC+2, OutSourceRox a écrit :

Hey Mitch,
Thanks for the tip. Jon, I guess we should add that field into the
forms soon as possible, just as precaution to prevent a similar incident
which Ricky (upload over what was already uploaded).

On Mon, May 14, 2012 at 1:36 PM, Mitch S mitche...@gmail.com wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you,
after using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form
definition.

<h:html xmlns="http://www.w3.org/2002/**xformshttp://www.w3.org/2002/xforms
"
xmlns:h="http://www.w3.org/**1999/xhtmlhttp://www.w3.org/1999/xhtml
"
xmlns:ev="http://www.w3.org/**2001/xml-eventshttp://www.w3.org/2001/xml-events
"
xmlns:xsd="http://www.w3.org/**2001/XMLSchemahttp://www.w3.org/2001/XMLSchema
"
xmlns:jr="http://openrosa.org/**javarosahttp://openrosa.org/javarosa
"
xmlns:orx="http://openrosa.**org/xforms/http://openrosa.org/xforms/
"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the
OpenRosa metadata standard ( https://bitbucket.org/**
javarosa/javarosa/wiki/**OpenRosaMetaDataSchemahttps://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema). Aggregate recognizes the
http://openrosa.org/xforms/ (orx) namespace or a blank (default)
namespace and looks for the first occurance of meta/instanceID in the form
and uses that value as the unique identifier for the submission. If a
filled-in form has a unique identifier, Aggregate can merge the multiple
uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons <j.pa...@globalcanopy.org wrote:

Hi

We have used briefcase in the field in Guyana, to get around internet
connection problems, as the phones just would not send data direct through
wifi and gave us all sorts of error and connection problems. BTW We only
had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the
    data still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then sent
    as a batch (we did 15 submissions) and it says sending 1 of 15 etc - if
    this fails at 10 of 15 due to poor internet, if you re action the send
    request it sends all the submissions again and then creates duplicates in
    the appspot.

Is there any way to get around these problems or is there a way to
manage the duplicates better in appspot.

Thanks

Jon

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss
of enthusiasm"
------------------------------------------------------------
------------------------------*---------------->
OutSourceRox
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--

Curtis BRODERICK
Head of training and user support

Tél : + 33(0)1.53.02.40.61 06.15.33.57.37

---------------------------EpiConcept---------------------------
Systèmes d'Information en Santé
Health Information System

           47, rue de Charenton - 75012 PARIS
     Tel : +33 (0)1 53 02 40 60  Fax : +33 (0)1 53 02 40 62

-----------------------www.epiconcept.fr------------------------

Ce message et toutes les pièces jointes sont établis à l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur, merci de le détruire et d'en avertir l'expéditeur.

The information contained in the Email and any attachments is confidential and intended solely and for the attention and use of the
named address(es). It may not be disclosed to any other person without the express authority of EpiConcept, or the intended recipient, or both.

If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it.

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

13

Hello Gaetano,
How do you get in or be part of the alpha testers?

The ODK Collect 2.0, seem like it will be exactly what we would need for
our project as it becomes more mature and other levels of data collecting
is being put forward by the crew members and team.

Rox.

··· On Wed, Aug 22, 2012 at 3:46 AM, Gaetano Borriello < gaetano@cs.washington.edu> wrote:

We are working on such a capability for ODK Collect 2.0. The next major
release - a couple of months away from alpha. The idea would be that when
you reach a repeated section, it would be a completely separate subform and
you would have the opportunity to start a new instance of that subform or
edit/delete a previously completed instance. It would have a UI that would
present a list of previously entered subform instances and "add a new
instance". We are planning on eliminating the concept of a "repeat group"
as it exists in javarosa. You may want to be in our alpha testers?

On Wed, Aug 22, 2012 at 12:57 AM, Curtis Broderick < c.broderick@epiconcept.fr> wrote:

Mitch,
Thanks again for your response. However, you did not comment, and I
wouldn't mind hearing from Gaetano either, about using repeating groups as
a sort of 1-N table where the questionnaire is the 1 and the repeating
group is the N table. It seems like a good idea and a bad idea at the same
time. A way to use ODK for research where you go back to the "thing" n
number of times to take measurements. Your thoughts?
Curtis

2012/8/17 Mitch S mitchellsundt@gmail.com

It is likely that you have multiple copies of the original form in the
ODK Briefcase storage area. Try downloading the data to a new ODK Briefcase
storage location, rather than reusing an existing location. Do you still
see duplicates?

If you do still see duplicates, is the PARENT_KEY field the same for
these rows, or is it different. If it is different, this would indicate
that you have two different submissions contributing to the apparent
duplicates.

Note that after pulling data from ODK Collect, the data remains on the
ODK Collect device, and you need to manually delete it via the Delete Saved
Forms screen. If you have entries with different PARENT_KEY values, it is
likely because you never deleted the earlier submissions.

Note that the instanceID does not prevent duplicate forms from
co-existing in ODK Briefcase. ODK Briefcase does not do the de-duplication
processing that ODK Aggregate does. De-duplication is currently handled in
ODK Aggregate only.

ODK Briefcase pulls data from data sources, and the last data source it
pulls from either "wins" (for ODK Aggregate) or "coexists" (for ODK
Collect). Once the data is pulled from ODK Collect, it is important that
you then go to the Delete Saved Forms screen and delete them from the
phone, otherwise, on the next pull, you will get a second copy of those
forms, plus an initial copy of any new form.

Additional inspection and comparison code would have to be added to
enable ODK Briefcase to decide whether two copies of a submission copied
from ODK Collect are identical. If two or more filled-in forms collide,
they will coexist in the Briefcase storage location as
.../instances/collect-directory-name/...
.../instances/collect-directory-name-2/...
etc.

You would need to decide whether these are actual duplicates or simply
due to two or more phones happening to pick the same collect-directory-name
for their different submissions. This is where the additional inspection
and comparision code would need to be written. In this case, since the
filled-in form reflects updated values, I would expect that this inspection
and comparison code would identify them as different and would still decide
to keep both.

Mitch

On Fri, Aug 17, 2012 at 5:36 AM, Curtis Broderick < c.broderick@epiconcept.fr> wrote:

Hello all,
Anyone for triplicates? I am getting a little lost here, and am not
sure if my duplicates from briefcase are happening in the same context. I
created a form with a repeating group. I use that repeat group each time I
want to save an event"s data. For example, the form starts with the
person's name, etc, but the repeating group gets his/her blood pressure and
weight. I save the form with the person's name. A month later, I can open
that form -- because I named it with the person's name, jump to the
repeating group and add another event (blood pressure and weight). The
problem is, when I do a pull and export to csv, the first time it is OK,
records look great, but the second time I do a pull & export (in the
meantime I added more events to a person's form), I see duplicates (and
even triplicates and quadruplets) in both the main and child csv files. I
know that repeat groups were not designed as a 1-n DB child table, but I
would think I could use it for that (Imagine getting three samples per day
of something from 10 people over a period of 10 days) -- the repeat groups
would act as the sample event data capture. So what am I doing wrong to get
duplicates and triplicates?
Curtis

Le lundi 14 mai 2012 19:52:57 UTC+2, OutSourceRox a écrit :

Hey Mitch,
Thanks for the tip. Jon, I guess we should add that field into the
forms soon as possible, just as precaution to prevent a similar incident
which Ricky (upload over what was already uploaded).

On Mon, May 14, 2012 at 1:36 PM, Mitch S mitche...@gmail.com wrote:

Briefcase DOES NOT delete any data off your phone. It is up to you,
after using briefcase, to select the forms on the phone and delete them.
Additionally, Briefcase does not attempt to find only the finalized forms
on your phone -- it will grab everything. So you should first finalize the
forms on the phone; if that isn't possible, you can go into Briefcase's
directory tree and delete the incomplete submissions from there, if needed
(they will generally have the same directory name at this point).

As for duplicates, you should define an instanceID in your form
definition.

<h:html xmlns="http://www.w3.org/2002/**xformshttp://www.w3.org/2002/xforms
"
xmlns:h="http://www.w3.org/**1999/xhtmlhttp://www.w3.org/1999/xhtml
"
xmlns:ev="http://www.w3.org/**2001/xml-eventshttp://www.w3.org/2001/xml-events
"
xmlns:xsd="http://www.w3.org/**2001/XMLSchemahttp://www.w3.org/2001/XMLSchema
"
xmlns:jr="http://openrosa.org/**javarosahttp://openrosa.org/javarosa
"
xmlns:orx="http://openrosa.**org/xforms/http://openrosa.org/xforms/
"
>
<h:head>
<h:title>My Data</h:title>



orx:meta
orx:instanceID/
</orx:meta>
...

       <bind nodeset="/nm/meta/instanceID" type="string"

readonly="true()" calculate="concat('uuid:', uuid())"/>
...

You can omit the orx: namespace prefix, but the above follows the
OpenRosa metadata standard ( https://bitbucket.org/**
javarosa/javarosa/wiki/**OpenRosaMetaDataSchemahttps://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema). Aggregate recognizes the
http://openrosa.org/xforms/ (orx) namespace or a blank (default)
namespace and looks for the first occurance of meta/instanceID in the form
and uses that value as the unique identifier for the submission. If a
filled-in form has a unique identifier, Aggregate can merge the multiple
uploads and prevent duplications.

Mitch

On Mon, May 14, 2012 at 9:23 AM, Jon Parsons < j.pa...@globalcanopy.org> wrote:

Hi

We have used briefcase in the field in Guyana, to get around
internet connection problems, as the phones just would not send data direct
through wifi and gave us all sorts of error and connection problems. BTW We
only had a 512kbs down and 256kbs up data rate through a satelite that was
giving us an actual 200kbs down and 50kbs up rate.

But I have two questions about briefcase and its use

  • We have noticed that when briefcase pulls data from the phone, the
    data still there in the send finalised form area, so if the phone is then
    finally connects to good internet, these forms will be submitted and create
    duplicates.
  • Also when the data is collected together in briefcase and then
    sent as a batch (we did 15 submissions) and it says sending 1 of 15 etc -
    if this fails at 10 of 15 due to poor internet, if you re action the send
    request it sends all the submissions again and then creates duplicates in
    the appspot.

Is there any way to get around these problems or is there a way to
manage the duplicates better in appspot.

Thanks

Jon

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--
Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@**googlegroups.com
Options: http://groups.google.com/**group/opendatakit?hl=enhttp://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss
of enthusiasm"
------------------------------------------------------------
------------------------------*---------------->
OutSourceRox
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--

Curtis BRODERICK
Head of training and user support

Tél : + 33(0)1.53.02.40.61 06.15.33.57.37

---------------------------EpiConcept---------------------------
Systèmes d'Information en Santé

                Health Information System

           47, rue de Charenton - 75012 PARIS
     Tel : +33 (0)1 53 02 40 60  Fax : +33 (0)1 53 02 40 62

-----------------------www.epiconcept.fr------------------------

Ce message et toutes les pièces jointes sont établis à l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur, merci de le détruire et d'en avertir l'expéditeur.

The information contained in the Email and any attachments is confidential and intended solely and for the attention and use of the
named address(es). It may not be disclosed to any other person without the express authority of EpiConcept, or the intended recipient, or both.

If you are not the intended recipient, you must not disclose, copy, distribute or retain this message or any part of it.

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Play it forward and Have a good one!

"Like a camera I use the negative to develop..uDig"
"Success is the ability to go from one failure to another with no loss of
enthusiasm"
---------------------------------------------------------------------------------------------------------->
OutSourceRox*
*
Roxroy K. Bollers
GIS and IT Consultant
tel. 592.685.2737
.........................................................
Follow Us On
Facebook http://www.facebook.com/OutSourceRox
Google+ https://plus.google.com/114064153117817313312
Web http://twitinternational.com/outsourcerox.htm

Portfolio: