Assigning or requiring the assignment of sequential numbers at the time
of data collection is a Really Bad Idea.
At the time you assign the value, it requires global knowledge of the
state of the system. As Waylon describes, this involves global locks on
shared databases, connectivity with the central server at the time the
value is selected, and this all impacts performance.
The solution is to have a partitioned value, where you construct a string
or number by concatenating two parts, e.g., a counter (1, 2, 3...) and a
namespace identifier, such as a deviceId, email, username, simserial or
other value that is unique across all your data collection devices (
http://opendatakit.org/help/form-design/examples/#Property_values ).
This enables the counter to be managed independently (on a per-device
basis), as the namespace identifier ensures that the combined value will
never collide with any other value in the system.
This, however, introduces gaps in your submission sequence number, e.g.,
if you have deviceID 'a21b21aa' and deviceID 'b2131132' so you would have
separate submission streams with (counter-deviceID, you would have ids of,
e.g.,
1-a21b21aa, 2-a21b21aa, 3-a21b21aa, 4-a21b21aa...
1-b2131132, 2-b2131132, 3-b2131132, 4-b2131132...
As an example of creating a sequential ordering out of such string-valued
sequences with gaps, within ODK Aggregate, once the data has been uploaded,
a static sequential order (1,2,3,...) is determined by first sorting the
data by its meta-date-marked-as-complete (ascending in time) and then by
meta-instance-id (ascending in string value). Because Google's data
store is only 'eventually consistent', we maintain a 3-second 'settling
period' between the time a submission is inserted into the server and the
time it is added to this ordering. This gives Google 3 seconds to make the
data we've requested to be stored become 'eventually consistent'.
This ordering is used when we publish data, e.g., to Google Spreadsheet.
The data in row 2 of the spreadsheet is always exactly the same data, each
time you publish all your accumulated data.
Sequential ordering is easy to achieve AFTER data collection because
the data is all centralized -- you no longer have a highly distributed
system.
Mitch
On Tue, Jun 4, 2013 at 4:09 AM, W. Brunette wbrunette@gmail.com wrote:
The sequence the submissions were received can already be determined by
the submission time.
With regards to adding a sequence number, yes you can do this but it will
require code changes....
This is actually more complicated than it sounds as a webserver can have
multiple instances active at any one time. Therefore, you need to save the
sequence number in the database and lock the sequence number so only one
one active session can update the sequence. There are other issues too but
those depend on the platform you are running ODK aggregate on. This will
also make Aggregate more inefficient and slower.
Waylon
On Tue, Jun 4, 2013 at 2:01 PM, Bashir Jahed admin@osilab.net wrote:
As a side note the this, if we wanted to add sequential numbers to
submissions based on sequence forms have been submitted to agregate, would
this be possible?
I.E. Each new record in ODK aggregate gets a sequential number on
submission?
On Thursday, 25 October 2012 07:58:13 UTC+2, Heath Smith wrote:
Hi,
I just started building a form for data collection. I'm most familiar
with using forms in ArcPAD so this is a little new for me. I'm wanting to
generate a sequential number for each new record. Is there an simple way of
having the form add 1 to the previous record? I've done this in ArcPAD but
couldn't find much info here or on simple google searches.
Thanks,
Heath
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.