Xlsform vesion tracking

Does anyone know of any good software packages for tracking form versions?

We are conducting a study in multiple countries, now 4 but soon to be 10. A
major difficulty lies in that we have to move several parallel files
(countries) forward that have some common overlap and some unique features.
We have multiple languages with multiple collaborators in each country
working on the forms. The overlap and the unique parts of the form
continue to be updated.

We have a detailed file naming system but keep imperfect change logs. It
would best to work within an environment designed specifically for tracking
version to help us keep track of branches, changes, etc.

Would anyone recommend github? I haven't used it and am not sure if it is
designed to produce a single software product or several parallel products.
Is there anything better suited for spreadsheets?

Thanks,

Luke

These sorts of things tend to become religious arguments.

You are looking for a 'source code management' system.
http://en.wikipedia.org/wiki/Source_code_management (... which seems to be
somewhat out of date).

The two dominant open-source systems are Git and Mercurial.

  • Git http://git-scm.com/ is extremely powerful. So powerful that if you
    have sets of files that you need to apply different sets of coordinated
    changes to, you will eventually do it incorrectly.
  • Mercurial http://mercurial.selenic.com/ is less powerful and a bit
    easier to understand. ODK uses Mercurial.

Subversion is also popular, as is Visual SourceSafe (Microsoft); both (in
my opinion) due to the extensive body of legacy software that uses them.

All of these systems keep track of every 'checked in' or 'committed' change
to your files (when you reach a point of completion in your work, you
instruct the system to save this copy of your file; that is called
'checking in' the file or 'committing' your changes). You can compare any
two check-ins from any point in time to see what files have changed.

In addition to supplying comments describing your changes at the time you
commit them, you can also 'tag' your checkins with important descriptive
labels (e.g., 'ODK Tables Alpha 1").
You can set up your own server to act as the 'repository of record' for
your projects, but there are several web hosting services that can host
this for you. This is rather nice because you don't have to worry about
your server crashing. The top 4 are:

For an example of Mercurial, see
http://code.google.com/p/opendatakit/source/list?repo=aggregate (for ODK
Aggregate). note that there can be multiple 'branches' in a repository, so
you can have different versions of the same file. A similar view on GitHub
is https://github.com/UW-ICTD/pyxform/commits/master (for the XLSForm
tool).

Mitch

··· On Tue, Jan 21, 2014 at 8:56 AM, Luke MacDonald wrote:

Does anyone know of any good software packages for tracking form versions?

We are conducting a study in multiple countries, now 4 but soon to be
10. A major difficulty lies in that we have to move several parallel files
(countries) forward that have some common overlap and some unique features.
We have multiple languages with multiple collaborators in each country
working on the forms. The overlap and the unique parts of the form
continue to be updated.

We have a detailed file naming system but keep imperfect change logs. It
would best to work within an environment designed specifically for tracking
version to help us keep track of branches, changes, etc.

Would anyone recommend github? I haven't used it and am not sure if it is
designed to produce a single software product or several parallel products.
Is there anything better suited for spreadsheets?

Thanks,

Luke

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Wow, this is pretty awesome help. Gives me some homework to do. Thanks
for the input.

··· On Tuesday, January 21, 2014 7:56:04 PM UTC+3, Luke MacDonald wrote: > > Does anyone know of any good software packages for tracking form versions? > > > We are conducting a study in multiple countries, now 4 but soon to be > 10. A major difficulty lies in that we have to move several parallel files > (countries) forward that have some common overlap and some unique features. > We have multiple languages with multiple collaborators in each country > working on the forms. The overlap and the unique parts of the form > continue to be updated. > > We have a detailed file naming system but keep imperfect change logs. It > would best to work within an environment designed specifically for tracking > version to help us keep track of branches, changes, etc. > > Would anyone recommend github? I haven't used it and am not sure if it is > designed to produce a single software product or several parallel products. > Is there anything better suited for spreadsheets? > > Thanks, > > Luke > >

Hi

Maybe take a look at git-annex

http://git-annex.branchable.com/

You could also use xls2txt in git which can provide human-readable output
from xls files. So in short, you should put this to your .gitattributes
file:

*.xls diff=xls

And in the .git/config:

[diff "xls"]
binary = true
textconv = /path/to/xls2txt

More on this technique is here
http://git-scm.com/book/en/Customizing-Git-Git-Attributes#Binary-Files

Regards

··· On Tue, Jan 21, 2014 at 2:56 PM, Luke MacDonald wrote:

Does anyone know of any good software packages for tracking form versions?

We are conducting a study in multiple countries, now 4 but soon to be
10. A major difficulty lies in that we have to move several parallel files
(countries) forward that have some common overlap and some unique features.
We have multiple languages with multiple collaborators in each country
working on the forms. The overlap and the unique parts of the form
continue to be updated.

We have a detailed file naming system but keep imperfect change logs. It
would best to work within an environment designed specifically for tracking
version to help us keep track of branches, changes, etc.

Would anyone recommend github? I haven't used it and am not sure if it is
designed to produce a single software product or several parallel products.
Is there anything better suited for spreadsheets?

Thanks,

Luke

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I should add that http://tortoisehg.bitbucket.org/ is a nice app that runs
on your PC and manages your interactions with a Mercurial server; it has
versions for both Windows and Mac.

And http://windows.github.com/ or http://mac.github.com/ are similar nice
apps for Git.

Note that your 'commits' are not public until you publish them to the
server. On Mercurial, this is called "Pushing" a change (and "Pulling a
change grabs the public changes and brings them down to your computer). On
Git, this is called Syncing (for both combined).

··· On Tue, Jan 21, 2014 at 9:49 AM, Mitch Sundt wrote:

These sorts of things tend to become religious arguments.

You are looking for a 'source code management' system.
http://en.wikipedia.org/wiki/Source_code_management (... which seems to
be somewhat out of date).

The two dominant open-source systems are Git and Mercurial.

  • Git http://git-scm.com/ is extremely powerful. So powerful that if
    you have sets of files that you need to apply different sets of coordinated
    changes to, you will eventually do it incorrectly.
  • Mercurial http://mercurial.selenic.com/ is less powerful and a bit
    easier to understand. ODK uses Mercurial.

Subversion is also popular, as is Visual SourceSafe (Microsoft); both (in
my opinion) due to the extensive body of legacy software that uses them.

All of these systems keep track of every 'checked in' or 'committed'
change to your files (when you reach a point of completion in your work,
you instruct the system to save this copy of your file; that is called
'checking in' the file or 'committing' your changes). You can compare any
two check-ins from any point in time to see what files have changed.

In addition to supplying comments describing your changes at the time you
commit them, you can also 'tag' your checkins with important descriptive
labels (e.g., 'ODK Tables Alpha 1").
You can set up your own server to act as the 'repository of record' for
your projects, but there are several web hosting services that can host
this for you. This is rather nice because you don't have to worry about
your server crashing. The top 4 are:

For an example of Mercurial, see
http://code.google.com/p/opendatakit/source/list?repo=aggregate (for ODK
Aggregate). note that there can be multiple 'branches' in a repository, so
you can have different versions of the same file. A similar view on GitHub
is https://github.com/UW-ICTD/pyxform/commits/master (for the XLSForm
tool).

Mitch

On Tue, Jan 21, 2014 at 8:56 AM, Luke MacDonald < luke.h.macdonald@gmail.com> wrote:

Does anyone know of any good software packages for tracking form
versions?

We are conducting a study in multiple countries, now 4 but soon to be
10. A major difficulty lies in that we have to move several parallel files
(countries) forward that have some common overlap and some unique features.
We have multiple languages with multiple collaborators in each country
working on the forms. The overlap and the unique parts of the form
continue to be updated.

We have a detailed file naming system but keep imperfect change logs. It
would best to work within an environment designed specifically for tracking
version to help us keep track of branches, changes, etc.

Would anyone recommend github? I haven't used it and am not sure if it is
designed to produce a single software product or several parallel products.
Is there anything better suited for spreadsheets?

Thanks,

Luke

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Couple of clarifications about what the services offer.

Github only supports git. It allows for private repos, but you have to
pay for that privacy. Pretty sure they offer free private repos for
academic institutions.

Bitbucket supports git and mercurial (aka hg). It allows for free
public and private repos for five users. And there are no user limits
for academic institutions.

I like hg and Bitbucket, but I think git and Github are the most
popular. So much so that I'm switching to git for most of my new
projects (but still on Bitbucket for now). As far as the differences,
there have been countless words written about the differences. Comes
down to your personal choice. As
http://www.ericsink.com/entries/hg_denzel.html put it...

Git is Wesley Snipes. Exciting, gives the world lots of talk about.
Loved by the youth.
Mercurial is Denzel Washington. Does great work, but boring otherwise.
Loved by the middle-aged.

There are lots of ways to track branches. I like git/hg flow as
explained at http://blog.sourcetreeapp.com/2012/08/01/smart-branching-with-sourcetree-and-git-flow.
If you use the free SourceTree app on that page to manage your repo,
it makes branching management very easy, but it's still very software
dev focused. A simpler option might be to use the Github app or
TortoiseHG which makes checking in changes a little easier for
non-developers. Then you can combine that with whatever branching
scheme makes the most sense to your users.

The higher level problem you'll have is that while git/hg are great at
managing text files (including XML), they aren't great at binary files
(like XLS). That is, if you have two XML documents, git/hg will show
you what you've changed in a elegant manner. If you have two XLS
documents, all it can tell you is that they are different.

Couple of ways you can manage this process.

  1. Commit the XML (or CSV) and XLS side by side to the repo. Since the
    XML is text, the diff of the XML will help you figure out what you
    changed in the XLS. In addition, use detailed comments about what
    you've changed.

  2. Skip the repo and use Google Spreadsheets to make and track changes
    to the form. You'll have to have some manual branching process, but if
    you really care about seeing the history, it'll definitely help there.
    I guess you could use Microsoft's track changes, but that sounds
    miserable to me.

  3. Try the diff tools at
    http://stackoverflow.com/questions/114698/how-do-i-diff-two-spreadsheets.

Hope that helps,

Yaw

P.S. What about Subversion? Subversion is Morgan Freeman. Works, but
becoming a relic. Loved by geriatrics.

P.P.S What repo tech is Samuel L. Jackson? I'll leave that as an
exercise for the reader.

··· On Tue, Jan 21, 2014 at 10:16 AM, Mitch Sundt wrote: > I should add that http://tortoisehg.bitbucket.org/ is a nice app that runs > on your PC and manages your interactions with a Mercurial server; it has > versions for both Windows and Mac. > > And http://windows.github.com/ or http://mac.github.com/ are similar nice > apps for Git. > > Note that your 'commits' are not public until you publish them to the > server. On Mercurial, this is called "Pushing" a change (and "Pulling a > change grabs the public changes and brings them down to your computer). On > Git, this is called Syncing (for both combined). > > > > On Tue, Jan 21, 2014 at 9:49 AM, Mitch Sundt wrote: >> >> These sorts of things tend to become religious arguments. >> >> You are looking for a 'source code management' system. >> http://en.wikipedia.org/wiki/Source_code_management (... which seems to be >> somewhat out of date). >> >> The two dominant open-source systems are Git and Mercurial. >> >> Git http://git-scm.com/ is extremely powerful. So powerful that if you >> have sets of files that you need to apply different sets of coordinated >> changes to, you will eventually do it incorrectly. >> Mercurial http://mercurial.selenic.com/ is less powerful and a bit easier >> to understand. ODK uses Mercurial. >> >> Subversion is also popular, as is Visual SourceSafe (Microsoft); both (in >> my opinion) due to the extensive body of legacy software that uses them. >> >> All of these systems keep track of every 'checked in' or 'committed' >> change to your files (when you reach a point of completion in your work, you >> instruct the system to save this copy of your file; that is called 'checking >> in' the file or 'committing' your changes). You can compare any two >> check-ins from any point in time to see what files have changed. >> >> In addition to supplying comments describing your changes at the time you >> commit them, you can also 'tag' your checkins with important descriptive >> labels (e.g., 'ODK Tables Alpha 1"). >> >> You can set up your own server to act as the 'repository of record' for >> your projects, but there are several web hosting services that can host this >> for you. This is rather nice because you don't have to worry about your >> server crashing. The top 4 are: >> >> Github https://github.com/ is the dominant free hosting site for Git-based >> software. Everything there is public. >> Google Code http://code.google.com/ (where ODK code is located) can >> support Mercurial or Git. Everything there is public. >> SourceForge http://sourceforge.net/ Everything is public. >> Bitbucket https://bitbucket.org/ These repositories can be private. >> >> For an example of Mercurial, see >> http://code.google.com/p/opendatakit/source/list?repo=aggregate (for ODK >> Aggregate). note that there can be multiple 'branches' in a repository, so >> you can have different versions of the same file. A similar view on GitHub >> is https://github.com/UW-ICTD/pyxform/commits/master (for the XLSForm >> tool). >> >> Mitch >> >> >> On Tue, Jan 21, 2014 at 8:56 AM, Luke MacDonald wrote: >>> >>> Does anyone know of any good software packages for tracking form >>> versions? >>> >>> We are conducting a study in multiple countries, now 4 but soon to be 10. >>> A major difficulty lies in that we have to move several parallel files >>> (countries) forward that have some common overlap and some unique features. >>> We have multiple languages with multiple collaborators in each country >>> working on the forms. The overlap and the unique parts of the form continue >>> to be updated. >>> >>> We have a detailed file naming system but keep imperfect change logs. It >>> would best to work within an environment designed specifically for tracking >>> version to help us keep track of branches, changes, etc. >>> >>> Would anyone recommend github? I haven't used it and am not sure if it is >>> designed to produce a single software product or several parallel products. >>> Is there anything better suited for spreadsheets? >>> >>> Thanks, >>> >>> Luke >>> >>> -- >>> -- >>> Post: opendatakit@googlegroups.com >>> Unsubscribe: opendatakit+unsubscribe@googlegroups.com >>> Options: http://groups.google.com/group/opendatakit?hl=en >>> >>> --- >>> You received this message because you are subscribed to the Google Groups >>> "ODK Community" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to opendatakit+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> -- >> Mitch Sundt >> Software Engineer >> University of Washington >> mitchellsundt@gmail.com > > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitchellsundt@gmail.com > > -- > -- > Post: opendatakit@googlegroups.com > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > Options: http://groups.google.com/group/opendatakit?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "ODK Community" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to opendatakit+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

Git is not really ideal for binary files like xls though unfortunately.
Would be better if we can represent the xlsform form in a text schema.

··· On Tuesday, January 21, 2014, Mitch Sundt wrote:

I should add that http://tortoisehg.bitbucket.org/ is a nice app that
runs on your PC and manages your interactions with a Mercurial server; it
has versions for both Windows and Mac.

And http://windows.github.com/ or http://mac.github.com/ are similar nice
apps for Git.

Note that your 'commits' are not public until you publish them to the
server. On Mercurial, this is called "Pushing" a change (and "Pulling a
change grabs the public changes and brings them down to your computer). On
Git, this is called Syncing (for both combined).

On Tue, Jan 21, 2014 at 9:49 AM, Mitch Sundt mitchellsundt@gmail.comwrote:

These sorts of things tend to become religious arguments.

You are looking for a 'source code management' system.
http://en.wikipedia.org/wiki/Source_code_management (... which seems to
be somewhat out of date).

The two dominant open-source systems are Git and Mercurial.

  • Git http://git-scm.com/ is extremely powerful. So powerful that if
    you have sets of files that you need to apply different sets of coordinated
    changes to, you will eventually do it incorrectly.
  • Mercurial http://mercurial.selenic.com/ is less powerful and a bit
    easier to understand. ODK uses Mercurial.

Subversion is also popular, as is Visual SourceSafe (Microsoft); both (in
my opinion) due to the extensive body of legacy software that uses them.

All of these systems keep track of every 'checked in' or 'committed'
change to your files (when you reach a point of completion in your work,
you instruct the system to save this copy of your file; that is called
'checking in' the file or 'committing' your changes). You can compare any
two check-ins from any point in time to see what files have changed.

In addition to supplying comments describing your changes at the time you
commit them, you can also 'tag' your checkins with important descriptive
labels (e.g., 'ODK Tables Alpha 1").
You can set up your own server to act as the 'repository of record' for
your projects, but there are several web hosting services that can host
this for you. This is rather nice because you don't have to worry about
your server crashing. The top 4 are:

For an example of Mercurial, see
http://code.google.com/p/opendatakit/source/list?repo=aggregate (for ODK
Aggregate). note that there can be multiple 'branches' in a repository, so
you can have different versions of the same file. A similar view on GitHub
is https://github.com/UW-ICTD/pyxform/commits/master (for the XLSForm
tool).

Mitch

On Tue, Jan 21, 2014 at 8:56 AM, Luke MacDonald < luke.h.macdonald@gmail.com> wrote:

Does anyone know of any good software packages for tracking form versions?

We are conducting a study in multiple countries, now 4 but soon to be
10. A major difficulty lies in that we have to move several parallel files
(countries) forward that have some common overlap and some unique features.
We have multiple languages with multiple collaborators in each country
working on the forms. The overlap and the unique parts of the form
continue to be updated.

We have a detailed file naming system but keep imperfect change logs. It
would best to work within an environment designed specifically for tracking
version to help us keep track of branches, changes, etc.

Would anyone recommend github? I haven't used it and am not sure if it is
designed to produce a single software product or several parallel products.
Is there anything better suited for spreadsheets?

Thanks,

Luke

--

Post:

Git is all about having friends. Not sure Wesley snipes has more friends
then Denzel. In terms of bad assery, I'd agree that GitHub is about as
awesome as blade 1.

··· On Tuesday, January 21, 2014, Yaw Anokwa wrote:

Couple of clarifications about what the services offer.

Github only supports git. It allows for private repos, but you have to
pay for that privacy. Pretty sure they offer free private repos for
academic institutions.

Bitbucket supports git and mercurial (aka hg). It allows for free
public and private repos for five users. And there are no user limits
for academic institutions.

I like hg and Bitbucket, but I think git and Github are the most
popular. So much so that I'm switching to git for most of my new
projects (but still on Bitbucket for now). As far as the differences,
there have been countless words written about the differences. Comes
down to your personal choice. As
http://www.ericsink.com/entries/hg_denzel.html put it...

Git is Wesley Snipes. Exciting, gives the world lots of talk about.
Loved by the youth.
Mercurial is Denzel Washington. Does great work, but boring otherwise.
Loved by the middle-aged.

There are lots of ways to track branches. I like git/hg flow as
explained at
http://blog.sourcetreeapp.com/2012/08/01/smart-branching-with-sourcetree-and-git-flow
.
If you use the free SourceTree app on that page to manage your repo,
it makes branching management very easy, but it's still very software
dev focused. A simpler option might be to use the Github app or
TortoiseHG which makes checking in changes a little easier for
non-developers. Then you can combine that with whatever branching
scheme makes the most sense to your users.

The higher level problem you'll have is that while git/hg are great at
managing text files (including XML), they aren't great at binary files
(like XLS). That is, if you have two XML documents, git/hg will show
you what you've changed in a elegant manner. If you have two XLS
documents, all it can tell you is that they are different.

Couple of ways you can manage this process.

  1. Commit the XML (or CSV) and XLS side by side to the repo. Since the
    XML is text, the diff of the XML will help you figure out what you
    changed in the XLS. In addition, use detailed comments about what
    you've changed.

  2. Skip the repo and use Google Spreadsheets to make and track changes
    to the form. You'll have to have some manual branching process, but if
    you really care about seeing the history, it'll definitely help there.
    I guess you could use Microsoft's track changes, but that sounds
    miserable to me.

  3. Try the diff tools at
    http://stackoverflow.com/questions/114698/how-do-i-diff-two-spreadsheets.

Hope that helps,

Yaw

P.S. What about Subversion? Subversion is Morgan Freeman. Works, but
becoming a relic. Loved by geriatrics.

P.P.S What repo tech is Samuel L. Jackson? I'll leave that as an
exercise for the reader.

On Tue, Jan 21, 2014 at 10:16 AM, Mitch Sundt mitchellsundt@gmail.com wrote:

I should add that http://tortoisehg.bitbucket.org/ is a nice app that
runs
on your PC and manages your interactions with a Mercurial server; it has
versions for both Windows and Mac.

And http://windows.github.com/ or http://mac.github.com/ are similar
nice
apps for Git.

Note that your 'commits' are not public until you publish them to the
server. On Mercurial, this is called "Pushing" a change (and "Pulling a
change grabs the public changes and brings them down to your computer).
On
Git, this is called Syncing (for both combined).

On Tue, Jan 21, 2014 at 9:49 AM, Mitch Sundt mitchellsundt@gmail.com wrote:

These sorts of things tend to become religious arguments.

You are looking for a 'source code management' system.
http://en.wikipedia.org/wiki/Source_code_management (... which seems
to be
somewhat out of date).

The two dominant open-source systems are Git and Mercurial.

Git http://git-scm.com/ is extremely powerful. So powerful that if you
have sets of files that you need to apply different sets of coordinated
changes to, you will eventually do it incorrectly.
Mercurial http://mercurial.selenic.com/ is less powerful and a bit
easier
to understand. ODK uses Mercurial.

Subversion is also popular, as is Visual SourceSafe (Microsoft); both
(in
my opinion) due to the extensive body of legacy software that uses them.

All of these systems keep track of every 'checked in' or 'committed'
change to your files (when you reach a point of completion in your
work, you
instruct the system to save this copy of your file; that is called
'checking
in' the file or 'committing' your changes). You can compare any two
check-ins from any point in time to see what files have changed.

In addition to supplying comments describing your changes at the time
you
commit them, you can also 'tag' your checkins with important descriptive
labels (e.g., 'ODK Tables Alpha 1").

You can set up your own server to act as the 'repository of record' for
your projects, but there are several web hosting services that can host
this
for you. This is rather nice because you don't have to worry about your
server crashing. The top 4 are:

Github https://github.com/ is the dominant free hosting site for
Git-based
software. Everything there is public.
Google Code http://code.google.com/ (where ODK code is located) can
support Mercurial or Git. Everything there is public.
SourceForge http://sourceforge.net/ Everything is public.
Bitbucket https://bitbucket.org/ These repositories can be private.

For an example of Mercurial, see
http://code.google.com/p/opendatakit/source/list?repo=aggregate (for
ODK
Aggregate). note that there can be multiple 'branches' in a repository,
so
you can have different versions of the same file. A similar view on
GitHub
is https://github.com/UW-ICTD/pyxform/commits/master (for the XLSForm
tool).

Mitch

On Tue, Jan 21, 2014 at 8:56 AM, Luke MacDonald luke.h.macdonald@gmail.com wrote:

Does