New VERSION of form won't upload to Aggregate

Hey,

I am building forms on excel and then using XLS to convert them to XML.
When I update the version and make edits (not including changing any
question types, the number of questions etc, just updating translations,
constraints and required questions) i get the following message upon trying
to upload the form to ODK Aggregate 'Form Already Exists for this
Namespace/Id attribute
org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'

Does anyone know of common errors associated with this problem? I have done
some careful reading and just can't figure out where I am going wrong!

Many thanks,

Sachin

My settings tab

Before
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number}, '-',
${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA}) 2015040901
After
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number}, '-',
${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA}) 2015041302

Hi

You need to change the form_id since both forms cannot use the same one

Regards

ยทยทยท On Mon, Apr 13, 2015 at 7:04 AM, Sachin wrote:

Hey,

I am building forms on excel and then using XLS to convert them to XML.
When I update the version and make edits (not including changing any
question types, the number of questions etc, just updating translations,
constraints and required questions) i get the following message upon trying
to upload the form to ODK Aggregate 'Form Already Exists for this
Namespace/Id attribute
org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'

Does anyone know of common errors associated with this problem? I have
done some careful reading and just can't figure out where I am going wrong!

Many thanks,

Sachin

My settings tab

Before
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015040901 After
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015041302

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hey Ian, thanks for the tip.

If I change in the form_id will future finalized forms be stored in the
same place or will they be treated as separate forms?

The literature says:
a separate version setting is used for revisions that do not change the
data being collected or their data types. These 'minor' revisions include
adding language translations, correcting spelling errors within the text,
changing external dataset files, and updating multi-media prompts.

NOTE: Once a form is uploaded to ODK Aggregate, the number of questions or
the type of question cannot be changed.
You either have to delete the form
from the datastore (deleting all of its submissions) or change the id or
xmlns attribute to re-upload the form (in which case it will be treated as
an entirely different form and can co-exist with the original). But, as
noted above, other changes to the form definition are allowed through the
use of the version setting.

Thanks again and sorry if I am missing something obvious

Sachin

ยทยทยท On Monday, 13 April 2015 13:34:43 UTC+2, Ian Lawrence wrote: > > Hi > > You need to change the form_id since both forms cannot use the same one > > Regards > > On Mon, Apr 13, 2015 at 7:04 AM, Sachin <sachinde...@gmail.com > wrote: > >> Hey, >> >> I am building forms on excel and then using XLS to convert them to XML. >> When I update the version and make edits (not including changing any >> question types, the number of questions etc, just updating translations, >> constraints and required questions) i get the following message upon trying >> to upload the form to ODK Aggregate '*Form Already Exists for this >> Namespace/Id attribute >> org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'* >> >> Does anyone know of common errors associated with this problem? I have >> done some careful reading and just can't figure out where I am going wrong! >> >> Many thanks, >> >> Sachin >> >> My settings tab >> >> *Before* >> form_title form_id instance_name version Teen Baseline 1 >> Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number}, >> '-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA}) >> 2015040901 *After* >> form_title form_id instance_name version Teen Baseline 1 >> Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number}, >> '-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA}) >> 2015041302 >> >> -- >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ODK Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > >

You should be able to simply update the version unless something has
materially changed in the form.

Changing the form_id will create a completely new set of data tables.

The server log might have more information on what difference is causing
this update to be rejected. If the log is not informative, send me the
before and after form/XML, and I will see what I can see.

ยทยทยท On Mon, Apr 13, 2015 at 5:57 AM, Sachin wrote:

Hey Ian, thanks for the tip.

If I change in the form_id will future finalized forms be stored in the
same place or will they be treated as separate forms?

The literature says:
a separate version setting is used for revisions that do not change the
data being collected or their data types. These 'minor' revisions include
adding language translations, correcting spelling errors within the text,
changing external dataset files, and updating multi-media prompts.

NOTE: Once a form is uploaded to ODK Aggregate, the number of questions
or the type of question cannot be changed.
You either have to delete the
form from the datastore (deleting all of its submissions) or change the id
or xmlns attribute to re-upload the form (in which case it will be treated
as an entirely different form and can co-exist with the original). But,
as noted above, other changes to the form definition are allowed through
the use of the version setting.

Thanks again and sorry if I am missing something obvious

Sachin

On Monday, 13 April 2015 13:34:43 UTC+2, Ian Lawrence wrote:

Hi

You need to change the form_id since both forms cannot use the same one

Regards

On Mon, Apr 13, 2015 at 7:04 AM, Sachin sachinde...@gmail.com wrote:

Hey,

I am building forms on excel and then using XLS to convert them to XML.
When I update the version and make edits (not including changing any
question types, the number of questions etc, just updating translations,
constraints and required questions) i get the following message upon trying
to upload the form to ODK Aggregate 'Form Already Exists for this
Namespace/Id attribute
org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'

Does anyone know of common errors associated with this problem? I have
done some careful reading and just can't figure out where I am going wrong!

Many thanks,

Sachin

My settings tab

Before
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015040901 After
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015041302

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Ah, OK. You DO have to use the same filename when generating newer versions
of your form.

The XLSForm generator does pay attention to the filename. This is a
compromise amongst the various teams; the ODK tools look for an "id"
attribute under the root element of the submission, and treat that as the
unique identifier for the form. Other tools use the name of the root
element of the submission instance to identify the form -- and for
convenience, they use the filename of the XLS file as the name for the root
element.

So when you generated your v2 form using a filename with a v2 after it, it
did produce an incompatible change in the form definition because the root
element of the submission instance changed to:

  <instance>
    <*Teen_Baseline1_STS_090415v2* id="Teen_Baseline_1_KWT_STS_2015"

version="2015041302">
...

From:

  <instance>
    <*Teen_Baseline1_STS_090415* id="Teen_Baseline_1_KWT_STS_2015"

version="2015041302">

ยทยทยท -------------------- To avoid this, maintain separate folders with the version designations, and keep the xlsx filenames the same.

Mitch

On Tue, Apr 14, 2015 at 3:41 AM, Sachin De Stone <sachindestoneerc@gmail.com wrote:

Hey Mitch,

Thanks so much for the offer - my PI says to thank you immensely :slight_smile: I
can't find anything illuminating but I might easily be missing something.

Here is the before and after XML form (v2 is the after form). I have made
one single edit to the note (label: Sipho lives in a roundavel whilst
Thobeka lives in a big brick house.) by removing the 'yes' from required. I
have then also updated the version number.

The only two ideas I currently have are:
a) The file name is different (but I have tested using the same file name
with earlier editions and it didn't seem to work)
b) The Meta-ui-version* is empty in the metadata - though I have no idea
what the meaning if this is (see attached)

Many, many thanks for your help. It is really appreciated!

Best,

Sachin

Project Manager
Sinovuyo Teen Study

100 Alexandra Road,
King Williams Town,
5600, Eastern Cape,
South Africa
+27 (0)73 748 6942

On 13 April 2015 at 20:18, Mitch Sundt mitchellsundt@gmail.com wrote:

You should be able to simply update the version unless something has
materially changed in the form.

Changing the form_id will create a completely new set of data tables.

The server log might have more information on what difference is causing
this update to be rejected. If the log is not informative, send me the
before and after form/XML, and I will see what I can see.

On Mon, Apr 13, 2015 at 5:57 AM, Sachin sachindestoneerc@gmail.com wrote:

Hey Ian, thanks for the tip.

If I change in the form_id will future finalized forms be stored in the
same place or will they be treated as separate forms?

The literature says:
a separate version setting is used for revisions that do not change
the data being collected or their data types. These 'minor' revisions
include adding language translations, correcting spelling errors within the
text, changing external dataset files, and updating multi-media prompts.

NOTE: Once a form is uploaded to ODK Aggregate, the number of questions
or the type of question cannot be changed.
You either have to delete
the form from the datastore (deleting all of its submissions) or change the
id or xmlns attribute to re-upload the form (in which case it will be
treated as an entirely different form and can co-exist with the original).
But, as noted above, other changes to the form definition are allowed
through the use of the version setting.

Thanks again and sorry if I am missing something obvious

Sachin

On Monday, 13 April 2015 13:34:43 UTC+2, Ian Lawrence wrote:

Hi

You need to change the form_id since both forms cannot use the same one

Regards

On Mon, Apr 13, 2015 at 7:04 AM, Sachin sachinde...@gmail.com wrote:

Hey,

I am building forms on excel and then using XLS to convert them to
XML. When I update the version and make edits (not including changing any
question types, the number of questions etc, just updating translations,
constraints and required questions) i get the following message upon trying
to upload the form to ODK Aggregate 'Form Already Exists for this
Namespace/Id attribute
org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'

Does anyone know of common errors associated with this problem? I have
done some careful reading and just can't figure out where I am going wrong!

Many thanks,

Sachin

My settings tab

Before
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015040901 After
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-', ${QD_family_number},
'-', ${QD_survey_location}, '-', ${QD_survey_date}, '-', ${QD_RA})
2015041302

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/rsaejLTkX_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

2 Likes

Mitch,

You are my new hero. I spent hours trying to work out what was going on.

Thank you so so so much from me and the team (you were even mentioned in
our team meeting notes with UNICEF SA ;).

Thanks a billion. I'll write up the solution on the group in case anyway
else is looking through.

Best,

Sachin

Project Manager
Sinovuyo Teen Study

100 Alexandra Road,
King Williams Town,
5600, Eastern Cape,
South Africa
+27 (0)73 748 6942

ยทยทยท On 14 April 2015 at 17:56, Mitch Sundt wrote:

Ah, OK. You DO have to use the same filename when generating newer
versions of your form.

The XLSForm generator does pay attention to the filename. This is a
compromise amongst the various teams; the ODK tools look for an "id"
attribute under the root element of the submission, and treat that as the
unique identifier for the form. Other tools use the name of the root
element of the submission instance to identify the form -- and for
convenience, they use the filename of the XLS file as the name for the root
element.

So when you generated your v2 form using a filename with a v2 after it, it
did produce an incompatible change in the form definition because the root
element of the submission instance changed to:

  <instance>
    <*Teen_Baseline1_STS_090415v2* id="Teen_Baseline_1_KWT_STS_2015"

version="2015041302">
...

From:

  <instance>
    <*Teen_Baseline1_STS_090415* id="Teen_Baseline_1_KWT_STS_2015"

version="2015041302">


To avoid this, maintain separate folders with the version designations,
and keep the xlsx filenames the same.

Mitch

On Tue, Apr 14, 2015 at 3:41 AM, Sachin De Stone < sachindestoneerc@gmail.com> wrote:

Hey Mitch,

Thanks so much for the offer - my PI says to thank you immensely :slight_smile: I
can't find anything illuminating but I might easily be missing something.

Here is the before and after XML form (v2 is the after form). I have made
one single edit to the note (label: Sipho lives in a roundavel whilst
Thobeka lives in a big brick house.) by removing the 'yes' from required. I
have then also updated the version number.

The only two ideas I currently have are:
a) The file name is different (but I have tested using the same file name
with earlier editions and it didn't seem to work)
b) The Meta-ui-version* is empty in the metadata - though I have no idea
what the meaning if this is (see attached)

Many, many thanks for your help. It is really appreciated!

Best,

Sachin

Project Manager
Sinovuyo Teen Study

100 Alexandra Road,
King Williams Town,
5600, Eastern Cape,
South Africa
+27 (0)73 748 6942

On 13 April 2015 at 20:18, Mitch Sundt mitchellsundt@gmail.com wrote:

You should be able to simply update the version unless something has
materially changed in the form.

Changing the form_id will create a completely new set of data tables.

The server log might have more information on what difference is causing
this update to be rejected. If the log is not informative, send me the
before and after form/XML, and I will see what I can see.

On Mon, Apr 13, 2015 at 5:57 AM, Sachin sachindestoneerc@gmail.com wrote:

Hey Ian, thanks for the tip.

If I change in the form_id will future finalized forms be stored in the
same place or will they be treated as separate forms?

The literature says:
a separate version setting is used for revisions that do not change
the data being collected or their data types. These 'minor' revisions
include adding language translations, correcting spelling errors within the
text, changing external dataset files, and updating multi-media prompts.

NOTE: Once a form is uploaded to ODK Aggregate, the number of
questions or the type of question cannot be changed.
You either have
to delete the form from the datastore (deleting all of its submissions) or
change the id or xmlns attribute to re-upload the form (in which case it
will be treated as an entirely different form and can co-exist with the
original). But, as noted above, other changes to the form definition
are allowed through the use of the version setting.

Thanks again and sorry if I am missing something obvious

Sachin

On Monday, 13 April 2015 13:34:43 UTC+2, Ian Lawrence wrote:

Hi

You need to change the form_id since both forms cannot use the same one

Regards

On Mon, Apr 13, 2015 at 7:04 AM, Sachin sachinde...@gmail.com wrote:

Hey,

I am building forms on excel and then using XLS to convert them to
XML. When I update the version and make edits (not including changing any
question types, the number of questions etc, just updating translations,
constraints and required questions) i get the following message upon trying
to upload the form to ODK Aggregate 'Form Already Exists for this
Namespace/Id attribute
org.opendatakit.aggregate.exception.ODKFormAlreadyExistsException'

Does anyone know of common errors associated with this problem? I
have done some careful reading and just can't figure out where I am going
wrong!

Many thanks,

Sachin

My settings tab

Before
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-',
${QD_family_number}, '-', ${QD_survey_location}, '-', ${QD_survey_date},
'-', ${QD_RA}) 2015040901 After
form_title form_id instance_name version Teen Baseline 1
Teen_Baseline_1_KWT_STS_2015 concat('TBL1', '-',
${QD_family_number}, '-', ${QD_survey_location}, '-', ${QD_survey_date},
'-', ${QD_RA}) 2015041302

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/rsaejLTkX_U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

This was the solution to my problem as well, thanks!

Looking at the documentation about this topic (http://docs.opendatakit.org/form-update/) I could not find this requirement for updating forms - in fact, the example forms have different filenames after their updates... Would it be an idea to add a note to this documentation page to remind people to use the same filename when they want to update a form without deleting the previous version?

I'd be happy to contribute to the docs myself but must say the workflow described in the guidelines looks a bit intimidating for a non-developer. Also, I remember that improving form versioning has been mentioned as a priority by TSC members so I'm not sure how long this problem will stay relevant.

1 Like

This is because we have to have different file names on the server,
though I agree it is a bit confusing.

Adding a note to the docs here would probably help.

Iโ€™d be happy to contribute to the docs myself but must say the workflow described in the guidelines looks a bit intimidating for a non-developer.

We'd love to have you!

The easiest way to make a contribution is to file an issue when you see that a doc needs to be improved. If you have the time to write the note the way you think it would be most helpful, and where it should go, we can get it into the docs pretty easily.

If you're interested in updating the docs yourself, and learning a little bit about dev practices, I encourage you to try it out. The contributor guide's instructions might seem intimidating, but that's because they actually walk you through all the details, as if you aren't a developer and don't know how to use those tools. We've tried to make everything as clear as possible, even though that has resulted in a very long contributor guide.

Also, if you get hung up you can message me here or on the developer slack and I'll do my best to help you out.

1 Like

Hi @Lieke, It would be great to have you contributing to the docs. I was a newbie to opensource as well but after going through the contributor's guide, it was pretty easy to setup and start contributing. It would be great if you would like to add a note about having same filenames when updating forms. Here is the relevant issue: https://github.com/opendatakit/docs/issues/507.
Also, if you would like to start with a simple one, you can do a line-edit as well: https://github.com/opendatakit/docs/issues/96 :slight_smile:

1 Like