Changing ID/name of form on App Engine

Hi,

I want to keep the media URL links active from the Blobstore of an old 

form but clean out most of the old records of the form on the AppEngine
instance of Aggregate. Using Briefcase, this is possible. I can download
all the old data, wipe the form, clean up the data with the relevant info I
wish to keep, and then use Briefcase to upload the photo files with
associated UUID's. This way I can keep the URL's associated with the
correct photos and the correct records in Sheets, CSVs, Fusion Tables, or
whatever.
All of this worked but then I wanted to rename the uploaded form so if the
original was "TestForm1" I could upload it and the media data as say,
"TestForm2014". I can't make this work at all, even a minor change from
something like "TestForm1" to "TestForm2" results in the following error
(using three test entries as an example):

Preparing for upload of form definition with 0 media attachments
Uploading form definition with no media attachments
retrieving next chunk of instanceIds from server...
Preparing for upload of submission (1 of 3) with 1 media attachments
Uploading submission (1 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Preparing for upload of submission (2 of 3) with 1 media attachments
Uploading submission (2 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Preparing for upload of submission (3 of 3) with 1 media attachments
Uploading submission (3 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Partially successful upload...

Keep in mind that as long as I keep the "Testform1" name this works without
a problem. As best as I can tell, the UUIDs of the original records
preserve the original name of the form (e.g. Testform1) as well.
So is it possible to change the name of the form somehow? Or is there a
better way to keep archived photos with easy to use links in the cloud?
Using another method I can download all this info locally along with the
photos and, with great annoyance, make local URLs but we're moving
completely to a Google environment soon and changing /media/photoname.jpg
will be really difficult to do other than one by one. One form alone has
92 photos attached.
Thank you for your assistance!

It is unclear how you are modifying the ODK Briefcase Storage content to
achieve this.

··· -----------------

Within the form directory,

  • update the form xml to have the new form id.

  • delete the info.db directory.

  • within each instances/.../submission.xml file, change the id="" attribute
    to specify the new form ID.

On Fri, Mar 18, 2016 at 1:32 PM, Greg greg.pommen@edmonton.ca wrote:

Hi,

I want to keep the media URL links active from the Blobstore of an old

form but clean out most of the old records of the form on the AppEngine
instance of Aggregate. Using Briefcase, this is possible. I can download
all the old data, wipe the form, clean up the data with the relevant info I
wish to keep, and then use Briefcase to upload the photo files with
associated UUID's. This way I can keep the URL's associated with the
correct photos and the correct records in Sheets, CSVs, Fusion Tables, or
whatever.
All of this worked but then I wanted to rename the uploaded form so if the
original was "TestForm1" I could upload it and the media data as say,
"TestForm2014". I can't make this work at all, even a minor change from
something like "TestForm1" to "TestForm2" results in the following error
(using three test entries as an example):

Preparing for upload of form definition with 0 media attachments
Uploading form definition with no media attachments
retrieving next chunk of instanceIds from server...
Preparing for upload of submission (1 of 3) with 1 media attachments
Uploading submission (1 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Preparing for upload of submission (2 of 3) with 1 media attachments
Uploading submission (2 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Preparing for upload of submission (3 of 3) with 1 media attachments
Uploading submission (3 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server
Error (500) while accessing: https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions are
all correct.
Partially successful upload...

Keep in mind that as long as I keep the "Testform1" name this works
without a problem. As best as I can tell, the UUIDs of the original
records preserve the original name of the form (e.g. Testform1) as well.
So is it possible to change the name of the form somehow? Or is there a
better way to keep archived photos with easy to use links in the cloud?
Using another method I can download all this info locally along with the
photos and, with great annoyance, make local URLs but we're moving
completely to a Google environment soon and changing /media/photoname.jpg
will be really difficult to do other than one by one. One form alone has
92 photos attached.
Thank you for your assistance!

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

That's pretty much what I did. More specifically:

-within the form xml:
-<h:title>Example Form 1</h:title> becomes <h:title>Example Form
2</h:title>
-
becomes
-all references to the original ID get changed,
e.g. nodeset="/exampleform1/Photo" changes to
nodeset="/exampleform2/Photo"
-the above were changed regex Find/Replace in Notepad++

-deleted the info.db directory

-I changed each submission.xml file so that the beginning of the file that
reads: <exampleform1 id="ExampleForm1" becomes <exampleform2
id="ExampleForm2"
-Probably not important but I used grepWin to change these files en
masse.

Then I uploaded the using the Push function in Briefcase. Even if I
uploaded to a different ODKAggregate instance (I have production and a test
instances of it) I get the same errors.

By the sound of your response this should be possible so I've very curious
to find out what I might have messed up along the way.
Thank you!

··· On Friday, March 18, 2016 at 3:32:50 PM UTC-6, Mitch Sundt wrote: > > It is unclear how you are modifying the ODK Briefcase Storage content to > achieve this. > ----------------- > > Within the form directory, > > - update the form xml to have the new form id. > > - delete the info.db directory. > > - within each instances/.../submission.xml file, change the id="" > attribute to specify the new form ID. > > > On Fri, Mar 18, 2016 at 1:32 PM, Greg <greg....@edmonton.ca > wrote: > >> Hi, >> >> I want to keep the media URL links active from the Blobstore of an >> old form but clean out most of the old records of the form on the AppEngine >> instance of Aggregate. Using Briefcase, this is possible. I can download >> all the old data, wipe the form, clean up the data with the relevant info I >> wish to keep, and then use Briefcase to upload the photo files with >> associated UUID's. This way I can keep the URL's associated with the >> correct photos and the correct records in Sheets, CSVs, Fusion Tables, or >> whatever. >> All of this worked but then I wanted to rename the uploaded form so if >> the original was "TestForm1" I could upload it and the media data as say, >> "TestForm2014". I can't make this work at all, even a minor change from >> something like "TestForm1" to "TestForm2" results in the following error >> (using three test entries as an example): >> >> >> Preparing for upload of form definition with 0 media attachments >> Uploading form definition with no media attachments >> retrieving next chunk of instanceIds from server... >> Preparing for upload of submission (1 of 3) with 1 media attachments >> Uploading submission (1 of 3) and 1 media attachments >> UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server >> Error (500) while accessing: https://example.appspot.com/submission >> Please verify that the URL, your user credentials and your permissions >> are all correct. >> Preparing for upload of submission (2 of 3) with 1 media attachments >> Uploading submission (2 of 3) and 1 media attachments >> UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server >> Error (500) while accessing: https://example.appspot.com/submission >> Please verify that the URL, your user credentials and your permissions >> are all correct. >> Preparing for upload of submission (3 of 3) with 1 media attachments >> Uploading submission (3 of 3) and 1 media attachments >> UPLOAD FAILED: Submission upload failed. Detailed error: Internal Server >> Error (500) while accessing: https://example.appspot.com/submission >> Please verify that the URL, your user credentials and your permissions >> are all correct. >> Partially successful upload... >> >> >> Keep in mind that as long as I keep the "Testform1" name this works >> without a problem. As best as I can tell, the UUIDs of the original >> records preserve the original name of the form (e.g. Testform1) as well. >> So is it possible to change the name of the form somehow? Or is there a >> better way to keep archived photos with easy to use links in the cloud? >> Using another method I can download all this info locally along with the >> photos and, with great annoyance, make local URLs but we're moving >> completely to a Google environment soon and changing /media/photoname.jpg >> will be really difficult to do other than one by one. One form alone has >> 92 photos attached. >> Thank you for your assistance! >> >> -- >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ODK Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >

What you did is what I think should work. The changing of the top-level
element name is not necessary within ODK Aggregate (other servers pay
attention to that name). So you could have left that as ODK
Aggregate uses the id attribute to match a submission with its form
definition. That avoids a lot of changes in the rest of the form definition
document (just a one-liner).

Make sure that the revised form validates with ODK Validate. Then try to
upload that form definition onto ODK Aggregate by itself. That would at
least verify that the form definition changes you did were all OK.

··· On Mon, Mar 21, 2016 at 8:28 AM, Greg wrote:

That's pretty much what I did. More specifically:

-within the form xml:
-<h:title>Example Form 1</h:title> becomes <h:title>Example Form
2</h:title>
-
becomes
-all references to the original ID get changed,
e.g. nodeset="/exampleform1/Photo" changes to
nodeset="/exampleform2/Photo"
-the above were changed regex Find/Replace in Notepad++

-deleted the info.db directory

-I changed each submission.xml file so that the beginning of the file that
reads: <exampleform1 id="ExampleForm1" becomes <exampleform2
id="ExampleForm2"
-Probably not important but I used grepWin to change these files en
masse.

Then I uploaded the using the Push function in Briefcase. Even if I
uploaded to a different ODKAggregate instance (I have production and a test
instances of it) I get the same errors.

By the sound of your response this should be possible so I've very curious
to find out what I might have messed up along the way.
Thank you!

On Friday, March 18, 2016 at 3:32:50 PM UTC-6, Mitch Sundt wrote:

It is unclear how you are modifying the ODK Briefcase Storage content to
achieve this.

Within the form directory,

  • update the form xml to have the new form id.

  • delete the info.db directory.

  • within each instances/.../submission.xml file, change the id=""
    attribute to specify the new form ID.

On Fri, Mar 18, 2016 at 1:32 PM, Greg greg....@edmonton.ca wrote:

Hi,

I want to keep the media URL links active from the Blobstore of an

old form but clean out most of the old records of the form on the AppEngine
instance of Aggregate. Using Briefcase, this is possible. I can download
all the old data, wipe the form, clean up the data with the relevant info I
wish to keep, and then use Briefcase to upload the photo files with
associated UUID's. This way I can keep the URL's associated with the
correct photos and the correct records in Sheets, CSVs, Fusion Tables, or
whatever.
All of this worked but then I wanted to rename the uploaded form so if
the original was "TestForm1" I could upload it and the media data as say,
"TestForm2014". I can't make this work at all, even a minor change from
something like "TestForm1" to "TestForm2" results in the following error
(using three test entries as an example):

Preparing for upload of form definition with 0 media attachments
Uploading form definition with no media attachments
retrieving next chunk of instanceIds from server...
Preparing for upload of submission (1 of 3) with 1 media attachments
Uploading submission (1 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal
Server Error (500) while accessing:
https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions
are all correct.
Preparing for upload of submission (2 of 3) with 1 media attachments
Uploading submission (2 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal
Server Error (500) while accessing:
https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions
are all correct.
Preparing for upload of submission (3 of 3) with 1 media attachments
Uploading submission (3 of 3) and 1 media attachments
UPLOAD FAILED: Submission upload failed. Detailed error: Internal
Server Error (500) while accessing:
https://example.appspot.com/submission
Please verify that the URL, your user credentials and your permissions
are all correct.
Partially successful upload...

Keep in mind that as long as I keep the "Testform1" name this works
without a problem. As best as I can tell, the UUIDs of the original
records preserve the original name of the form (e.g. Testform1) as well.
So is it possible to change the name of the form somehow? Or is there a
better way to keep archived photos with easy to use links in the cloud?
Using another method I can download all this info locally along with the
photos and, with great annoyance, make local URLs but we're moving
completely to a Google environment soon and changing /media/photoname.jpg
will be really difficult to do other than one by one. One form alone has
92 photos attached.
Thank you for your assistance!

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Mr. Sundt, you are a genius! It worked!! This part "So you could have
left that as " is the key. You have to leave that one alone
but the Title and ID can be changed. That will work for me though. The
Title and ID are all I need to change to keep them names of archive
materials separate from current in use forms and maintain the URL links
for data already collected. I'm hoping that made sense! This is great
news!

Thank you very, very much!

··· On Wednesday, March 23, 2016 at 12:28:15 PM UTC-6, Mitch Sundt wrote: > > What you did is what I think should work. The changing of the top-level > element name is not necessary within ODK Aggregate (other servers pay > attention to that name). So you could have left that as ODK > Aggregate uses the id attribute to match a submission with its form > definition. That avoids a lot of changes in the rest of the form definition > document (just a one-liner). > > Make sure that the revised form validates with ODK Validate. Then try to > upload that form definition onto ODK Aggregate by itself. That would at > least verify that the form definition changes you did were all OK. > > > > On Mon, Mar 21, 2016 at 8:28 AM, Greg <greg....@edmonton.ca > wrote: > >> That's pretty much what I did. More specifically: >> >> -within the form xml: >> -Example Form 1 becomes Example Form >> 2 >> - >> becomes >> -all references to the original ID get changed, >> e.g. nodeset="/exampleform1/Photo" changes to >> nodeset="/exampleform2/Photo" >> -the above were changed regex Find/Replace in Notepad++ >> >> -deleted the info.db directory >> >> -I changed each submission.xml file so that the beginning of the file >> that reads: <exampleform1 id="ExampleForm1" becomes > id="ExampleForm2" >> -Probably not important but I used grepWin to change these files en >> masse. >> >> >> Then I uploaded the using the Push function in Briefcase. Even if I >> uploaded to a different ODKAggregate instance (I have production and a test >> instances of it) I get the same errors. >> >> By the sound of your response this should be possible so I've very >> curious to find out what I might have messed up along the way. >> Thank you! >> >> >> >> >> On Friday, March 18, 2016 at 3:32:50 PM UTC-6, Mitch Sundt wrote: >>> >>> It is unclear how you are modifying the ODK Briefcase Storage content to >>> achieve this. >>> ----------------- >>> >>> Within the form directory, >>> >>> - update the form xml to have the new form id. >>> >>> - delete the info.db directory. >>> >>> - within each instances/.../submission.xml file, change the id="" >>> attribute to specify the new form ID. >>> >>> >>> On Fri, Mar 18, 2016 at 1:32 PM, Greg wrote: >>> >>>> Hi, >>>> >>>> I want to keep the media URL links active from the Blobstore of an >>>> old form but clean out most of the old records of the form on the AppEngine >>>> instance of Aggregate. Using Briefcase, this is possible. I can download >>>> all the old data, wipe the form, clean up the data with the relevant info I >>>> wish to keep, and then use Briefcase to upload the photo files with >>>> associated UUID's. This way I can keep the URL's associated with the >>>> correct photos and the correct records in Sheets, CSVs, Fusion Tables, or >>>> whatever. >>>> All of this worked but then I wanted to rename the uploaded form so if >>>> the original was "TestForm1" I could upload it and the media data as say, >>>> "TestForm2014". I can't make this work at all, even a minor change from >>>> something like "TestForm1" to "TestForm2" results in the following error >>>> (using three test entries as an example): >>>> >>>> >>>> Preparing for upload of form definition with 0 media attachments >>>> Uploading form definition with no media attachments >>>> retrieving next chunk of instanceIds from server... >>>> Preparing for upload of submission (1 of 3) with 1 media attachments >>>> Uploading submission (1 of 3) and 1 media attachments >>>> UPLOAD FAILED: Submission upload failed. Detailed error: Internal >>>> Server Error (500) while accessing: >>>> https://example.appspot.com/submission >>>> Please verify that the URL, your user credentials and your permissions >>>> are all correct. >>>> Preparing for upload of submission (2 of 3) with 1 media attachments >>>> Uploading submission (2 of 3) and 1 media attachments >>>> UPLOAD FAILED: Submission upload failed. Detailed error: Internal >>>> Server Error (500) while accessing: >>>> https://example.appspot.com/submission >>>> Please verify that the URL, your user credentials and your permissions >>>> are all correct. >>>> Preparing for upload of submission (3 of 3) with 1 media attachments >>>> Uploading submission (3 of 3) and 1 media attachments >>>> UPLOAD FAILED: Submission upload failed. Detailed error: Internal >>>> Server Error (500) while accessing: >>>> https://example.appspot.com/submission >>>> Please verify that the URL, your user credentials and your permissions >>>> are all correct. >>>> Partially successful upload... >>>> >>>> >>>> Keep in mind that as long as I keep the "Testform1" name this works >>>> without a problem. As best as I can tell, the UUIDs of the original >>>> records preserve the original name of the form (e.g. Testform1) as well. >>>> So is it possible to change the name of the form somehow? Or is there >>>> a better way to keep archived photos with easy to use links in the cloud? >>>> Using another method I can download all this info locally along with >>>> the photos and, with great annoyance, make local URLs but we're moving >>>> completely to a Google environment soon and changing /media/photoname.jpg >>>> will be really difficult to do other than one by one. One form alone has >>>> 92 photos attached. >>>> Thank you for your assistance! >>>> >>>> -- >>>> -- >>>> Post: opend...@googlegroups.com >>>> Unsubscribe: opendatakit...@googlegroups.com >>>> Options: http://groups.google.com/group/opendatakit?hl=en >>>> >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "ODK Community" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to opendatakit...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Mitch Sundt >>> Software Engineer >>> University of Washington >>> mitche...@gmail.com >>> >> -- >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ODK Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >