Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
that they were uploaded successfully. However on logging in to aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they are not
appearing as instances that can be uploaded. I can still see them under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
On Aggregate, these submissions may have been determined to be incomplete
-- this occurs if the captured media attachments for the filled-in form
were not all successfully uploaded. You can look at incomplete submissions
on the Forms Management / Submission Admin tab. If you "Mark As Complete"
the form, that action will remove all traces of the missing attachments.
Alternatively, on that tab you can "Manually Upload Submission Data" and
navigate to the phone's /sdcard/odk/instances directory, into the directory
of the incomplete submission, and manually upload the submission's xml file
and missing media attachments to complete the submission.
Once the submission is complete, it will move to the Submissions tab and
disappear from the Submission Admin tab.
If you are using an older version of ODK Collect, it may be that the media
file formats were not recognized by that older version (1.1.5 and earlier).
Mitch
···
On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon wrote:
Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
that they were uploaded successfully. However on logging in to aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they are not
appearing as instances that can be uploaded. I can still see them under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Hi Mitch,
Thanks for the heads up. Surprisingly I had never noticed this tab or just
didn't know its function prior to this! But we have some problems!
It seems that something weird happened during submissions. Under the
Submissions Admin Tab, I have 2 instances each of my un-submitted form. I
can see clearly that duplicate instances data is identical, down to the
auto inserted device-id, start-time and end-time. The only discrepancy
comes in the media data collected. Each instance is meant to have a minimum
of 5 photos and a no maximum, as there is a looping logic to add as many
photos as possible. These photos have been split between the duplicate
instances, in that some photos appear in one instance and not in the other
and vice versa. This is a perfect case of merging the duplicate instances.
I guess I have to do this at the DB level.
I don't know how many have experienced this scenario, but I think it makes
the case for the merge feature in case of in-complete submissions.
On another note, I tried using the "Manually Upload Submission Data"
function as a 2nd option, but it only offered 1 option of uploading the
media files. On using this feature, I ended up with an incomplete instance
again!
This can be solved by zipping all the media files and uploading them as a
zip file and doing the un-zipping and merging on the server side...just a
thought!
···
*
*
Regards
Kihara Absolomon
On Mon, Jun 4, 2012 at 9:05 PM, Mitch S mitchellsundt@gmail.com wrote:
On Aggregate, these submissions may have been determined to be incomplete
-- this occurs if the captured media attachments for the filled-in form
were not all successfully uploaded. You can look at incomplete submissions
on the Forms Management / Submission Admin tab. If you "Mark As Complete"
the form, that action will remove all traces of the missing attachments.
Alternatively, on that tab you can "Manually Upload Submission Data" and
navigate to the phone's /sdcard/odk/instances directory, into the directory
of the incomplete submission, and manually upload the submission's xml file
and missing media attachments to complete the submission.
Once the submission is complete, it will move to the Submissions tab and
disappear from the Submission Admin tab.
If you are using an older version of ODK Collect, it may be that the media
file formats were not recognized by that older version (1.1.5 and earlier).
Mitch
On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon soloincc@gmail.comwrote:
Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
that they were uploaded successfully. However on logging in to aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they are not
appearing as instances that can be uploaded. I can still see them under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
The correct solution is to define an in your form. That
instanceID will be used to merge the multiple submissions necessary to
upload the data. The form submission is already effectively a zip of the
data. The problem comes when that data is very large. The larger it is,
the more likely the transmission will fail, and the more resources are
needed on the server to handle the data in one pass. By explicitly
providing an instanceID, the data can be sent in smaller chunks and merged
on the server.
This mechanism is defined here:
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema
For ODK Aggregate, you don't need to specify the namespace, just have a
group in your form.
With ODK Collect 1.1.7 and later, the bind for the instanceID would be:
...
...
...
You can construct your own instanceID expressions. However, you should
avoid symbols and punctuation other than colons and dashes since the
parsing logic within Aggregate is likely fragile if you go wild with
punctuation (and that is used later on when retrieving images, repeat
groups, etc.). If you do construct your own values, they must be less than
80 characters long (a limitation of ODK Aggregate) and must be unique
across all devices and submissions.
Mitch
···
On Mon, Jun 4, 2012 at 3:42 PM, Kihara Absolomon wrote:
Hi Mitch,
Thanks for the heads up. Surprisingly I had never noticed this tab or just
didn't know its function prior to this! But we have some problems!
It seems that something weird happened during submissions. Under the
Submissions Admin Tab, I have 2 instances each of my un-submitted form. I
can see clearly that duplicate instances data is identical, down to the
auto inserted device-id, start-time and end-time. The only discrepancy
comes in the media data collected. Each instance is meant to have a minimum
of 5 photos and a no maximum, as there is a looping logic to add as many
photos as possible. These photos have been split between the duplicate
instances, in that some photos appear in one instance and not in the other
and vice versa. This is a perfect case of merging the duplicate instances.
I guess I have to do this at the DB level.
I don't know how many have experienced this scenario, but I think it
makes the case for the merge feature in case of in-complete submissions.
On another note, I tried using the "Manually Upload Submission Data"
function as a 2nd option, but it only offered 1 option of uploading the
media files. On using this feature, I ended up with an incomplete instance
again!
This can be solved by zipping all the media files and uploading them as
a zip file and doing the un-zipping and merging on the server side...just a
thought!
*
*
Regards
Kihara Absolomon
On Mon, Jun 4, 2012 at 9:05 PM, Mitch S mitchellsundt@gmail.com wrote:
On Aggregate, these submissions may have been determined to be incomplete
-- this occurs if the captured media attachments for the filled-in form
were not all successfully uploaded. You can look at incomplete submissions
on the Forms Management / Submission Admin tab. If you "Mark As Complete"
the form, that action will remove all traces of the missing attachments.
Alternatively, on that tab you can "Manually Upload Submission Data" and
navigate to the phone's /sdcard/odk/instances directory, into the directory
of the incomplete submission, and manually upload the submission's xml file
and missing media attachments to complete the submission.
Once the submission is complete, it will move to the Submissions tab and
disappear from the Submission Admin tab.
If you are using an older version of ODK Collect, it may be that the
media file formats were not recognized by that older version (1.1.5 and
earlier).
Mitch
On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon soloincc@gmail.comwrote:
Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
that they were uploaded successfully. However on logging in to aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they are not
appearing as instances that can be uploaded. I can still see them under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Wow! Thanks a lot for the pointers. The metadata thingie is soo cool.
Thanks again.
What do I do with the partially uploaded submissions? I have tried to zip
an instance folder and uploading it but it fails. Zipping the photos on
their own while the form is uploaded as a plain xml also fails!
How should I upload these partial submissions?
Regards
Kihara Absolomon
···
On Tue, Jun 5, 2012 at 2:50 AM, Mitch S wrote:
The correct solution is to define an in your form.
That instanceID will be used to merge the multiple submissions necessary to
upload the data. The form submission is already effectively a zip of the
data. The problem comes when that data is very large. The larger it is,
the more likely the transmission will fail, and the more resources are
needed on the server to handle the data in one pass. By explicitly
providing an instanceID, the data can be sent in smaller chunks and merged
on the server.
This mechanism is defined here:
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema
For ODK Aggregate, you don't need to specify the namespace, just have a
group in your form.
With ODK Collect 1.1.7 and later, the bind for the instanceID would be:
...
...
...
You can construct your own instanceID expressions. However, you should
avoid symbols and punctuation other than colons and dashes since the
parsing logic within Aggregate is likely fragile if you go wild with
punctuation (and that is used later on when retrieving images, repeat
groups, etc.). If you do construct your own values, they must be less than
80 characters long (a limitation of ODK Aggregate) and must be unique
across all devices and submissions.
Mitch
On Mon, Jun 4, 2012 at 3:42 PM, Kihara Absolomon soloincc@gmail.comwrote:
Hi Mitch,
Thanks for the heads up. Surprisingly I had never noticed this tab or
just didn't know its function prior to this! But we have some problems!
It seems that something weird happened during submissions. Under the
Submissions Admin Tab, I have 2 instances each of my un-submitted form. I
can see clearly that duplicate instances data is identical, down to the
auto inserted device-id, start-time and end-time. The only discrepancy
comes in the media data collected. Each instance is meant to have a minimum
of 5 photos and a no maximum, as there is a looping logic to add as many
photos as possible. These photos have been split between the duplicate
instances, in that some photos appear in one instance and not in the other
and vice versa. This is a perfect case of merging the duplicate instances.
I guess I have to do this at the DB level.
I don't know how many have experienced this scenario, but I think it
makes the case for the merge feature in case of in-complete submissions.
On another note, I tried using the "Manually Upload Submission Data"
function as a 2nd option, but it only offered 1 option of uploading the
media files. On using this feature, I ended up with an incomplete instance
again!
This can be solved by zipping all the media files and uploading them as
a zip file and doing the un-zipping and merging on the server side...just a
thought!
*
*
Regards
Kihara Absolomon
On Mon, Jun 4, 2012 at 9:05 PM, Mitch S mitchellsundt@gmail.com wrote:
On Aggregate, these submissions may have been determined to be
incomplete -- this occurs if the captured media attachments for the
filled-in form were not all successfully uploaded. You can look at
incomplete submissions on the Forms Management / Submission Admin tab. If
you "Mark As Complete" the form, that action will remove all traces of the
missing attachments. Alternatively, on that tab you can "Manually Upload
Submission Data" and navigate to the phone's /sdcard/odk/instances
directory, into the directory of the incomplete submission, and manually
upload the submission's xml file and missing media attachments to complete
the submission.
Once the submission is complete, it will move to the Submissions tab and
disappear from the Submission Admin tab.
If you are using an older version of ODK Collect, it may be that the
media file formats were not recognized by that older version (1.1.5 and
earlier).
Mitch
On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon soloincc@gmail.comwrote:
Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
that they were uploaded successfully. However on logging in to aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they are not
appearing as instances that can be uploaded. I can still see them under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
If you are trying to manually upload a submission to Aggregate then
assuming you have form permissions and are using the current version
of Aggregate.
Goto the "Form Management" Tab and the "Submission Admin" sub tab you
will see a button called "Manually Upload Submission Data". When you
click on that button a pop called 'Submission Upload' will appear.
On the 'Submission data file' point the browser at the submission xml
On the 'Associated data file(s):' point the browser at the media
files. NOTE: you can select multiple files at once using the
Associated data file file picker.
Waylon
···
On Mon, Jun 4, 2012 at 8:54 PM, Kihara Absolomon wrote:
> Wow! Thanks a lot for the pointers. The metadata thingie is soo cool. Thanks
> again.
>
> What do I do with the partially uploaded submissions? I have tried to zip an
> instance folder and uploading it but it fails. Zipping the photos on their
> own while the form is uploaded as a plain xml also fails!
>
> How should I upload these partial submissions?
>
> Regards
> Kihara Absolomon
>
>
>
> On Tue, Jun 5, 2012 at 2:50 AM, Mitch S wrote:
>>
>> The correct solution is to define an in your form.
>> That instanceID will be used to merge the multiple submissions necessary to
>> upload the data. The form submission is already effectively a zip of the
>> data. The problem comes when that data is very large. The larger it is,
>> the more likely the transmission will fail, and the more resources are
>> needed on the server to handle the data in one pass. By explicitly
>> providing an instanceID, the data can be sent in smaller chunks and merged
>> on the server.
>>
>> This mechanism is defined here:
>> https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema
>> For ODK Aggregate, you don't need to specify the namespace, just have a
>> group in your form.
>>
>> With ODK Collect 1.1.7 and later, the bind for the instanceID would be:
>>
>> ...
>>
>>
>>
>>
>>
>>
>> ...
>>
>>
>>
>> > calculate="concat('uuid:', uuid())"/>
>> ...
>>
>> You can construct your own instanceID expressions. However, you should
>> avoid symbols and punctuation other than colons and dashes since the parsing
>> logic within Aggregate is likely fragile if you go wild with punctuation
>> (and that is used later on when retrieving images, repeat groups, etc.). If
>> you do construct your own values, they must be less than 80 characters long
>> (a limitation of ODK Aggregate) and must be unique across all devices and
>> submissions.
>>
>> Mitch
>>
>> On Mon, Jun 4, 2012 at 3:42 PM, Kihara Absolomon wrote:
>>>
>>> Hi Mitch,
>>>
>>> Thanks for the heads up. Surprisingly I had never noticed this tab or
>>> just didn't know its function prior to this! But we have some problems!
>>>
>>> It seems that something weird happened during submissions. Under the
>>> Submissions Admin Tab, I have 2 instances each of my un-submitted form. I
>>> can see clearly that duplicate instances data is identical, down to the auto
>>> inserted device-id, start-time and end-time. The only discrepancy comes in
>>> the media data collected. Each instance is meant to have a minimum of 5
>>> photos and a no maximum, as there is a looping logic to add as many photos
>>> as possible. These photos have been split between the duplicate instances,
>>> in that some photos appear in one instance and not in the other and vice
>>> versa. This is a perfect case of merging the duplicate instances. I guess I
>>> have to do this at the DB level.
>>>
>>> I don't know how many have experienced this scenario, but I think it
>>> makes the case for the merge feature in case of in-complete submissions.
>>>
>>> On another note, I tried using the "Manually Upload Submission Data"
>>> function as a 2nd option, but it only offered 1 option of uploading the
>>> media files. On using this feature, I ended up with an incomplete instance
>>> again!
>>>
>>> This can be solved by zipping all the media files and uploading them as a
>>> zip file and doing the un-zipping and merging on the server side...just a
>>> thought!
>>>
>>> Regards
>>> Kihara Absolomon
>>>
>>>
>>>
>>> On Mon, Jun 4, 2012 at 9:05 PM, Mitch S wrote:
>>>>
>>>> On Aggregate, these submissions may have been determined to be
>>>> incomplete -- this occurs if the captured media attachments for the
>>>> filled-in form were not all successfully uploaded. You can look at
>>>> incomplete submissions on the Forms Management / Submission Admin tab. If
>>>> you "Mark As Complete" the form, that action will remove all traces of the
>>>> missing attachments. Alternatively, on that tab you can "Manually Upload
>>>> Submission Data" and navigate to the phone's /sdcard/odk/instances
>>>> directory, into the directory of the incomplete submission, and manually
>>>> upload the submission's xml file and missing media attachments to complete
>>>> the submission.
>>>>
>>>> Once the submission is complete, it will move to the Submissions tab and
>>>> disappear from the Submission Admin tab.
>>>>
>>>> If you are using an older version of ODK Collect, it may be that the
>>>> media file formats were not recognized by that older version (1.1.5 and
>>>> earlier).
>>>>
>>>> Mitch
>>>>
>>>>
>>>> On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I have some instances which were marked as finalized and ready to be
>>>>> uploaded to the server. During the upload process there was some sort of
>>>>> error and 6 of them weren't uploaded. I re-uploaded them and ODK reported
>>>>> that they were uploaded successfully. However on logging in to aggregate,
>>>>> the 6 instances are not reflecting in Aggregate and on Collect they are not
>>>>> appearing as instances that can be uploaded. I can still see them under the
>>>>> instances folder in the SD card.
>>>>>
>>>>> How do I re-upload them to the server?
>>>>>
>>>>> Regards
>>>>> Kihara Absolomon
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Mitch Sundt
>>>> Software Engineer
>>>> University of Washington
>>>> mitchellsundt@gmail.com
>>>
>>>
>>
>>
>>
>> --
>> Mitch Sundt
>> Software Engineer
>> University of Washington
>> mitchellsundt@gmail.com
>
>
As Waylon notes, if you still have the submission on the phone, you can
delete the two partial submissions on the Submission Admin tab, then
manually upload the submission through that UI. You can use Chrome or
Firefox to select all the media files and upload them in one step.
Internet Explorer is broken in that regard.
It is very difficult to merge these if you don't have the original
submission.
Mitch
···
On Mon, Jun 4, 2012 at 9:54 PM, W. Brunette wrote:
If you are trying to manually upload a submission to Aggregate then
assuming you have form permissions and are using the current version
of Aggregate.
Goto the "Form Management" Tab and the "Submission Admin" sub tab you
will see a button called "Manually Upload Submission Data". When you
click on that button a pop called 'Submission Upload' will appear.
On the 'Submission data file' point the browser at the submission xml
On the 'Associated data file(s):' point the browser at the media
files. NOTE: you can select multiple files at once using the
Associated data file file picker.
Waylon
On Mon, Jun 4, 2012 at 8:54 PM, Kihara Absolomon soloincc@gmail.com wrote:
Wow! Thanks a lot for the pointers. The metadata thingie is soo cool.
Thanks
again.
What do I do with the partially uploaded submissions? I have tried to
zip an
instance folder and uploading it but it fails. Zipping the photos on
their
own while the form is uploaded as a plain xml also fails!
How should I upload these partial submissions?
Regards
Kihara Absolomon
On Tue, Jun 5, 2012 at 2:50 AM, Mitch S mitchellsundt@gmail.com wrote:
The correct solution is to define an in your form.
That instanceID will be used to merge the multiple submissions
necessary to
upload the data. The form submission is already effectively a zip of
the
data. The problem comes when that data is very large. The larger it
is,
the more likely the transmission will fail, and the more resources are
needed on the server to handle the data in one pass. By explicitly
providing an instanceID, the data can be sent in smaller chunks and
merged
on the server.
This mechanism is defined here:
https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema
For ODK Aggregate, you don't need to specify the namespace, just have a
group in your form.
With ODK Collect 1.1.7 and later, the bind for the instanceID would be:
...
...
...
You can construct your own instanceID expressions. However, you should
avoid symbols and punctuation other than colons and dashes since the
parsing
logic within Aggregate is likely fragile if you go wild with punctuation
(and that is used later on when retrieving images, repeat groups,
etc.). If
you do construct your own values, they must be less than 80 characters
long
(a limitation of ODK Aggregate) and must be unique across all devices
and
submissions.
Mitch
On Mon, Jun 4, 2012 at 3:42 PM, Kihara Absolomon soloincc@gmail.com wrote:
Hi Mitch,
Thanks for the heads up. Surprisingly I had never noticed this tab or
just didn't know its function prior to this! But we have some problems!
It seems that something weird happened during submissions. Under the
Submissions Admin Tab, I have 2 instances each of my un-submitted
form. I
can see clearly that duplicate instances data is identical, down to
the auto
inserted device-id, start-time and end-time. The only discrepancy
comes in
the media data collected. Each instance is meant to have a minimum of 5
photos and a no maximum, as there is a looping logic to add as many
photos
as possible. These photos have been split between the duplicate
instances,
in that some photos appear in one instance and not in the other and
vice
versa. This is a perfect case of merging the duplicate instances. I
guess I
have to do this at the DB level.
I don't know how many have experienced this scenario, but I think it
makes the case for the merge feature in case of in-complete
submissions.
On another note, I tried using the "Manually Upload Submission Data"
function as a 2nd option, but it only offered 1 option of uploading the
media files. On using this feature, I ended up with an incomplete
instance
again!
This can be solved by zipping all the media files and uploading them
as a
zip file and doing the un-zipping and merging on the server
side...just a
thought!
Regards
Kihara Absolomon
On Mon, Jun 4, 2012 at 9:05 PM, Mitch S mitchellsundt@gmail.com wrote:
On Aggregate, these submissions may have been determined to be
incomplete -- this occurs if the captured media attachments for the
filled-in form were not all successfully uploaded. You can look at
incomplete submissions on the Forms Management / Submission Admin
tab. If
you "Mark As Complete" the form, that action will remove all traces
of the
missing attachments. Alternatively, on that tab you can "Manually
Upload
Submission Data" and navigate to the phone's /sdcard/odk/instances
directory, into the directory of the incomplete submission, and
manually
upload the submission's xml file and missing media attachments to
complete
the submission.
Once the submission is complete, it will move to the Submissions tab
and
disappear from the Submission Admin tab.
If you are using an older version of ODK Collect, it may be that the
media file formats were not recognized by that older version (1.1.5
and
earlier).
Mitch
On Mon, Jun 4, 2012 at 8:42 AM, Kihara Absolomon soloincc@gmail.com wrote:
Hi all,
I have some instances which were marked as finalized and ready to be
uploaded to the server. During the upload process there was some
sort of
error and 6 of them weren't uploaded. I re-uploaded them and ODK
reported
that they were uploaded successfully. However on logging in to
aggregate,
the 6 instances are not reflecting in Aggregate and on Collect they
are not
appearing as instances that can be uploaded. I can still see them
under the
instances folder in the SD card.
How do I re-upload them to the server?
Regards
Kihara Absolomon
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com