Web interface to odk collect

Hi,
I'm working on a project that captures various surveillance data, I have
opted to use ODK. I have installed a local instance of ODKAggregate and
have people in field with ODKcollect, the setup works fine.

Tricky part is, In addition to have people sending data via odk collect to
odk aggregate, some collectors are stationary, and I will want them to send
data via a web interface. I have been struggling to build the interface,
but I have finally accomplished

  1. Automatic html form generation via details in _form_data_model table
    (was unable to process xform via _form_info_xform_blb)

Now the problem comes in submitting this to ODKAggregate what I have done
is following

  1. Created and XML representation of the html form data (root element being
    - excluding the _core)
  2. used libcurl library and set url to
    http://myserver:8080/ODKAggregate/submission
  3. Enabled anonymous data collection

Problem is submission reply(curl) is successfull, but I don't see data in
the table, and I have no idea where the logfiles for aggregate are located

May anyone point me in the right direction ??

First, I trust that you tried refreshing or navigating off of and then back
onto your Submissions tab to see if the record would appear. ODK Aggregate
refreshes that tab every 6 seconds if the site is being actively used, but,
if idle, will not refresh the display (to reduce AppEngine charges).

Does your filled-in form have any audio, video or image attachments?

If so, the submission may be tagged as incomplete, and would appear under
the Form Management / Submission Admin tab until those attachments are also
submitted (along with the submission, as additional file attachments in the
post).

If you are using an instanceId in your form, subsequent submissions will
not update any existing submission for that instanceId (only the first
upload alters the database). So you would not see any new submissions in
your table, or any value changes to the existing submission with the same
instanceId.

I recommend writing the filled-in form to a file. You can then compare the
format of that file to a filled-in form submission file produced by ODK
Collect (under /sdcard/odk/instances/). Are the tags identical?

From the code, the order of the tags in the XML is not required to exactly
match that of the form definition, but, if you have trouble, you might also
try placing them in the exact order of the form definition. Note that the
top level tag must match the top-level element of the
block submitted to ODK Aggregate. If it doesn't, I believe the form will
"submit" but show as an empty record on ODK Aggregate (but it sounds like
you are not seeing any submission, even a blank one, appear on ODK
Aggregate).

Once you have the filled-in form saved to a file, you can manually submit
it to ODK Aggregate on the Form Management / Submission Admin tab, to
isolate whether the "success" is an issue with the libcurl POST somehow not
reporting an error back to your app, or if there is further debugging to do
inside ODK Aggregate.

How are you retrieving the prompts for the survey questions?

If I were doing this, I would access the form definition file ( in the _blb
) via ODK Aggregate's

Form f = FormFactory.retrieveFormByFormId( id, cc);
String str = f.getFormXml();

and you could then parse that with the javarosa parser by deriving from
BaseFormParserForJavarosa and access the questions and render the web page
by traversing the Javarosa FormDef.

Or, if you are not modifying ODK Aggregate, but working with a separate
program, you can retrieve the download URL for the XML via either

http://opendatakit.appspot.com/xformsList (this is a debugging interface)
or, if you follow OpenRosa spec, the
http://opendatakit.appspot.com/formListinterface (same as ODK Collect)

··· --------------- If your code can handle any form without repeats (or even those with repeats), if you can contribute it back to the project, I'd like to discuss how that could be done.

Mitch

On Mon, Aug 13, 2012 at 2:36 AM, Eric Beda eric.beda@sacids.org wrote:

Hi,
I'm working on a project that captures various surveillance data, I have
opted to use ODK. I have installed a local instance of ODKAggregate and
have people in field with ODKcollect, the setup works fine.

Tricky part is, In addition to have people sending data via odk collect to
odk aggregate, some collectors are stationary, and I will want them to send
data via a web interface. I have been struggling to build the interface,
but I have finally accomplished

  1. Automatic html form generation via details in _form_data_model table
    (was unable to process xform via _form_info_xform_blb)

Now the problem comes in submitting this to ODKAggregate what I have done
is following

  1. Created and XML representation of the html form data (root element
    being - excluding the _core)
  2. used libcurl library and set url to
    http://myserver:8080/ODKAggregate/submission
  3. Enabled anonymous data collection

Problem is submission reply(curl) is successfull, but I don't see data in
the table, and I have no idea where the logfiles for aggregate are located

May anyone point me in the right direction ??

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Eric,

Dimagi's touchforms does a nice job handling this.

We've integrated it into formhub. Here's a sample form that submits back to
the form in formhub.

http://formhub.org/mberg/forms/hh_polio_survey/enter-data

You can run both formhub and touchforms locally if needed.

Thanks,

Matt

··· On Mon, Aug 13, 2012 at 1:16 PM, Mitch S wrote:

First, I trust that you tried refreshing or navigating off of and then
back onto your Submissions tab to see if the record would appear. ODK
Aggregate refreshes that tab every 6 seconds if the site is being actively
used, but, if idle, will not refresh the display (to reduce AppEngine
charges).

Does your filled-in form have any audio, video or image attachments?

If so, the submission may be tagged as incomplete, and would appear under
the Form Management / Submission Admin tab until those attachments are also
submitted (along with the submission, as additional file attachments in the
post).

If you are using an instanceId in your form, subsequent submissions will
not update any existing submission for that instanceId (only the first
upload alters the database). So you would not see any new submissions in
your table, or any value changes to the existing submission with the same
instanceId.

I recommend writing the filled-in form to a file. You can then compare the
format of that file to a filled-in form submission file produced by ODK
Collect (under /sdcard/odk/instances/). Are the tags identical?

From the code, the order of the tags in the XML is not required to exactly
match that of the form definition, but, if you have trouble, you might also
try placing them in the exact order of the form definition. Note that the
top level tag must match the top-level element of the
block submitted to ODK Aggregate. If it doesn't, I believe the form will
"submit" but show as an empty record on ODK Aggregate (but it sounds like
you are not seeing any submission, even a blank one, appear on ODK
Aggregate).

Once you have the filled-in form saved to a file, you can manually submit
it to ODK Aggregate on the Form Management / Submission Admin tab, to
isolate whether the "success" is an issue with the libcurl POST somehow not
reporting an error back to your app, or if there is further debugging to do
inside ODK Aggregate.

How are you retrieving the prompts for the survey questions?

If I were doing this, I would access the form definition file ( in the
_blb ) via ODK Aggregate's

Form f = FormFactory.retrieveFormByFormId( id, cc);
String str = f.getFormXml();

and you could then parse that with the javarosa parser by deriving from
BaseFormParserForJavarosa and access the questions and render the web page
by traversing the Javarosa FormDef.

Or, if you are not modifying ODK Aggregate, but working with a separate
program, you can retrieve the download URL for the XML via either

http://opendatakit.appspot.com/xformsList (this is a debugging interface)
or, if you follow OpenRosa spec, the
http://opendatakit.appspot.com/formList interface (same as ODK Collect)


If your code can handle any form without repeats (or even those with
repeats), if you can contribute it back to the project, I'd like to discuss
how that could be done.

Mitch

On Mon, Aug 13, 2012 at 2:36 AM, Eric Beda eric.beda@sacids.org wrote:

Hi,
I'm working on a project that captures various surveillance data, I have
opted to use ODK. I have installed a local instance of ODKAggregate and
have people in field with ODKcollect, the setup works fine.

Tricky part is, In addition to have people sending data via odk collect
to odk aggregate, some collectors are stationary, and I will want them to
send data via a web interface. I have been struggling to build the
interface, but I have finally accomplished

  1. Automatic html form generation via details in _form_data_model table
    (was unable to process xform via _form_info_xform_blb)

Now the problem comes in submitting this to ODKAggregate what I have done
is following

  1. Created and XML representation of the html form data (root element
    being - excluding the _core)
  2. used libcurl library and set url to
    http://myserver:8080/ODKAggregate/submission
  3. Enabled anonymous data collection

Problem is submission reply(curl) is successfull, but I don't see data in
the table, and I have no idea where the logfiles for aggregate are located

May anyone point me in the right direction ??

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

Hi Mitch,

Initial testing, forms do not have audio, video or image attachments, I've
stayed clear of them until i get the basics working

My ODKAggregate is on a LAMP setup, thus after submitting I check the db
table via phpmyadmin, and no new entries appear

I did write the filled in form into a file (displayed it in the browser)
and its identical to the one in instances (under odk/forms/instances). But
I am not sure on the id i.e. what I place as ID is the table
name excluding the _core is that correct ??

The initial idea was to retrieve form definitions from the _blb, but I have
been having problems accessing the blob within code via PHP, apparently
working with blobs in php is a bit of a mission but will get my hands
around it

Update:
Just tested, the form submission (pne that i wrote to file) via web
interface, it didn't work,,, apparently the form id was slightly
different,,, will correct and test again,,,,,

··· On Monday, August 13, 2012 8:16:38 PM UTC+3, Mitch wrote: > > First, I trust that you tried refreshing or navigating off of and then > back onto your Submissions tab to see if the record would appear. ODK > Aggregate refreshes that tab every 6 seconds if the site is being actively > used, but, if idle, will not refresh the display (to reduce AppEngine > charges). > > Does your filled-in form have any audio, video or image attachments? > > If so, the submission may be tagged as incomplete, and would appear under > the Form Management / Submission Admin tab until those attachments are also > submitted (along with the submission, as additional file attachments in the > post). > > If you are using an instanceId in your form, subsequent submissions will > not update any existing submission for that instanceId (only the first > upload alters the database). So you would not see any new submissions in > your table, or any value changes to the existing submission with the same > instanceId. > > I recommend writing the filled-in form to a file. You can then compare the > format of that file to a filled-in form submission file produced by ODK > Collect (under /sdcard/odk/instances/). Are the tags identical? >

From the code, the order of the tags in the XML is not required to exactly
match that of the form definition, but, if you have trouble, you might also
try placing them in the exact order of the form definition. Note that the
top level tag must match the top-level element of the
block submitted to ODK Aggregate. If it doesn't, I believe the form will
"submit" but show as an empty record on ODK Aggregate (but it sounds like
you are not seeing any submission, even a blank one, appear on ODK
Aggregate).

Once you have the filled-in form saved to a file, you can manually submit
it to ODK Aggregate on the Form Management / Submission Admin tab, to
isolate whether the "success" is an issue with the libcurl POST somehow not
reporting an error back to your app, or if there is further debugging to do
inside ODK Aggregate.

How are you retrieving the prompts for the survey questions?

If I were doing this, I would access the form definition file ( in the
_blb ) via ODK Aggregate's

Form f = FormFactory.retrieveFormByFormId( id, cc);
String str = f.getFormXml();

and you could then parse that with the javarosa parser by deriving from
BaseFormParserForJavarosa and access the questions and render the web page
by traversing the Javarosa FormDef.

Or, if you are not modifying ODK Aggregate, but working with a separate
program, you can retrieve the download URL for the XML via either

http://opendatakit.appspot.com/xformsList (this is a debugging interface)
or, if you follow OpenRosa spec, the
http://opendatakit.appspot.com/formList interface (same as ODK Collect)


If your code can handle any form without repeats (or even those with
repeats), if you can contribute it back to the project, I'd like to discuss
how that could be done.

Mitch

On Mon, Aug 13, 2012 at 2:36 AM, Eric Beda <eric...@sacids.org<javascript:> wrote:

Hi,
I'm working on a project that captures various surveillance data, I have
opted to use ODK. I have installed a local instance of ODKAggregate and
have people in field with ODKcollect, the setup works fine.

Tricky part is, In addition to have people sending data via odk collect
to odk aggregate, some collectors are stationary, and I will want them to
send data via a web interface. I have been struggling to build the
interface, but I have finally accomplished

  1. Automatic html form generation via details in _form_data_model table
    (was unable to process xform via _form_info_xform_blb)

Now the problem comes in submitting this to ODKAggregate what I have done
is following

  1. Created and XML representation of the html form data (root element
    being - excluding the _core)
  2. used libcurl library and set url to
    http://myserver:8080/ODKAggregate/submission
  3. Enabled anonymous data collection

Problem is submission reply(curl) is successfull, but I don't see data in
the table, and I have no idea where the logfiles for aggregate are located

May anyone point me in the right direction ??

--
Post: opend...@googlegroups.com <javascript:>
Unsubscribe: opendatakit...@googlegroups.com <javascript:>
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com <javascript:>