Integrating a new server with ODK Collect

Support
1

Hi folks,

Many thanks to Carl who was gracious enough to tolerate me expressing my
annoyances while trying to hook up ODK into my own server. Here is a FAQ
that I hope will be useful if anyone has to go through this again. It's been
delivered with a side of snark, but I really do think that ODK is a great
product that more than anything else suffered from a lack of documentation,
especially in the realm of standards and integration. Hopefully the below
will help towards that path...

cheers,
Cory

··· ---- 1. What URL should I use and how should I configure it?

Your URL can be anything, as long as it ends in /submission. In the phone
you should put the root of everything before submission, and ODK will
happily add /submission for you. You don't even have to tell it to! And in
fact, you can't tell it to without editing code and compiling ODK yourself.
So...

You put in phone: http://myserver.com/odk (note no trailing slash, this is
important)
Your server accepts posts at: http://myserver.com/odk/submission.

In django you can work around this pretty easily by redirecting both to the
same view. e.g.

urlpatterns = patterns('myapp.views',
url(r'^$', 'post', name='receiver_post'),
url(r'^/submission$', 'post', name="receiver_odk_post"),
....

--
2. How do I access the form data?

The form is submitted as a multipart POST. The xml formdata is in a part
named "xml_submission_file". To my knowledge this cannot be changed or
configured, but this isn't took big a problem. In django you can access it
like so:

xml_instance = request.FILES['xml_submission_file'].read()

--
3. What about multimedia?

Again these are just named parts. Again, in django each part will correspond
everything else in request.FILES, with the keys being the filenames and the
values being the handles to the file contents.

--
4. How do I respond?

However you want! Well almost.

You can put anything in the body.
Your status code MUST be 201 or ODK thinks it failed.
You MUST add a "Location" header
This location header MUST be part of the url that ODK submitted to. (If
you're sneaky you can always just set the Location header to "h", "t", "p",
":", or ".", but you should probably use the real server location in case
ODK decides to change this on you.)

Again, in django the easiest way to do this is the following:

response = HttpResponse("thanks ODK!", status=201)
response["Location"] = "." # but really use something like "http://%s" %
Site.objects.get_current().domain
return response

--
Cory L. Zue
Dimagi, Inc

2

Hi Cory,

Thanks, great info.

I tried to start a page for the wiki on this last year:

https://docs.google.com/Doc?docid=0Acr_BkmfRmy-ZGN6aHZxcW1fMjE3ZDVka3FiYzg&hl=en

It didn't get much traction, but perhaps with this additional information it
could be more useful.

Best,
--Michael

··· On Thu, Feb 10, 2011 at 2:37 AM, Cory Zue wrote:

Hi folks,

Many thanks to Carl who was gracious enough to tolerate me expressing my
annoyances while trying to hook up ODK into my own server. Here is a FAQ
that I hope will be useful if anyone has to go through this again. It's been
delivered with a side of snark, but I really do think that ODK is a great
product that more than anything else suffered from a lack of documentation,
especially in the realm of standards and integration. Hopefully the below
will help towards that path...

cheers,
Cory

  1. What URL should I use and how should I configure it?

Your URL can be anything, as long as it ends in /submission. In the phone
you should put the root of everything before submission, and ODK will
happily add /submission for you. You don't even have to tell it to! And in
fact, you can't tell it to without editing code and compiling ODK yourself.
So...

You put in phone: http://myserver.com/odk (note no trailing slash, this is
important)
Your server accepts posts at: http://myserver.com/odk/submission.

In django you can work around this pretty easily by redirecting both to the
same view. e.g.

urlpatterns = patterns('myapp.views',
url(r'^$', 'post', name='receiver_post'),
url(r'^/submission$', 'post', name="receiver_odk_post"),
....

--
2. How do I access the form data?

The form is submitted as a multipart POST. The xml formdata is in a part
named "xml_submission_file". To my knowledge this cannot be changed or
configured, but this isn't took big a problem. In django you can access it
like so:

xml_instance = request.FILES['xml_submission_file'].read()

--
3. What about multimedia?

Again these are just named parts. Again, in django each part will
correspond everything else in request.FILES, with the keys being the
filenames and the values being the handles to the file contents.

--
4. How do I respond?

However you want! Well almost.

You can put anything in the body.
Your status code MUST be 201 or ODK thinks it failed.
You MUST add a "Location" header
This location header MUST be part of the url that ODK submitted to. (If
you're sneaky you can always just set the Location header to "h", "t", "p",
":", or ".", but you should probably use the real server location in case
ODK decides to change this on you.)

Again, in django the easiest way to do this is the following:

response = HttpResponse("thanks ODK!", status=201)
response["Location"] = "." # but really use something like "http://%s" %
Site.objects.get_current().domain
return response

--
Cory L. Zue
Dimagi, Inc
http://www.dimagi.com/

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

3

Hmmm. I (clearly don't know what I'm doing but...) before seeing this I
tried to access my server as http://myserver.com/test.php where the first
thing that test.php does is add an entry in a table saying it was started.
Then it stores all the POST variables in the table. Well when attempting to
send data it (apparently) attempted because test.php registered that access
in the database but there were no POST variables. Is a multipart POST not
accessed the same way as other POSTed values? And if it appends
'submission' if its not included why did my test.php file start up?

-steve-

··· On Wed, Feb 9, 2011 at 3:37 PM, Cory Zue wrote:

Hi folks,

Many thanks to Carl who was gracious enough to tolerate me expressing my
annoyances while trying to hook up ODK into my own server. Here is a FAQ
that I hope will be useful if anyone has to go through this again. It's been
delivered with a side of snark, but I really do think that ODK is a great
product that more than anything else suffered from a lack of documentation,
especially in the realm of standards and integration. Hopefully the below
will help towards that path...

cheers,
Cory

  1. What URL should I use and how should I configure it?

Your URL can be anything, as long as it ends in /submission. In the phone
you should put the root of everything before submission, and ODK will
happily add /submission for you. You don't even have to tell it to! And in
fact, you can't tell it to without editing code and compiling ODK yourself.
So...

You put in phone: http://myserver.com/odk (note no trailing slash, this is
important)
Your server accepts posts at: http://myserver.com/odk/submission.

In django you can work around this pretty easily by redirecting both to the
same view. e.g.

urlpatterns = patterns('myapp.views',
url(r'^$', 'post', name='receiver_post'),
url(r'^/submission$', 'post', name="receiver_odk_post"),
....

--
2. How do I access the form data?

The form is submitted as a multipart POST. The xml formdata is in a part
named "xml_submission_file". To my knowledge this cannot be changed or
configured, but this isn't took big a problem. In django you can access it
like so:

xml_instance = request.FILES['xml_submission_file'].read()

--
3. What about multimedia?

Again these are just named parts. Again, in django each part will
correspond everything else in request.FILES, with the keys being the
filenames and the values being the handles to the file contents.

--
4. How do I respond?

However you want! Well almost.

You can put anything in the body.
Your status code MUST be 201 or ODK thinks it failed.
You MUST add a "Location" header
This location header MUST be part of the url that ODK submitted to. (If
you're sneaky you can always just set the Location header to "h", "t", "p",
":", or ".", but you should probably use the real server location in case
ODK decides to change this on you.)

Again, in django the easiest way to do this is the following:

response = HttpResponse("thanks ODK!", status=201)
response["Location"] = "." # but really use something like "http://%s" %
Site.objects.get_current().domain
return response

--
Cory L. Zue
Dimagi, Inc
http://www.dimagi.com/

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

4

Perhaps someone else who has developed an alternative server can help you.

The submissions interaction is described here:
https://bitbucket.org/javarosa/javarosa/wiki/FormSubmissionAPI

See 'Extended Transmission Considerations' for a description of the use of
a HEAD request prior to the submission body. ODK Collect expects a 204
return code from the HEAD request. This may be your difficulty.

Mitch

··· On Thu, Nov 22, 2012 at 8:31 AM, wrote:

On Wednesday, February 9, 2011 11:37:54 PM UTC, Cory Zue wrote:

Hi folks,

Many thanks to Carl who was gracious enough to tolerate me expressing my
annoyances while trying to hook up ODK into my own server. Here is a FAQ
that I hope will be useful if anyone has to go through this again. It's
been delivered with a side of snark, but I really do think that ODK is a
great product that more than anything else suffered from a lack of
documentation, especially in the realm of standards and integration.
Hopefully the below will help towards that path...

cheers,
Cory

  1. What URL should I use and how should I configure it?

Your URL can be anything, as long as it ends in /submission. In the
phone you should put the root of everything before submission, and ODK
will happily add /submission for you. You don't even have to tell it to!
And in fact, you can't tell it to without editing code and compiling ODK
yourself. So...

You put in phone: http://myserver.com/odk (note no trailing slash, this
is important)
Your server accepts posts at: http://myserver.com/odk/submission.

In django you can work around this pretty easily by redirecting both to
the same view. e.g.

urlpatterns = patterns('myapp.views',
url(r'^$', 'post', name='receiver_post'),

url(r'^/submission$',  'post', name="receiver_odk_post"),
....

--
2. How do I access the form data?

The form is submitted as a multipart POST. The xml formdata is in a part
named "xml_submission_file". To my knowledge this cannot be changed or
configured, but this isn't took big a problem. In django you can access it
like so:

xml_instance = request.FILES['xml_submission_file'].read()

--
3. What about multimedia?

Again these are just named parts. Again, in django each part will
correspond everything else in request.FILES, with the keys being the
filenames and the values being the handles to the file contents.

--
4. How do I respond?

However you want! Well almost.

You can put anything in the body.
Your status code MUST be 201 or ODK thinks it failed.

You MUST add a "Location" header
This location header MUST be part of the url that ODK submitted to. (If
you're sneaky you can always just set the Location header to "h", "t", "p",
":", or ".", but you should probably use the real server location in case
ODK decides to change this on you.)

Again, in django the easiest way to do this is the following:

response = HttpResponse("thanks ODK!", status=201)
response["Location"] = "." # but really use something like "http://%s"
% Site.objects.get_current().domain

return response

--
Cory L. Zue
Dimagi, Inc
http://www.dimagi.com/

Hi guys,

I'm trying to process my ODK submissions, I can't access any data sent by
ODK e.g.

$ImportLog = ClassRegistry::init('ImportLog');
$ImportLog->create;
if (!isset($HTTP_RAW_POST_DATA))
$HTTP_RAW_POST_DATA =
file_get_contents("php://input");
$import_log = array('user_id' =>
$_SESSION['Auth']['User']['id'],'extra_info'=>$HTTP_RAW_POST_DATA);

This kind of thing used to work! I can't get to any variables, I can see
the request coming in:

array (
'REDIRECT_REDIRECT_STATUS' => '200',
'REDIRECT_STATUS' => '200',
'HTTP_X_OPENROSA_VERSION' => '1.0',
'HTTP_DATE' => 'Thu, 22 Nov 2012 04:27:28 GMT+00:00',
'HTTP_HOST' => 'myurl.com',
'HTTP_CONNECTION' => 'Keep-Alive',
'HTTP_COOKIE' => 'CAKEPHP=session; CakeCookie[company_id]=company_id',
'PATH' => '/sbin:/usr/sbin:/bin:/usr/bin',
'SERVER_SIGNATURE' => 'Apache/2.2.3 (Red Hat) Server at
myurl.com Port 80
',
'SERVER_SOFTWARE' => 'Apache/2.2.3 (Red Hat)',
'SERVER_NAME' => 'myurl.com',
'SERVER_ADDR' => '192.168.100.2',
'SERVER_PORT' => '80',
'REMOTE_ADDR' => 'server_ip',
'DOCUMENT_ROOT' => '/var/www/html/miles/',
'SERVER_ADMIN' => 'root@localhost',
'SCRIPT_FILENAME' => '/var/www/html/miles/webroot/index.php',
'REMOTE_PORT' => '55503',
'REDIRECT_QUERY_STRING' =>
'url=users/xml_login/company_id/someauthstring/display/inspections_audit_due_this_week_xml/submission&deviceID=imei%device',
'REDIRECT_URL' =>
'/webroot/users/xml_login/company_id/someauthstring/display/inspections_audit_due_this_week_xml/submission',
'GATEWAY_INTERFACE' => 'CGI/1.1',
'SERVER_PROTOCOL' => 'HTTP/1.1',
'REQUEST_METHOD' => 'HEAD',
'QUERY_STRING' =>
'url=users/xml_login/company_id/someauthstring/display/inspections_audit_due_this_week_xml/submission&deviceID=imei%device',
'REQUEST_URI' =>
'/users/xml_login/company_id/someauthstring/display/inspections_audit_due_this_week_xml/submission?deviceID=imei%device',
'SCRIPT_NAME' => '/webroot/index.php',
'PHP_SELF' => '/webroot/index.php',
'REQUEST_TIME' => 1353601645,
)

Does ODK only POST data after it's had it's handshake with the server i.e.
sucessfuly seen the two headers? I can't seem to get that right either...

$url = "http://";
$extra_info.=$url;
header("Status 201 OK",false,201);
header('Location: http://',false);
$this->layout = false;
$this->autoRender = false;

No matter what I set the url to!

Any help appreciated! Thanks.

--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com