Feature Request for Collect - update to HttpsURLConnection for SNI?

Development
1

Hi Group:
I've been working for a while now with Collect, using it in conjunction
with custom back-end services running on non-Aggregate web hosts, with form
downloading and uploading working over secure SSL connections.
It all works nicely, but increasingly I'm finding that budget web hosting
provides SSL connections using shared IP addresses, which may be
accommodating multiple SSL certs. Browser and other app clients don't mind
this because they use SNI
(https://en.wikipedia.org/wiki/Server_Name_Indication), but Collect doesn't
use SNI - it uses an older http connection class which doesn't support
SNI. Without SNI, the server doesn't know which cert to present in
response to SSL connection requests, and so Collect receives a cert which
doesn't match the requested hostname - so the connection fails.
Would the maintainers consider a RFC to update the http/s mechanism used in
Collect to one which supports SNI? I understand this could be done using the
HttpsURLConnection class - I'm not knowledgeable enough to implement it
myself, but I think it would be a beneficial update.
Grateful for any thoughts on this.
Thanks
Nik

2

Hi Nik,

I've filed it at https://github.com/opendatakit/collect/issues/199 as
a feature request. What would be ideal is that those folks who built
the custom back-ends take the lead on this. Have you run this by them?

Yaw

··· -- Need ODK consultants? Nafundi provides form design, server setup, in-field training, and software development for ODK. Go to https://nafundi.com to get started.

On Tue, Sep 6, 2016 at 5:27 PM, blitheringeejit via ODK Developers opendatakit-developers@googlegroups.com wrote:

Hi Group:
I've been working for a while now with Collect, using it in conjunction with
custom back-end services running on non-Aggregate web hosts, with form
downloading and uploading working over secure SSL connections.
It all works nicely, but increasingly I'm finding that budget web hosting
provides SSL connections using shared IP addresses, which may be
accommodating multiple SSL certs. Browser and other app clients don't mind
this because they use SNI
(https://en.wikipedia.org/wiki/Server_Name_Indication), but Collect doesn't
use SNI - it uses an older http connection class which doesn't support SNI.
Without SNI, the server doesn't know which cert to present in response to
SSL connection requests, and so Collect receives a cert which doesn't match
the requested hostname - so the connection fails.
Would the maintainers consider a RFC to update the http/s mechanism used in
Collect to one which supports SNI? I understand this could be done using
the HttpsURLConnection class - I'm not knowledgeable enough to implement it
myself, but I think it would be a beneficial update.
Grateful for any thoughts on this.
Thanks
Nik

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Yaw - thanks for filing this.

I'm not sure what you mean by "those folks who built the custom back-ends"

  • unless you mean folks like me, because that's what I've done! But there
    isn't any solution for this issue in back-end code. Without an
    SNI-compatible connection, the Collect requests never make it to the
    back-end service - they are kiled by Collect at the TLS connection stage,
    before the request is sent.

I've made a few minor changes to Collect in my own fork, but I'm really not
knowledgeable enough about Android to be able to unpick and modify the http
connection code - especially as there are several different connections at
different points in the app. I'm happy to contribute to the project in
areas I know about (I've done a lot with PHP/MySQL backends for Collect)
but I'm really not a Java programmer, and the Collect code is mostly beyond
my understanding.

I only submitted the request because I thought quite a lot of Collect users
might want secure connections to custom back-ends, and not supporting SNI
at the connection level makes this harder to do with budget hosting and DIY
projects.

Thanks
Nik

··· On Wednesday, September 7, 2016 at 4:37:37 PM UTC+1, Yaw Anokwa wrote: > > Hi Nik, > > I've filed it at https://github.com/opendatakit/collect/issues/199 as > a feature request. What would be ideal is that those folks who built > the custom back-ends take the lead on this. Have you run this by them? > > Yaw > -- > Need ODK consultants? Nafundi provides form design, server setup, > in-field training, and software development for ODK. Go to > https://nafundi.com to get started. > > On Tue, Sep 6, 2016 at 5:27 PM, blitheringeejit via ODK Developers <opendatakit...@googlegroups.com > wrote: > > Hi Group: > > I've been working for a while now with Collect, using it in conjunction > with > > custom back-end services running on non-Aggregate web hosts, with form > > downloading and uploading working over secure SSL connections. > > It all works nicely, but increasingly I'm finding that budget web > hosting > > provides SSL connections using shared IP addresses, which may be > > accommodating multiple SSL certs. Browser and other app clients don't > mind > > this because they use SNI > > (https://en.wikipedia.org/wiki/Server_Name_Indication), but Collect > doesn't > > use SNI - it uses an older http connection class which doesn't support > SNI. > > Without SNI, the server doesn't know which cert to present in response > to > > SSL connection requests, and so Collect receives a cert which doesn't > match > > the requested hostname - so the connection fails. > > Would the maintainers consider a RFC to update the http/s mechanism used > in > > Collect to one which supports SNI? I understand this could be done > using > > the HttpsURLConnection class - I'm not knowledgeable enough to > implement it > > myself, but I think it would be a beneficial update. > > Grateful for any thoughts on this. > > Thanks > > Nik > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "ODK Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to opendatakit-developers+unsubscribe@googlegroups.com > . > > For more options, visit https://groups.google.com/d/optout. >
4

Hi Nik,

I think what Yaw might be indicating here:

  • Those developing custom back-ends have likely already hit this with
    Collect
  • They likely can easily confirm whether any change to support SNI is
    working or not
  • They already have a solution and perhaps could contribute it

I took a quick peek at what this and it appears that SNI is supported with
HttpsUrlConnection since Android 2.3. However, Collect uses what appears to
be a custom fork of Apache HttpClient.

Brent

··· On Thu, Sep 8, 2016 at 11:29 AM, blitheringeejit via ODK Developers < opendatakit-developers@googlegroups.com> wrote:

Hi Yaw - thanks for filing this.

I'm not sure what you mean by "those folks who built the custom back-ends"

  • unless you mean folks like me, because that's what I've done! But there
    isn't any solution for this issue in back-end code. Without an
    SNI-compatible connection, the Collect requests never make it to the
    back-end service - they are kiled by Collect at the TLS connection stage,
    before the request is sent.

I've made a few minor changes to Collect in my own fork, but I'm really
not knowledgeable enough about Android to be able to unpick and modify the
http connection code - especially as there are several different
connections at different points in the app. I'm happy to contribute to the
project in areas I know about (I've done a lot with PHP/MySQL backends for
Collect) but I'm really not a Java programmer, and the Collect code is
mostly beyond my understanding.

I only submitted the request because I thought quite a lot of Collect
users might want secure connections to custom back-ends, and not supporting
SNI at the connection level makes this harder to do with budget hosting and
DIY projects.

Thanks
Nik

On Wednesday, September 7, 2016 at 4:37:37 PM UTC+1, Yaw Anokwa wrote:

Hi Nik,

I've filed it at https://github.com/opendatakit/collect/issues/199 as
a feature request. What would be ideal is that those folks who built
the custom back-ends take the lead on this. Have you run this by them?

Yaw

Need ODK consultants? Nafundi provides form design, server setup,
in-field training, and software development for ODK. Go to
https://nafundi.com to get started.

On Tue, Sep 6, 2016 at 5:27 PM, blitheringeejit via ODK Developers opendatakit...@googlegroups.com wrote:

Hi Group:
I've been working for a while now with Collect, using it in conjunction
with
custom back-end services running on non-Aggregate web hosts, with form
downloading and uploading working over secure SSL connections.
It all works nicely, but increasingly I'm finding that budget web
hosting
provides SSL connections using shared IP addresses, which may be
accommodating multiple SSL certs. Browser and other app clients don't
mind
this because they use SNI
(https://en.wikipedia.org/wiki/Server_Name_Indication), but Collect
doesn't
use SNI - it uses an older http connection class which doesn't support
SNI.
Without SNI, the server doesn't know which cert to present in response
to
SSL connection requests, and so Collect receives a cert which doesn't
match
the requested hostname - so the connection fails.
Would the maintainers consider a RFC to update the http/s mechanism
used in
Collect to one which supports SNI? I understand this could be done
using
the HttpsURLConnection class - I'm not knowledgeable enough to
implement it
myself, but I think it would be a beneficial update.
Grateful for any thoughts on this.
Thanks
Nik

--
You received this message because you are subscribed to the Google
Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

*Please verify that you are using the latest version of ODK Collect. *

ODK Collect 1.4.9 is using Apache HTTP Client 4.5.2.
Earlier versions were using an earlier 4.2.1 library that did not support
SNI.

This SHOULD support SNI. https://hc.apache.org/news.html

As far as I know, this should automatically enable SNI support. (e.g.,

)

If there are other client-side configuration steps required to support SNI
on Android, please provide a link.

··· On Thu, Sep 8, 2016 at 10:31 AM, Brent Atkinson wrote:

Hi Nik,

I think what Yaw might be indicating here:

  • Those developing custom back-ends have likely already hit this with
    Collect
  • They likely can easily confirm whether any change to support SNI is
    working or not
  • They already have a solution and perhaps could contribute it

I took a quick peek at what this and it appears that SNI is supported with
HttpsUrlConnection since Android 2.3. However, Collect uses what appears to
be a custom fork of Apache HttpClient.

Brent

On Thu, Sep 8, 2016 at 11:29 AM, blitheringeejit via ODK Developers < opendatakit-developers@googlegroups.com> wrote:

Hi Yaw - thanks for filing this.

I'm not sure what you mean by "those folks who built the custom
back-ends" - unless you mean folks like me, because that's what I've done!
But there isn't any solution for this issue in back-end code. Without an
SNI-compatible connection, the Collect requests never make it to the
back-end service - they are kiled by Collect at the TLS connection stage,
before the request is sent.

I've made a few minor changes to Collect in my own fork, but I'm really
not knowledgeable enough about Android to be able to unpick and modify the
http connection code - especially as there are several different
connections at different points in the app. I'm happy to contribute to the
project in areas I know about (I've done a lot with PHP/MySQL backends for
Collect) but I'm really not a Java programmer, and the Collect code is
mostly beyond my understanding.

I only submitted the request because I thought quite a lot of Collect
users might want secure connections to custom back-ends, and not supporting
SNI at the connection level makes this harder to do with budget hosting and
DIY projects.

Thanks
Nik

On Wednesday, September 7, 2016 at 4:37:37 PM UTC+1, Yaw Anokwa wrote:

Hi Nik,

I've filed it at https://github.com/opendatakit/collect/issues/199 as
a feature request. What would be ideal is that those folks who built
the custom back-ends take the lead on this. Have you run this by them?

Yaw

Need ODK consultants? Nafundi provides form design, server setup,
in-field training, and software development for ODK. Go to
https://nafundi.com to get started.

On Tue, Sep 6, 2016 at 5:27 PM, blitheringeejit via ODK Developers opendatakit...@googlegroups.com wrote:

Hi Group:
I've been working for a while now with Collect, using it in
conjunction with
custom back-end services running on non-Aggregate web hosts, with form
downloading and uploading working over secure SSL connections.
It all works nicely, but increasingly I'm finding that budget web
hosting
provides SSL connections using shared IP addresses, which may be
accommodating multiple SSL certs. Browser and other app clients don't
mind
this because they use SNI
(https://en.wikipedia.org/wiki/Server_Name_Indication), but Collect
doesn't
use SNI - it uses an older http connection class which doesn't support
SNI.
Without SNI, the server doesn't know which cert to present in response
to
SSL connection requests, and so Collect receives a cert which doesn't
match
the requested hostname - so the connection fails.
Would the maintainers consider a RFC to update the http/s mechanism
used in
Collect to one which supports SNI? I understand this could be done
using
the HttpsURLConnection class - I'm not knowledgeable enough to
implement it
myself, but I think it would be a beneficial update.
Grateful for any thoughts on this.
Thanks
Nik

--
You received this message because you are subscribed to the Google
Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"ODK Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit-developers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com