Get Blank Forms from ODK Collect not working on local ODK central server

[im.techpm]

Hi ODK team !
I am trying to setup a local ODK Central server on a machine running with NixOS.
I have managed to get the server running, access the web interface, create a user, upload a Form from an existing XlsForm, but I am facing issues with Getting Blank Forms on my ODK Collect App, and also seeing previews with Enketo. Both don't work...

1. What is the problem? Be very detailed.

When uploading a Form, creating an App User, giving him access to the Form, and scanning the App User QR code with my ODK Collect app, I can't manage to download the blank forms on my phone.
I also get an error when trying to preview a form with Enketo.

2. What app or server are you using and on what device and operating system? Include version numbers.

I am using ODK Central v1.0.2, and my server OS is NixOS v20.03.3081.629fe7b1450 (Markhor).
I added git, docker and docker-compose packages.

 # List packages installed in system profile. To search, run:
 # $ nix search wget
 environment.systemPackages = with pkgs; [
   wget
   docker
   docker-compose
  git
   openssl
 ];

I also disabled the firewall and opened the ports for SSH, HTTP and HTTPS.

  # Open ports in the firewall.
  networking.firewall.allowedTCPPorts = [ 22 80 443 ];
  # networking.firewall.allowedUDPPortRanges = [ ];
  # Or disable the firewall altogether.
  networking.firewall.enable = false;

And my user has sudo privileges + docker access, and docker is enabled.

  users.users.nixos = {
    isNormalUser = true;
    extraGroups = [ "wheel" "docker" ];
  };

  virtualisation.docker = {
    enable = true;
    enableOnBoot = true;
  };

I followed all the installation steps, and modified the .env file with the following configuration:

SSL_TYPE=selfsign
DOMAIN=local
SYSADMIN_EMAIL=im.techpm@solidarites-liban.org

This was my only change to the repository.

I suspect the issue could come from the fact that my server doesn't have an ssl certificate, but for a local server, is that really an issue ?
My webpage looks like this:

3. What you have you tried to fix the problem?

When deploying to an AWS server (following the given instructions on the ODK doc), and giving my ODK Central a DOMAIN name, this was all working fine.
Now with a local server using selfsign, I am facing issues...

4. What steps can we take to reproduce the problem?

Install NixOS on a server, with the same configuration, and following the ODK Central installation steps.

5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.

I can provide any additional information if you need it !

Thanks in advance

Jonathan

[aurdipas]

Probably you can find an asnwer on one of the following posts:

Use easy-rsa to set up a self-signed certificate authority for Central

I have spent a whole day googling not to understand this: Generate appropriate fullchain.pem (-out) and privkey.pem (-keyout) files. Says the ODK Doc. Now I have ca.crt, ca.key, myform.key and myform.crt, and I know how to convert these to pem and concatenate private and public. But I am lost why I need fullchain.pem (contains private+public) and privkey.pem(private of what)?

Selfsign certificate with latest Central

I don't think there are any issues specific to Central with this. I recommend reading through some threads like https://security.stackexchange.com/questions/121163/how-do-i-run-proper-https-on-an-internal-network and https://serverfault.com/questions/964119/enable-https-on-a-private-network to get a feel for the challenges and possible approaches. I think it's also possible to get a specific Android device to accept a self-signed cert following instructions like these.

[dmenne]

It is possible, but a bit messy to install a custom-made certificate. The real challenge you will face is to get the Android App working: For Android 5.0, you can install a custom certificate, but for more recent versions you have to rebuild the Android App with the certificate burnt-in. See the thread cited by @aurdipas

[im.techpm]

Hi,

Thanks for your answers ! I will try to go for a custom ssl certificate.
@dmenne, could you provide more information about how you managed to get everyhting working with the customssl ?
I saw your post on SuperUser, and have been trying to reproduce the same thing, with no success...
I can't figure out what to write in all of the commands, and what to do on my client side (Windows PC for now, to make it trust the website, I'll get into the Android App later...)

• Do I need a separate machine to act as CA ? Or can it be my server where ODK central is deployed ?

• Is any domain name fine (subdomain from FreeDNS for example), or are there some restrictions ? A domain routing to a Private IP address is acceptable ? Is writing the private IP address as DOMAIN= in the .env file possible ?

• Do you need to "Install the certificate" on the server ? How is it done ? When I trying to run openssl verify to check my .crt file I get the following error:

    error 20 at 0 depth lookup: unable to get local issuer certificate
    error CA/pki/issued/odk.crt: verification failed
  

Sorry for all the questions thanks in advance for your help !

Jonathan

[dmenne]

could you provide more information about how you managed to get everyhting working with the customssl ?

Here is my post-hoc installation documentation - I created it from from memory when it worked and did not re-check, so there may be errors.

Certificates

Server

• My router give a name of sgformulare to my local server, so https://sgformulare is what I use to access it.
• See ODK Forum and SuperUser
• Use easyrca to generate to build a root certificate with files public ca.crt and private ca.key
• Make sure that your server can be reached at sgformulare; check settings in WLAN router.
• Use easyrca to generate certificate sgformulare with files sgformulare.crt and sgformulare.key without password. Assuming you are in the easyrsa root folder:

./easyrsa build-serverClient-full sgformulare nopass

• Keep ca.key on USB stick in safe, it is not required for operation, just to create additional certificates.
• Copy sgformulare.key to central/files/local/customssl/privkey.pem. Check destination base directory in the following example:

cp pki/private/sgformulare.key ../central/files/local/customssl/privkey.pem

• Copy pki/issued/sgformulare.crt to central/files/local/customssl/fullchain.pem.

cp pki/issued/sgformulare.crt ../central/files/local/customssl/fullchain.pem

• Rebuild docker-compose. Check if container nginx is running; if there is something wrong, docker ps will show “restarting x seconds”; use docker logs nginx to indicate what could be wrong.

Tablet

• For Android >5, a custom rebuild of ODK Collect is required to accept the certificate. I have posted it on the ODK Forum

Is any domain name fine (subdomain from FreeDNS for example), or are there some restrictions ? A domain routing to a Private IP address is acceptable ? Is writing the private IP address as DOMAIN= in the .env file possible ?

If you use a local installation for internal use only - we have everything closed to the outside by firewall - you could use something like myserver.local. When you google, you will find strong reasons against doing this, there are no names "reserved for internal use" like there are IP addresses that are safe.
I used the name given to my server from the router as domain name (sgformulare). I also tried to use something else by editing the hosts file the client, but that go sour after I noted that more recent Android versions cry fire when you do this without jail break. Since I am not allowed to use a jail-breaked tablet in a hospital, I have to use the available name.

Strangely, rebuilding the app for the Android tablet was easy. You certainly already found my thread on the subject, it is complete and together with the super-clear google documentation it worked withing 2 hours - I won't tell you how long I needed until I got the private certificate stuff right.

[im.techpm]

Hello again !
After struggling a while longer, watching a bunch of tutorial videos, scouring the endless forum threads, and testing many things, I finally got my SSL Certificate to work !
The following tutorials helped me a lot :

• https://www.youtube.com/watch?v=T4Df5_cojAs&ab_channel=kubucation
• https://deliciousbrains.com/ssl-certificate-authority-for-local-https-development/
• https://medium.com/@antelle/how-to-generate-a-self-signed-ssl-certificate-for-an-ip-address-f0dd8dddf754

If anyone is interested, these are the steps I followed (some commands might require sudo privileges):

** SSL Certificate

**** Create a Certificate Authority (CA)
openssl genrsa -des3 -out myCA.key 2048
openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.pem

**** Create a private key pair for the web server
openssl genrsa -out odkcentral.key 2048

**** Generate a Certificate Signing Request (CSR)
openssl req -new -key odkcentral.key -out odkcentral.csr

**** Create a .ext configuration file for the server, with the following text
nano odkcentral.ext

----
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names

[alt_names]
IP.0 = 127.0.0.1
IP.1 = XXX.XXX.XXX.XXX
DNS.0 = localhost
DNS.1 = XXXXX.XXX
----

**** Generate a server Certificate from the CSR + CA key pair
openssl x509 -req -in odkcentral.csr -CA myCA.pem -CAkey myCA.key -CAcreateserial -out odkcentral.crt -days 1825 -sha256 -extfile odkcentral.ext

**** Copy server certificate, server key and CA public key
**** Add to ODK Central project, in the nginx configuration
cp odkcentral.crt <central/files/local/customssl>
cp odkcentral.key <central/files/local/customssl>
cp myCA.pem <central/files/local/customssl>

**** Change nginx host configuration files to point to these files
-  ssl_certificate /etc/${SSL_TYPE}/live/${CNAME}/fullchain.pem;
-  ssl_certificate_key /etc/${SSL_TYPE}/live/${CNAME}/privkey.pem;
-  ssl_trusted_certificate /etc/${SSL_TYPE}/live/${CNAME}/fullchain.pem;
+  ssl_certificate /etc/${SSL_TYPE}/live/${CNAME}/odkcentral.crt;
+  ssl_certificate_key /etc/${SSL_TYPE}/live/${CNAME}/odkcentral.key;
+  ssl_trusted_certificate /etc/${SSL_TYPE}/live/${CNAME}/myCA.pem;

**** Change nginx dockerfile copy statement to copy all files, not only .pem
-COPY files/local/customssl/*.pem /etc/customssl/live/local/
+COPY files/local/customssl /etc/customssl/live/local

I'm not sure if the last 2 steps are really necessary, or if the .crt renamed as fullchain.pem and .key file renamed as privkey.pem would have done the trick. Maybe I will test this later and edit the post.
So when I install this new certificate on my Windows PC client (see instructions here) I could see my website at safe.

This works when typing the IP address in the search bar, and also with the domain name I got from FreeDNS.

HOWEVER I still have some problems with Enketo and the ODK Collect app.
In Enketo, I get the following error message:

Is there a way to make Enketo work with the customssl setup on a local network ? Did someone manage to do this ?
On the ODK Collect App (on a Huawei, Android 8), I can't get my blank forms, no error message is displayed, just a very short loading screen and then nothing.
But I have installed the certificate on my Huawei phone, and I can access the web application through the Chrome browser.
I will try the solution @dmenne suggested, in this post, and check if that works. (but I need to install Android Studio, get the code, etc .. first).
Anyway thanks already for your help, I am moving forward, even though not everything works yet...

Jonathan

[dmenne]

Is there a way to make Enketo work with the customssl setup on a local network ? Did someone manage to do this ?

I have not yet tried this one, because Enketo not that important for me

Central/Enketo integration with self-signed cert

Another related? question We have installed ODK Central 0.9 and self signed a certificate I can login, and administer ODK central no problem e.g. upload new forms to projects etc. However when I try to preview a newly published form I get a loading error self-signed certificate Sorry I'm new to ODK so still finding my way with the system [enketo-certificate-error] Thanks in advance

[im.techpm]

Hello everyone,

To close this topic, here is everything I did to manage to get the SSL certificate + Enketo + ODK Collect to work on a local area network.

1. ODK Central basic configuration in .env file :

    SSL_TYPE=customssl
    DOMAIN=xxxxx.yyyyy.zzz
    SYSADMIN_EMAIL=x@y.zzz
   

2. Enketo configuration in enketo.dockerfile :
   ENV NODE_TLS_REJECT_UNAUTHORIZED='0'
   This setting is NOT recommended for production, according to the ODK developers.

3. SSL Certificate creation and installation :

    **** Create a Certificate Authority (CA)
    openssl genrsa -des3 -out my_ca.key 2048
    openssl req -x509 -new -nodes -key my_ca.key -sha256 -days 1825 -out my_ca.pem
   
    **** Install certificate on all client devices that need to acess the ODK central web app
   
    **** Create a private key pair for the web server
    openssl genrsa -out odkcentral.key 2048
   
    **** Generate a Certificate Signing Request (CSR)
    openssl req -new -key odkcentral.key -out odkcentral.csr
   
    **** Create a .ext configuration file for the server, with the following text
    nano odkcentral.ext
   
    ----
    authorityKeyIdentifier=keyid,issuer
    basicConstraints=CA:FALSE
    keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
    subjectAltName = @alt_names
   
    [alt_names]
    IP.0 = 127.0.0.1
    IP.1 = aaa.bbb.ccc.ddd
    DNS.0 = localhost
    DNS.1 = xxxxx.yyyyy.zzz
    ----
   
    **** Generate a server Certificate from the CSR + CA key pair
    openssl x509 -req -in odkcentral.csr -CA my_ca.pem -CAkey my_ca.key -CAcreateserial -out odkcentral.crt -days 1825 -sha256 -extfile odkcentral.ext
   
    **** Copy server certificate and server key
    cp odkcentral.crt <odk central path>/files/local/customssl/fullchain.pem
    cp odkcentral.key <odk central path>/files/local/customssl/privkey.pem
   

   If you want to install a second ODK central web server, you can reuse the same Certificate Authority to do so, it will be easier.

4. ODK Collect

    **** Get latest stable version of collect from GitHub
   
    **** Download latest Android Studio
   
    **** Add network security configuration to Android Manifest
    In collect_app/src/main/AndroidManifest.xml
   
        <application
            android:name=".application.Collect"
            [...]
            android:networkSecurityConfig="@xml/network_security_config">
            
    **** Create a network_security_config.xml file in collect_app/src/main/res/xml
    <?xml version="1.0" encoding="utf-8"?>
    <network-security-config>
        <base-config>
            <trust-anchors>
                <certificates src="@raw/my_ca"/>
                <certificates src="system"/>
            </trust-anchors>
        </base-config>
    </network-security-config>
   
    **** Paste the my_ca.pem file to a collect_app/src/main/res/raw folder
   
    **** Build the APK and install on devices
   

I hope this will help those who want to setup a local ODK Central server.
All the basic configuration is provided in the ODK Central installation documentation.
Thanks for your help !!

Jonathan