Could not connect with Form Server Preview

sowe1 · November 16, 2021, 11:59am

Hi Support and Community:
I am trying to deploy an odk-central for testing and I am seeing various problems.
The main one and the one that I can't understand is this.
error

I'm read all documentation

This is my .env:

SSL_TYPE=selfsign
DOMAIN=odk.test.org
SYSADMIN_EMAIL=xxxxx@gmail.com

odk.test.org is my IP with mapping in my /etc/hosts:

127.0.0.1       localhost
127.0.1.1       prcs-ODK
72.17.0.1       postgresd
10.20.60.102    odk.test.org

I'm lost , this is the log https://gist.github.com/ravamo/4ba3ad2e5a4ce3bde821490fa3046348

yanokwa · November 16, 2021, 1:52pm

I strongly recommend deploying Central, even a test server, on server on the Internet with a proper cert.

For this particular issue, try adding this extra_hosts line to your enketo container config in docker-compose.yml.

  enketo:
    volumes:
      - secrets:/etc/secrets
    build:
      context: .
      dockerfile: enketo.dockerfile
    restart: always
    depends_on:
      - secrets
      - enketo_redis_main
      - enketo_redis_cache
    environment:
      - DOMAIN=${DOMAIN}
      - SUPPORT_EMAIL=${SYSADMIN_EMAIL}
    extra_hosts:
      - 'odk.test.org:10.20.60.102'

sowe1 · November 16, 2021, 2:39pm

Hi @yanokwa
thanks for your reply but doesn't work the same problem.
I'm adding this new tag with the server-internal IP but I've the same problem.

I'm agree with you, as far as I know, I need a domain to generate the certificate, since when I have tried to generate it, it asks for a domain.

yanokwa · November 16, 2021, 3:24pm

And to confirm, did you do a docker-compose build; docker-compose stop; docker-compose up --detach after you made the change?

sowe1 · November 16, 2021, 4:23pm

Hi !
Yes , attached the docker-compose.yaml , .env and enketo.dockerfile.
documantation
Enketo : * http code: 500*
Best

delcroip · November 16, 2021, 6:30pm

Hi,

I manage to get it work locally on windows, the biggest issue was the lack of support for port other than 443 (mine is already taken by another container)

I update a bit the code and it works

git clone according to doc + recursive to get linked project

git clone https://github.com/getodk/central.git --recursive -c core.autocrlf=false odk-central

.env

HTTP_PORT=2080
HTTPS_PORT=2443
DOMAIN=host.docker.internal
SSL_TYPE=selfsign
SYSADMIN_EMAIL=pat@domain.org

compose where I added the port for service and enketo

version: "3"
services:
  postgres:
    image: "postgres:9.6"
    volumes:
      - pg_data:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: odk
      POSTGRES_PASSWORD: odk
      POSTGRES_DATABASE: odk
    restart: always
  mail:
    image: "itsissa/namshi-smtp:4.92-8.deb10u6"
    volumes:
      - "./files/dkim/config:/etc/exim4/_docker_additional_macros:ro"
      - "./files/dkim/rsa.private:/etc/exim4/domain.key:ro"
    environment:
      - MAILNAME=${DOMAIN}
    restart: always
  service:
    build:
      context: .
      dockerfile: service.dockerfile
    depends_on:
      - secrets
      - postgres
      - mail
      - pyxform
      - enketo
    volumes:
      - secrets:/etc/secrets
      - "./data/transfer:/data/transfer"
    environment:
      - DOMAIN=${DOMAIN}
      - HTTPS_PORT=${HTTPS_PORT:-443}
      - SYSADMIN_EMAIL=${SYSADMIN_EMAIL}
    command: [ "./wait-for-it.sh", "postgres:5432", "--", "./start-odk.sh" ]
    restart: always
    logging:
      driver: local
  nginx:
    build:
      context: .
      dockerfile: nginx.dockerfile
    depends_on:
      - service
      - enketo
    environment:
      - SSL_TYPE=${SSL_TYPE:-letsencrypt}
      - DOMAIN=${DOMAIN}
      - CERTBOT_EMAIL=${SYSADMIN_EMAIL}
    ports:
      - "${HTTP_PORT:-80}:80"
      - "${HTTPS_PORT:-443}:443"
    healthcheck:
      test: [ "CMD-SHELL", "nc -z localhost 80 || exit 1" ]
    restart: always
    logging:
      driver: local
      options:
        max-file: "30"
  pyxform:
    image: 'ghcr.io/getodk/pyxform-http:v1.7.0'
    restart: always
  secrets:
    volumes:
      - secrets:/etc/secrets
    build:
      context: .
      dockerfile: secrets.dockerfile
    command: './generate-secrets.sh'
  enketo:
    volumes:
      - secrets:/etc/secrets
    build:
      context: .
      dockerfile: enketo.dockerfile
    restart: always
    depends_on:
      - secrets
      - enketo_redis_main
      - enketo_redis_cache
    environment:
      - DEBUG=*
      - DOMAIN=${DOMAIN}
      - SUPPORT_EMAIL=${SYSADMIN_EMAIL}
      - NODE_TLS_REJECT_UNAUTHORIZED=0
      - HTTPS_PORT=${HTTPS_PORT:-443}
  enketo_redis_main:
    image: redis:5
    volumes:
      - "./files/enketo/redis-enketo-main.conf:/usr/local/etc/redis/redis.conf:ro"
      - enketo_redis_main:/data
    command:
      - redis-server
      - /usr/local/etc/redis/redis.conf
    restart: always
  enketo_redis_cache:
    image: redis:5
    volumes:
      - "./files/enketo/redis-enketo-cache.conf:/usr/local/etc/redis/redis.conf:ro"
      - enketo_redis_cache:/data
    command:
      - redis-server
      - /usr/local/etc/redis/redis.conf
    restart: always
volumes:
  pg_data:
  transfer:
  enketo_redis_main:
  enketo_redis_cache:
  secrets:

sowe1 · November 17, 2021, 8:19am

Hi @delcroip ,
Thanks for your comment.
I'm run the nectcat and any other process use this port . And i'm remove all docker container, images, network, etc.. with a docker system prune

$ sh netstat -nepal|grep 443
tcp        0      0 10.20.60.102:55940      45.77.97.47:443         ESTABLISHED 1000       75712888   2060892/firefox     
tcp        0      0 10.20.60.102:46942      34.117.237.239:443      ESTABLISHED 1000       75714375   2060892/firefox     
tcp        0      0 10.20.60.102:46624      44.239.205.250:443      ESTABLISHED 1000       73443521   2060892/firefox

I don't know if the problem is my version 1.3.3 or a miss something because is really complex to run odk-central for a developer environment.
My steps was :

git clone for the project
add the new tag for the server IP
create an .env file (attache in this post)
docker-compose build and docker-compose up

I miss something?

My .env

SSL_TYPE=selfsign
DOMAIN=host.docker.internal
SYSADMIN_EMAIL=email@cruzroja.es

The only exception is this , nginx show me unhealthy but I don't see any error :

4c792fc02639   central2_nginx  "/bin/bash /scripts/…"   12 minutes ago   Up 11 minutes (unhealthy)   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   central2_nginx_1

Matthew_White · November 17, 2021, 4:44pm

I just wanted to note that we normally don't use Docker during local development. Locally, it's easiest to work on the backend or the frontend by cloning the individual repository and following the setup instructions in the readme. For example, you can find the backend at https://github.com/getodk/central-backend, and the readme there has instructions for setting up a development environment. You can also work with Enketo and pyxform-http locally: see the frontend readme for more information.

If you really must use Docker as part of development, I'll echo @yanokwa in recommending that you set up your environment as closely as possible to the typical production setup.

sowe1 · November 17, 2021, 8:07pm

Hi @Matthew_White
Thanks for your tip and feedback. My principal problem is I have the same issue in my development, staging and production environment and I don't know what the problem is ( certificated, network, etc..).

I'm trying to find in the official forum this error and I'm try apply any solution like modifying my docker-compose, changing the dockerfile, add my local ip in /etc/host etc.. and nothing works.

But for me now it is that odk-central works in all respects in some environments (I've two Spanish red cross projects waiting for resolve this problem )for . I will continue fighting and have if I understand the problem.

Thanks

delcroip · November 17, 2021, 8:14pm

In my case this error happen when enketo was not finding the form server (when I clicked on preview)

if you look on the enkoto and service config files (modified on the PR) you can see where the form server address is defined.

if you have multiple server, simply check if the form server address can be resolved form Enketo (you should be able to use IP)

NGINX won't give that error, in the worst case you can check the logs

sowe1 · November 18, 2021, 11:10am

Hi @delcroip

Thanks for your tip . No only have one server for every environment. And testing your branch but for now, but I have the same problem.

I'm using your more or less configuration but my only difference is the ports and the host because host.docker.internal doesn't work in Linux.

The big problem is I don't have any errors in logs and I insert into containers and run a ping to servers and I've connectivity

root@9847f2f2dcf3:/usr/odk# ping enketo
PING enketo (192.168.48.5) 56(84) bytes of data.
64 bytes from odk-central_enketo_1.odk-central_default (192.168.48.5): icmp_seq=1 ttl=64 time=0.095 ms

root@34c586c9b471:/data# ping service
PING service (192.168.48.8) 56(84) bytes of data.
64 bytes from odk-central_service_1.odk-central_default (192.168.48.8): icmp_seq=1 ttl=64 time=0.166 ms
64 bytes from odk-central_service_1.odk-central_default (192.168.48.8): icmp_seq=2 ttl=64 time=0.089 ms
64 bytes from odk-central_service_1.odk-central_default (192.168.48.8): icmp_seq=3 ttl=64 time=0.089 ms

I never thought that with a docker-compose it would be so difficult to do a test. I'm still working on that .
Thanks

PD: I did a telnet server to enketo and enketo to server and works

equipoim@equipoim:~/Documentos/odk-central$ docker exec -it 9847f2f2dcf3  bash
root@9847f2f2dcf3:/usr/odk# telnet enketo 8005
Trying 192.168.48.5...
Connected to enketo.
Escape character is '^]'.

and

root@42dccfe0f742:/srv/src/enketo_express# telnet service 8383
Trying 192.168.48.8...
Connected to service.
Escape character is '^]'.

delcroip · November 18, 2021, 1:47pm

From the docker script, the form server address is defined by https://${DOMAIN} meaning that when you connect to enketo, you should be able to reach the "domain"

docker-compose exec enketo bash
wget https://put-the-domain-from-.env

Normally the extra_hosts config from @yanokwa should have worked if you configured with the "docker host" IP

I am not sure why they bind this with domain instead of either nginx container or even the "form server" container (I guess it is the "service" one), maybe due to some certificate constraint.

br

sowe1 · November 18, 2021, 2:42pm

Hi @delcroip
Thanks, i think I'm fixed my error with two different workarounds.
Thanks all

sowe1 · November 19, 2021, 8:55am

Well in the end it keeps failing odk-central out-of-the-box. I don't know what the problem is, I'm going to leave this thread open in case someone knows how to solve the problem.

I will use an odk-aggreate for my projects.

Thank you @delcroip , @Matthew_White and @yanokwa for your time and help.

yanokwa · November 23, 2021, 6:50pm

@sowe1 I wanted to follow up to see if I can understand the high-level problem. It sounds like you want to run multiple Central installs (e.g., dev, staging, prod) on a single machine?

sowe1 · November 23, 2021, 7:49pm

Hi @yanokwa ,
Thanks , no I'm only one central for server .

yanokwa · November 24, 2021, 12:07am

OK. Let's focus on your production environment then. I'm assuming you have 1 Central install on 1 server. I'm also assuming you have a stock install with no changes besides what is required in the .env file.

What does your .env look like?
Have you mapped your fully qualified domain name to the server's external IP address?
Are all firewalls (e.g., ufw) on the server are disabled?
Are port 80s and port 443 on your server's external IP open and unfiltered?

sowe1 · November 24, 2021, 8:21am

What does your .env look like?

DOMAIN=10.98.20.119
SSL_TYPE=selfsign
SYSADMIN_EMAIL=pat@domain.org

Have you mapped your fully qualified domain name to the server's external IP address?
I did a test with local server IP address and the IP for the service container

      ** local server **
        eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.98.20.119  netmask 255.255.252.0  broadcast 10.98.23.255
        inet6 fe80::c456:a5bf:1cf1:9c44  prefixlen 64  scopeid 0x20<link>
        ether dc:fe:07:1a:5f:cf  txqueuelen 1000  (Ethernet)
        RX packets 124304237  bytes 114234590970 (114.2 GB)
        RX errors 1  dropped 2  overruns 0  frame 0
        TX packets 4594522  bytes 558678514 (558.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 17
       ** IP container **
        docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' 220cbae01edd

  enketo:
    volumes:
      - secrets:/etc/secrets
    build:
      context: .
      dockerfile: enketo.dockerfile
    restart: always
    depends_on:
      - secrets
      - enketo_redis_main
      - enketo_redis_cache
    environment:
      - DOMAIN=${DOMAIN}
      - SUPPORT_EMAIL=${SYSADMIN_EMAIL}
      - NODE_TLS_REJECT_UNAUTHORIZED=0
      - HTTPS_PORT=${HTTPS_PORT:-443}
    extra_hosts:
      - "localhost.example.org:10.98.20.119"

Are all firewalls (e.g., ufw) on the server are disabled?
Yes
Are port 80s and port 443 on your server's external IP open and unfiltered?
Yes, I did a netcat to see that they are not being used.

yanokwa · November 24, 2021, 2:38pm

10.98.20.119 is not a fully qualified domain, it's a private IP. The domain must be something like example.org and it must be mapped to a public IP like 93.184.216.34. Try a fully qualified domain and SSL_TYPE=letsencrypt and get that working first.

sowe1 · November 26, 2021, 8:36am

Thanks @yanokwa ,
Thanks that's right but for now, we don't have a domain because I need to run one odk-central and make tests .
is mandatory to have a domain for running odk-central?
Thanks