Central upgrade failing due to pre-existing nginx install

I have followed the version upgrade guide so we can upgrade from v 1.2 to the latest version. I followed the guide carefully reading all of the notes and steps and did not hit one hitch along the way. Unfortunately, I am not able to browse to our site.

Via Chrome I am getting a certificate error. Via Firefox I can bypass the error but all it says is Welcome to central.camfed.org! and I cannot proceed past it.

We are using a AWS EC2 instance and custom postgres database in AWS - details of which I added to the nano.env file as per the instructions. Hopefully there is no issue accessing the database.

I notice when I run the command 'docker compose ps' I am not seeing the central-nginx-1 row which according to the Digital Ocean instructions, I should still be seeing. Could there be an issue with nginx somewhere?

image.png

Thank you

Hello again.

Things have progressed a bit. The certificate error I had is now gone and I can browse to a site but I am still seeing the welcome message and not the original sign in page we are used to seeing. I am wondering if this is still an issue with nginx. When I run the command 'systemctl status nginx' I can see nginx is active. But when I run docker compose ps I do not see it listed

I am a novice when it comes to this sort of thing. Assistance greatly appreciated.

Thanks
Tony

For more information whilst waiting. I have spun up a new linux instance and followed the Digital Ocean guidance. When I run docker compose ps, I can see the nginx service listed which is missing on our original instance post the version upgrade.

You have a copy of nginx installed on the host machine and that nginx install is preventing the nginx container that comes with Central from running.

What you do to resolve this issue depends on why you have nginx installed, so maybe we can start there. Why do you have nginx installed?

1 Like

Thank you for the response! I was able to remove the copy of nginx and the nginx container that comes with Central is now running.

I am still seeing the message below in my browser when I visit our site something still not quite right.
image

Presumably it is looking at an incorrect site, perhaps a test one from way back when Central was first configured. I've been hunting around looking for .html files but I can't find anything. Perhaps because nginx was installed and was creating a conflict and blocking the nginx container....

If I was too attempt the upgrade steps again, would this create any issues?

Retrying the upgrade shouldn't hurt anything, but I don't think that's the issue, so I would hold off on that for now.

From the screenshot you shared, it looks like everything is working on the host machine, so I'd reboot the machine and then confirm the networking.

Make sure the DNS record is mapped to the correct IP. You can confirm your public IP on AWS, by running curl http://169.254.169.254/latest/meta-data/public-ipv4.

Check your AWS security group for that EC2 instance allows traffic to port 80 and 443.

Machine rebooted. Can confirm the DNS is correctly mapped to the public IP.

After confirming that the domain name was properly mapped to the host IP and confirming incoming traffic to port 80/443 was allowed (both external and internal firewalls), I had a chance to further investigate this issue and found a number of problems. I wanted to report back on the root problems in case someone else runs into this problem. The root problems were:

  • nginx and certbot were both installed on the host machine (outside Docker). With nginx listening on port 80, Central's containerized install of nginx was unable to fetch an HTTPS cert from LetsEncrypt.
  • There were two copies of Docker (via snap and via apt) installed. When two copies of Docker are installed, strange things can happen. In this case, it resulted in a zombie process holding on to port 80, even after I had removed the host machine's nginx.

The fix was to uninstall the host machine's nginx/certbot, uninstall Docker via snap, reboot, then rebuild Central's containers.