ODK Central not Sending Emails to Users

[dan_p]

1. What is the problem? Be very detailed.
I've just managed to successfully set up and ODK Central Server on Amazon Web Services! Unfortunately, when I create new Web Users in the backend, they don't get emails notifying them that they have an account. Similarly if I click to reset their password, they do not recieve these emails either. We've tried using our personal and professional email addresses and have checked the Junk/Spam folders which are empty. It would seem that the emails are not getting sent. How can I determine where things are going wrong? (e.g. log files, config files etc)

2. What app or server are you using and on what device and operating system? Include version numbers.
I have installed ODK Central v1.0.0 onto an Ubuntu 16.04 operating system on Amazon EC2 t2.micro instance. I installed ODK Central following the instructions for EC2 listed here:
https://docs.getodk.org/central-install/

Note that I did not follow the instructions for "Using a Custom Mail Server" that are here provided here:
https://docs.getodk.org/central-install-digital-ocean/
because I understood the instructions to mean that there is already a basic webserver setup in ODK Central (please correct me if I am wrong).

3. What you have you tried to fix the problem?
I have tried searching the forums for this issue, but haven't found anyone posting the same problem. I have tried creating a number of user accounts using different email services and none of them work.

4. What steps can we take to reproduce the problem?
Following the EC2 instructions listed here:
https://docs.getodk.org/central-install/

5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.
Nothing else.

[danbjoseph]

If you run docker-compose ps does it show that "mail" is in an up state?

[dan_p]

Thanks for your reply Dan, yes my console is showing exactly the same states as yours (screenshot attached)

[danbjoseph]

When you say "create new Web Users in the backend", do you mean by running this command:

docker-compose exec service odk-cmd --email YOUREMAIL@ADDRESSHERE.com user-create

or on the Central website as described here: https://docs.getodk.org/central-users/#creating-a-web-user

---

Do the Central server audit logs show "user create" actions?

---

Are you resetting the password via the web interface?

---

For a previous issue with Let's Encrypt, @yanokwa had me run:

cd ~;
docker cp nginx:/var/log/letsencrypt/ /tmp/letsencrypt_logs;
tar -zcvf letsencrypt_logs.tar.gz /tmp/letsencrypt_logs;

I'm not sure if there are mail server specific logs that can be pulled?

[Saad]

Hi,

I am facing the same issue with email (emails not reaching to inbox). Is there a solution for this?

Regards,
Saad

[yanokwa]

Try a password reset and then run docker-compose logs --tail 50 service and docker-compose logs --tail 50 mail to see if there are any obvious errors.

Assuming no errors, then you have a deliverability issue. Email deliverability will vary depending on:

1. Whether or not your server IP is blacklisted.
2. Whether or not you have DKIM/SPIF enabled.
3. How aggressive your recipient's email provider is.

You can test deliverability at https://www.mail-tester.com.

I've found the best way to ensure a message is delivered is to use a mail provider like sendgrid.com, mailgun.com or amazon.com/ses. https://docs.getodk.org/central-install-digital-ocean/#using-a-custom-mail-server has instructions on how to use a custom provider.

[dan_p]

Hi Saad, are you using Amazon EC2 for your ODK Central server? That's my current setup and where I hit this problem originally. After a lot of reading I found out that most email servers block emails that are being sent from EC2 servers because it's commonly used for spam and scams. I haven't had time to find a good work around for this on EC2 yet, but I think Amazon offer some sort of trusted email service that can be set up from within the AWS Console.

Thanks to everyone for the suggestions too. I'm really loving this community.

[Saad]

Hi Dan,

Thanks for the reply. Yes, I am using AWS EC2, and your diagnosis might be very true. I will check for relevant things on AWS. The strange thing is that I don't see this activity and its associated actions (password reset + sending email) in the system audit logs on central. Assuming that everything is fine on Central side (since email server is an external entity), the system should generate a line item about this activity. Can anyone confirm if they have ever seen any log item in audit logs for this?

Regards,
Saad Omer

[dokita700]

Hi all,
I am having the same issue like Dan and Saad have narrated above.
a. Despite using ses credentials and verifying identities on aws, Central is not able to send mails out to users.
b. It seems Central is not able to provide logs when custom mail server is used
c. When a custom mail server (aws ses) is used all I get is "Something went wrong: the server returned an invalid error."
d. I seems aws ses wants all receivers email addresses to be verified as well.
e. Below is a section of 'docker-compose logs --tail 50 service' command.

[dokita700]

Possible solution: "docker-compose ps" shows that mail is perpetually on port 25 despite whatever port that is configured in files/service/config.json.template. while port 25 is highly restricted on AWS EC2 by default.

Please, is there a way to change the port mail is running?

[Florian_May]

Have you got a config.json? Copy from config.json.template and change mail server settings. Does that fix it?

[dokita700]

Thanks for your response. I had tried that severally, it didn't work for EC2 on AWS. Perhaps I am still missing something.

[Florian_May]

Is there a config/local.json file?

[LN]

@Saad

the system should generate a line item about this activity

Currently there is no logging from Central when it is successfully able to hand off an email for delivery.

@dokita700 Are you sure that your SES account is out of sandbox mode? See more suggestions to address the error you are getting at https://stackoverflow.com/a/37528929/137744.

[Razvan_Zaharia]

For Central running on digital ocean with the domain (actually sub domain) on namecheap the solution for this problem was adding the subdomain in the SPF record on namecheap (include: spf.subdomain). If anybody still has this problem. Actually is not even a problem but maybe it should be included in the installation guide.

[yanokwa]

Note that email delivery is a tricky problem and it's unlikely DKIM/SPF from your IP will be enough to guarantee delivery to a wide range of domains. That's why we suggest dedicated mail providers in the users aren't receiving emails section of our troubleshooting guide. That link is also highlighted at the bottom of the Logging into Central section of the installation guide.

We do document setting up DKIM and SPF. It sounds like you didn't find that document. Any suggestions on where we could put it to make it more visible?