Email notifications tribulations

Some of you may have noticed fewer email notifications than usual lately. This seems to be due mostly to 2 separate issues which happened coincidentally.. one of which took much longer to be corrected than the other.

An honest mistake

Even though we had made no changes to the live version of our email engine, our delivery rate (% of emails making it to the intended recipient's inbox) went down all of a sudden last month.  We quickly found out that we had been mistakenly put on a spam list operated by Spamhaus. This is a reputable company and their lists are being used by many email providers worldwide.

Remedial action: we informed them of the mistake and were able to get removed from the list within hours of noticing the issue.

Unfortunately, even though it was very effective in bringing our delivery rates back to normal levels, this was short lived. A number of our emails were now being dropped again.

A change of policy

Many of our emails were not being put in a spam folder for the most part, but simply silently dropped when received, and some business customers were affected more than others. After investigation we determined that the problem was affecting Yahoo! email addresses mostly and set out to find out why.

It turns out Yahoo! changed their global email policy last month and started rejecting all emails sent by third party servers (like ours) on behalf of Yahoo! users. So any email sent from a Yahoo! email address to anyone (even outside Yahoo!)), but not sent by Yahoo! itself, would be rejected. Since we have a significant portion of our business users and their clients using Yahoo! email addresses, it quickly became problematic as we generally send our notifications using staff and client email addresses to make it easier for people to reply to each other. But this is no longer allowed for Yahoo! email addresses.. and now for AOL's too. We expect other big name email providers to follow suite and eventually we will not be able to send emails on behalf of anyone anymore.

Remedial action: we then started transitioning our email infrastructure to no longer send emails on behalf of other people at all, and instead use our own no_reply email address. From now on, all messages will come from "Online scheduler" using our no_reply email address, but we are also specifying a "Reply-To" header, instructing email softwares to reply to you (when your clients receive a notification) or to the client (when staff receives a notification). So even though you will now always see all emails come from us, you should still be able to reply and the message to go to whoever should get it... just as before, but in a different way.

For more information about the change and how it affected lots of online services, you can type "yahoo email policy change" in Google (good articles here, here and here).. and straight from the horse's mouth here and here.

Going forward

In order to better stay on top of things in the future, we are now hiring outside help (email delivery experts) to handle most of our email traffic going forward. Email delivery has become a complicated science requiring expert knowledge and infrastructure. We could have avoided the first issue (being mistakenly put on a spam list) by negotiating to be put on whitelists instead and we could have mitigated the severity of the Yahoo! policy change by having a better monitoring system alerting us to the specific problem sooner.

In an ideal world, all spam messages would be blocked and none of the legitimate messages would. And ideally, we wouldn't need to worry about how to make our messages not look like spam to the computers sorting them. Instead, emails are getting increasingly expensive and difficult to manage.

Reliable email communications are vital to you, vital to us, and we are going to invest more to ensure that we are among the least affected when things outside our control change or break all of a sudden.