Archive

Archive for May, 2010

Sender Policy Framework (SPF) may be preventing your emails from reaching your customers

May 15th, 2010

Emails Not Getting Delivered?

I’ve been spending this week solving a problem that I noticed with one of our real estate specific software packages – a number of customers would report to me that they were not receiving their reports from our system, even though I could email them the same email (copied and pasted) from my normal email client (gmail, via google apps).  From this, I discovered that in order for people to receive email from some of our 3rd party tools, we might need to implement a change to our DNS to make use of a technology called the Sender Policy Framwork.  Essentially, this framework allows a domain name owner to specify which mail servers or networks are allowed to send mail on the behalf of the domain.  Some internet service providers check the SPF records of the sender, and if the mail server they are receiving the mail from is not listed as a legitimate email sender for the from: domain, they will silently fail to deliver the email.  This is especially sneaky because it does not generate a bounce, since the purpose of the framework is to defeat spammers who are sending phishing emails pretending to be websites like paypal or banking sites.

After researching further, I found that a number of large internet email hosts (ISPs, or internet service providers) honor the SPF standards, and are currently at least checking SPF for “negative” configurations where specific senders are “blocked”.  These include the larger mail hosts like google, yahoo, and hotmail, as well as a number of ISPs.  From the openSPF website, current surveys are showing between 5-12% of ISPs implementing the SPF standard, which means that there are likely to be a lot of emails that are not getting through from senders that do not have SPF configured, especially email delivered to the more stringent ISPs.  The broad adoption by “common carriers” like google, hotmail, and yahoo also means it is likely that these ISPs will turn up the heat on spammers in the future by slowly becoming more strict in their enforcement of SPF standards, since it reduces their costs of routing spam email.

After this research, I decided it was time to act to make sure that our customers received email from all of our services.

How can you tell if your email is not complying with SPF?

The easiest way is to send mail to a 3rd party that has SPF rule checking turned on.   We use Google for our mail services, and they do check SPF records, so all I need to do was use the “view original” menu option (it’s in the “reply” menu on every message) to see the results of the SPF testing.  I did this with a email that a customer reported they did not receive (we are copied on these emails).

What you are looking for in the internet headers is this:

Received-SPF: pass

This means that the 3rd party mail sender is property authorized to send mail on behalf of your domain.  This is the “happy” result.

What I found was:

Received-SPF: softfail

This means that there is no SPF record configured that says the email sender is allowed to send on behalf of the domain.  Basically, anything other than “pass” means that these emails may not reach some destinations, depending on how strictly the receiving mail host has configured their system.

Some internet service providers will reject messages that evaluate to a “softfail”; google is a little more tolerant and allowed it through to me, but apparently not to our customer of this particular email.

In order to fix the problem, I had to identify what SMTP server was sending email on behalf of goodlifeteam.com; in the message header below the softfail, I found this:

google.com: domain of transitioning experience@goodlifeteam.com does not designate 206.131.185.72 as permitted sender)

206.131.185.172 happens to be one of the mail servers used by Top Producer’s Market Snapshot, so now I can “white list” this server in our SPF record so that these messages do not receive a soft fail – thus improving the likelihood of delivery.

How do I fix my SPF records?

First, I audited whether there was a problem by looking at the mail headers of my email messages from any of our 3rd party services that send email.

Next, I had to write a handful of changes into our SPF record for our domain to include those 3rd party services as valid senders for our domain.  This included Top Producer and Top Marketer (for Market Snapshot), one of our two 3rd party IDX systems, our current website host, and of course, our mail host, which is google via google apps.  I had already configured our domain for google apps and our website, so I needed to generate the additional SPF rules so that allowed all our mails from Market Snapshot and our 3rd party IDX to be properly delivered.

After doing this research, I wrote up an SPF rule using the SPF wizard at http://old.openspf.org/wizard.html

I found the wizard exceedingly easy to use, plus it gives you a nice verbose explanation of what each rule means.

This is what was produced:.

The SPF record:

v=spf1 ip4:62.13.148.0/24 ip4:206.131.185.0/24 a mx a:realestate.goodlifeteam.com include:aspmx.googlemail.com ?all

can be explained as:

v=spf1 This identifies the TXT record as an SPF string.
ip4:62.13.148.0/24 Every host in the range 62.13.148.0-62.13.148.255 is allowed to send mail from goodlifeteam.com. (THIS IS OUR 3RD PARTY IDX, REALZI)
ip4:206.131.185.0/24 Every host in the range 206.131.185.0-206.131.185.255 is allowed to send mail from goodlifeteam.com.n (THIS IS THE IP RANGE FOR MARKET SNAPSHOT AND TOP PRODUCER)
a goodlifeteam.com’s IP address is 205.186.187.189 (ekiaiomcem.c06.mtsvc.net).
That server is allowed to send mail from goodlifeteam.com. (THIS IS OUR WORDPRESS WEBSITE HOST)
mx This wizard found 13 names for the MX servers for goodlifeteam.com: gy-in-f27.1e100.net, ALT1.ASPMX.L.GOOGLE.com, mu-in-f27.1e100.net, ASPMX4.GOOGLEMAIL.com, ALT2.ASPMX.L.GOOGLE.com, ASPMX3.GOOGLEMAIL.com, ASPMX2.GOOGLEMAIL.com, mail-yx0-f60.google.com, ww-in-f27.1e100.net, ASPMX5.GOOGLEMAIL.com, gv-in-f27.1e100.net, and ASPMX.L.GOOGLE.com.
(A single machine may go by more than one hostname. All of them are shown.)
The servers behind those names are allowed to send mail from goodlifeteam.com.

(THESE ARE OUR GOOGLE MX RECORDS)

a:realestate.goodlifeteam.com realestate.goodlifeteam.com is also allowed to send mail from goodlifeteam.com. (THIS IS THE WEBSITE THAT COMES WITH OUR 3rd PARTY IDX – MOSTLY AS A PRECAUTION IN CASE THIS SERVER STARTS SENDING MAIL INSTEAD OF THE SMTP THEY CURRENTLY USE)
include:aspmx.googlemail.com Any server allowed to send mail from aspmx.googlemail.com is also allowed to send mail from goodlifeteam.com.
?all SPF queries that do not match any other mechanism will return “neutral”.
Messages that are not sent from an approved server should still be accepted as if the SPF record did not exist.

After that, I added a TXT record to our domain name, and included the SPF rule as generated by the wizard, and waited for DNS to propogate the record.  A few hours later, I checked a recent Market Snapshot inquiry and looked in the header and Voila!

Received-SPF: pass (google.com: domain of experience@goodlifeteam.com designates 209.85.221.224 as permitted sender)

Messages coming from all of our 3rd party services are now listed “pass” by google and are more likely to be delivered to our clients.  Hurry for emails that make it to mail boxes!

The Goodlife Team, Austin Real Estate Agents , are now proud to represent The Riverside Grove Project located South of Riverside on Grove Boulevard.  Visit the fan page for the project at the Riverside Grove project on Facebook.

email, technology ,