Apr 16

SPF-Identified Servers - How and Why

To turn DMARC into something useful to people, dmarcian processes DMARC data using a big pile of rules.  These rules identify sources of email, and dmarcian presents users with DMARC compliance information based on email source.

One identified source of email is called "SPF-Identified Servers", and dmarcian users are often curious as to how data ends up in this source.  How and why this source works is explained below. Read the rest of this entry »

Jan 16

How can you eventually move to a p=reject policy when third parties are unable to send email properly on your behalf?

In many cases, a DMARC compliant SMTP relay server can be used to do the trick. In this article, we’ll explore some of the facets of sending DMARC compliant email from third parties, what to look for, and how common hosted solutions such as Google Apps, Office 365, Amazon Simple Email Service (SES), can be leveraged as SMTP relays. Read the rest of this entry »

Oct 15

Broken SPF.. what does it mean?

People sometimes write in and ask "what is the impact of a broken SPF record"?

The net effect of a broken SPF record is that receivers can't reliably use SPF to determine the legitimacy of the domain's email.  *Some* receivers might ignore the broken parts of an SPF record and keep checking, but out of the box all SPF implementations will barf, and you'll be left with a record that is introducing uncertainty into email performance.

We've discovered an odd story related to SPF deployment.  When delivering a lot of email, you need an SPF record just to get to the welcome mat.  What happens is that new senders go through their pre-flight check list, start sending with working SPF records, and things operate in a nice steady state for a while.

Over time, the servers that are authorized using SPF earn enough good reputation so that senders get through the front door and into the content-based/user-engagement world.  At any time, SPF records may get extended until they break or become inaccurate, but since the original set of authorized servers already has a good enough reputation, the inaccurate/broken SPF records do not cause too much trouble.  This is how SPF has traditionally been used prior to DMARC.  In this way, SPF might have been a "reputation bootstrapping tool".

Unfortunately, when SPF breaks, the newly authorized servers are left out in the cold.  People then hire deliverability experts, build volume ramps, and chase symptoms as most of their email is doing OK.  Worse is the resulting notion that email infrastructure is a heavy weight and cannot be replaced or upgraded without serious investment.

DMARC changes the above story in a very significant way.  First is visibility — you can actually see what is going on and where SPF isn't performing like it should.  Second is that SPF results now matter in terms of email content and identifier alignment.  In sum, accurate SPF records are no longer a "reputation bootstrapping tool", but necessary for reliable email delivery.

Oct 15

PTR mechanisms in SPF records

If PTR mechanisms are detected, the current diagnostic output is:

Warning: PTR mechanisms SHOULD NOT be used and cannot be resolved using this diagnostic tool.  More info at <this page!>.

What does the PTR mechanism mean?  When an email receiver gets a piece of email and the PTR mechanism is in the sender's SPF record, the receiver will look at the incoming IP address and do a "PTR" lookup.  For example, if the sender is sending email from IP address, the receiver will perform a PTR lookup of to attempt to retrieve a hostname.  Lastly, if a hostname is discovered for IP address, then that hostname's domain is compared to the domain that was originally used to lookup the SPF record.

3 important things about the above:

  1. The PTR mechanism has been deprecated.  See the relevant RFC for more info.
  2. The SPF Surveyor cannot resolve PTR mechanisms because a real connection from a real sender is necessary to complete the lookup.
  3. MOST IMPORTANTLY: Some large receivers will skip the mechanism – or worse they'll skip the entire SPF record – because such mechanisms cannot be easily cached.  Imagine a large receiver doing a PTR lookup for millions of different connections... the size of the local cache explodes.


Oct 15

Meaning of "WARNING: No A or AAAA records found"

If you publish an SPF record and use the "a" mechanism, but your domain doesn't actually have a "A" record in place, then you'll see this warning.

Here's a sample SPF record that contains the "a" mechanism (the "a" is in bold):

v=spf1 a include:_spf.google.com ~all

The "A" DNS record is how you use the DNS to associate an IP address with your domain.  The "AAAA" DNS record (also called "quad-A") is used to associate an IPv6 address with your domain.

If you don't have an IP address associated with your domain, then you shouldn't use the "a" mechanism in your SPF records.  This might be case if you're only using your domain for email.

Oct 15

What is the difference between SPF ~all and -all?

SPF is all about publishing a list of servers that are authorized to send on behalf of a domain.

After writing out a list of servers in the form of an SPF record, the right thing to do is to end an SPF record with something that says "and everything else on the Internet is NOT authorized".

The way the above is written is to use the "all" mechanism.  This mechanism matches everything.  By adding a prefix of "~" or "-", the meaning of the mechanism is changed to be:

  • "softfail" in the case of "~"
  • "fail" in the case of "-"

Both mean "NOT PASS", but there is a subtle difference, and it has to do with history.

Before DMARC came along, SPF tried to allow its users to express policy -- that is, what should be done if SPF fails.  "softfail" was largely interpreted to mean "NOT PASS".   "fail" was interpreted by a few operators to mean "NOT PASS AND DISCARD THIS FAILING EMAIL".

Discarding email based on SPF results ended up causing too much legitimate email to be dropped, and almost all receivers ended up using SPF as input for anti-spam engines.  However, a few operators still interpreted "fail" as meaning "NOT PASS AND DISCARD".

Fast forward to the world of DMARC.  Now receivers are using DMARC to find any positive assertion that a domain is associated with a piece of email.  In this context, ~all is the same thing as -all... NOT PASS.

However, if you operate with "-all" in your SPF record, you might run into an operator (once in a blue moon) that discards otherwise legitimate email.  Debugging this issue can be difficult.  This issue can be sidestepped by using "~all" instead of "-all".