SPF's 10-lookup limit, DMARC and CAA: checking your domain from the outside

A domain usually breaks quietly. The marketing team adds a newsletter service, its setup page says "add our include to your SPF record", and a week later invoices from the accounting system start landing in spam. Or a certificate renewal fails at night because someone added a CAA record a year ago that names only the old certificate authority. Both problems live in DNS, where only other people's servers read them. This guide shows what mail servers and certificate authorities look up for your domain, how to count SPF's ten lookups by hand and in code, how to move DMARC from monitoring to protection, and what a CAA record changes. The Python sample was run on Python 3.12 and the dig commands on a Linux machine, as printed.

#What other servers look up

Four small DNS answers decide most of it: the MX records say which servers accept mail for the domain, the SPF record (a TXT record that starts with v=spf1) says which servers may send mail as the domain, the DMARC record at _dmarc says what receivers should do with mail that fails, and the CAA record says which certificate authorities may issue certificates. You can read all four with dig:

dig +short MX example.com
dig +short TXT example.com
dig +short TXT _dmarc.example.com
dig +short CAA example.com

For example.com on the day we ran them, the answers were 0 ., "v=spf1 -all", "v=DMARC1;p=reject;sp=reject;adkim=s;aspf=s" and nothing for CAA. That is a model for a domain that sends and receives no mail at all: a null MX (RFC 7505) tells senders not to try, an SPF record of -all says no server may send as the domain, and a DMARC policy of reject tells receivers to refuse anything that claims to. Parked domains and domains used only for a website deserve the same three records, because spammers like a domain that has none.

#SPF: one record, a clear ending, ten lookups

RFC 7208 has three rules that catch most real records.

• Exactly one record. Two TXT records that both start with v=spf1 make the check fail with a permanent error. This happens when a second service's instructions are followed literally. Merge them into one.
• A clear ending. ~all (soft fail) or -all (fail) tell receivers what to think of a server you did not list. ?all says nothing, a record with no all at all means the same, and +all allows every server in the world.
• At most ten DNS lookups. The mechanisms include, a, mx, ptr and exists and the redirect modifier each cost one lookup, and the includes of your includes count too. ip4, ip6 and all are free. Over ten, the result is a permanent error, and DMARC treats that as a failure. The RFC also asks receivers to allow no more than two lookups that return nothing, and says ptr should not be used.

The ten-lookup limit is the one that catches people out, because it is invisible in the record itself. A record with three terms can need twelve lookups. This sample counts them the way the RFC does, following every include and stopping at a real loop:

import re

# The terms that cost a DNS lookup (RFC 7208, section 4.6.4). The limit is 10.
COSTLY = {"include", "a", "mx", "ptr", "exists", "redirect"}


def spf_record(txt_records):
    found = [t for t in txt_records if t.lower() == "v=spf1" or t.lower().startswith("v=spf1 ")]
    if len(found) != 1:
        raise ValueError(f"{len(found)} SPF records (exactly one is allowed)")
    return found[0]


def count_lookups(domain, txt, path=()):
    """Worst case: every term is evaluated, as for a sender that matches nothing."""
    if domain in path:
        raise ValueError("loop: " + " > ".join(path + (domain,)))
    terms = []
    for term in spf_record(txt(domain)).split()[1:]:
        name, target = (re.split(r"[:=/]", term.lstrip("+-~?"), maxsplit=1) + [""])[:2]
        terms.append((name.lower(), target))
    has_all = any(name == "all" for name, _ in terms)
    total = 0
    for name, target in terms:
        if name == "redirect" and has_all:
            continue  # redirect= is ignored when the record has an all
        if name in COSTLY:
            total += 1
        if name in ("include", "redirect"):
            total += count_lookups(target.lower(), txt, path + (domain,))
    return total


DNS = {
    "shop.example": ["v=spf1 mx include:_spf.mailer.example include:_spf.crm.example ~all"],
    "_spf.mailer.example": ["v=spf1 include:_spf.cloud.example ip4:192.0.2.0/24 ~all"],
    "_spf.crm.example": ["v=spf1 include:_spf.cloud.example a ~all"],
    "_spf.cloud.example": ["v=spf1 include:a.cloud.example include:b.cloud.example include:c.cloud.example ~all"],
    "a.cloud.example": ["v=spf1 ip4:198.51.100.0/24 ~all"],
    "b.cloud.example": ["v=spf1 ip4:203.0.113.0/24 ~all"],
    "c.cloud.example": ["v=spf1 ip6:2001:db8::/32 ~all"],
}

print("shop.example needs", count_lookups("shop.example", lambda d: DNS.get(d, [])), "lookups")

DNS["_spf.mailer.example"] = ["v=spf1 include:shop.example ~all"]
try:
    count_lookups("shop.example", lambda d: DNS.get(d, []))
except ValueError as error:
    print(error)

It prints shop.example needs 12 lookups: one for mx, one for each of the two includes, and then the mailer and the CRM both include the same cloud provider, whose record has three includes of its own. That shared provider is counted twice, because a receiver evaluates it twice; it is not a loop. A real loop is a record that leads back to itself, and after the edit at the end the sample says loop: shop.example > _spf.mailer.example > shop.example. For a live domain, pass a txt function that asks DNS.

Ways back under ten, in order of preference: remove includes for services you no longer use; replace a and mx with the ip4 and ip6 addresses of your own servers, which cost nothing; and send bulk mail from a subdomain such as news.shop.example, which has its own SPF record and its own ten lookups. Copying a provider's addresses into your record ("flattening") also works, until the provider changes them without telling you.

#DMARC: from watching to protecting

SPF and DKIM say whether a message came from an allowed server or carries a valid signature. DMARC adds the part people see: the domain in the From line has to match the domain that passed. Its record says what a receiver should do with mail that fails: p=none only reports, p=quarantine sends it to spam and p=reject refuses it. The rua= address receives daily aggregate reports from the big mailbox providers, listing every server that sent mail as your domain.

Google's sender guidelines ask every sender for SPF or DKIM, and anyone who sends more than 5,000 messages a day to Gmail for SPF, DKIM and a DMARC record, where p=none is enough. p=none protects nobody, though. A safe path is to start with p=none and a rua= address, read the reports for a few weeks until every legitimate sender passes, then move to quarantine and later to reject. A subdomain without its own DMARC record falls under the parent domain's record, with sp= as its policy when that tag is set.

#CAA: who may issue your certificates

A CAA record lists the certificate authorities allowed to issue certificates for the domain, for example 0 issue "letsencrypt.org", with issuewild for wildcard certificates. Certificate authorities must check it before they issue. The check climbs the tree (RFC 8659): for www.shop.example the authority looks at that name first and, if it has no CAA records, at shop.example, and the first name with records decides. Without any CAA record, every authority may issue. With one, the authority you actually use has to be on it: a hosting move to a provider that uses a different authority, and a CAA record written for the old one, is a classic way to lose a renewal.

#The web side in the same pass

While you are looking from outside, check the front door as well: http:// should go to https:// in one redirect; the certificate should have more than three weeks left and match the name; TLS 1.0 and 1.1 should be refused; the HTTPS answer from the host itself should carry Strict-Transport-Security; and /.well-known/security.txt should give a Contact: and an Expires: date in the future, as RFC 9116 asks.

#A tool that checks all of this on a schedule

We made Gatepost, a command-line tool that checks a domain the way these servers see it: DNS and CAA with inheritance from the parent domain, MX and null MX, one valid SPF record with every include followed and the ten lookups counted, DMARC including a parent's sp=, the certificate and TLS versions, the redirect, security headers, cookie flags and security.txt. Each finding has a code and a severity, the report comes as a table, JSON, Markdown or CSV, and exit codes make it a nightly job or a CI step for a list of domains. Its tests run against local DNS and HTTPS servers, and only the Linux build was run. It counts the lookups the way receivers do: a provider included from two places, like the shared cloud record here, counts each time, so for shop.example it reports 12 lookups and no loop. A real loop is reported with its chain, and more than two void lookups are flagged as well. It is listed as Coming soon.

#Frequently asked questions

Why does my SPF record fail when it looks fine?

Usually because it needs more than ten DNS lookups once every include is followed, or because the domain has two records that start with v=spf1. Both give a permanent error, whatever the record says.

Should SPF end with ~all or -all?

Either is a clear policy. With DMARC in place, receivers act on the DMARC policy, so ~all is the common choice while you are still finding all your senders; -all is right for a domain that sends no mail.

Is p=none in DMARC enough?

It meets the requirement of the big mailbox providers and gives you reports, but it does not stop anyone from sending mail as your domain. Treat it as the first step towards quarantine and reject.

Do I need a CAA record?

No, but it limits who can issue certificates for your domain. If you add one, list every authority you use, including the one your hosting or CDN uses for automatic certificates.

#Sources

• RFC 7208: Sender Policy Framework (SPF)
• RFC 7489: DMARC
• RFC 7505: A "Null MX" No Service Resource Record
• RFC 8659: DNS Certification Authority Authorization (CAA)
• RFC 9116: security.txt
• Google: Email sender guidelines