RFC 7817: Updated TLS Server Identity Check for Email

Current Standard Transport Security TLSCertificate VerificationSMTP Security

ELI5: When you connect to a website over HTTPS, your browser checks that the certificate matches the domain name. Email servers need to do the same thing when they connect to each other over TLS — but email routing through MX records makes it more complicated. This RFC defines exactly which name the sending server should check the certificate against.

Why This Exists

When an SMTP client starts a TLS connection to a remote mail server (via STARTTLS per RFC 3207 or implicit TLS per RFC 8314), it receives a certificate. But which hostname should appear in that certificate?

In email, you do not connect directly to the recipient's domain. You look up the MX record for example.com, which might point to mail.hosting-provider.net. Should the certificate say example.com or mail.hosting-provider.net? The answer matters because:

Checking the wrong name means legitimate certificates get rejected
Not checking at all means TLS provides encryption but no authentication — a man-in-the-middle can present any certificate
Email hosting providers serve thousands of domains from the same servers and cannot get certificates for every customer domain

How It Works

RFC 7817 defines a clear hierarchy for which reference identifiers a client should check against the server's certificate, depending on how the server was discovered:

Identity Check Order

MX hostname — if the server was found via MX lookup, check the certificate against the MX hostname (e.g., mail.hosting-provider.net), not the recipient domain.
Recipient domain — if there is no MX record and the client falls back to an A/AAAA record for the domain itself, check against the recipient domain.
Explicit configuration — if the server was configured manually (e.g., a submission server), check against the configured hostname.

TLS Connection Flow

; Step 1: Resolve MX for recipient domain
dig example.com MX
example.com.  IN  MX  10 mail.provider.net.

; Step 2: Connect to MX host, start SMTP
220 mail.provider.net ESMTP ready
EHLO sender.example.org
250-mail.provider.net
250-STARTTLS
STARTTLS
220 Go ahead

; Step 3: TLS handshake — check certificate against "mail.provider.net"
; Certificate SAN: DNS:mail.provider.net, DNS:*.provider.net
; Match found → identity verified

Key Technical Details

Certificate Matching Rules

Field	Check	Notes
Subject Alternative Name (SAN)	Preferred	Check DNS-ID entries in the SAN extension first
Common Name (CN)	Fallback	Only if no SAN DNS-ID entries exist (deprecated practice)
Wildcard	Left-most label only	`*.provider.net` matches `mail.provider.net` but not `a.b.provider.net`

Reference Identifier by Discovery Method

Discovery Method	Check Certificate Against	Example
MX lookup	MX hostname	Certificate must match `mail.provider.net`
A/AAAA fallback (no MX)	Recipient domain	Certificate must match `example.com`
SRV record (RFC 6186)	SRV target hostname	Certificate must match the SRV target
Manual configuration	Configured hostname	Certificate must match what the admin set
MTA-STS policy (RFC 8461)	MX hostnames in the policy	Policy constrains which MX names are acceptable

Interaction with DANE

When DANE (RFC 7672) is deployed, the TLSA record in DNS pins the certificate or public key for the MX hostname. DANE provides a stronger identity guarantee than traditional CA-based verification because it does not depend on trusting a certificate authority — the domain owner publishes the expected certificate data directly in DNSSEC-signed DNS.

Common Mistakes

Checking the certificate against the recipient domain. If MX for example.com points to mail.google.com, the certificate will say mail.google.com, not example.com. Checking the wrong name causes TLS failures or forces you to skip verification entirely.
Falling back to unauthenticated TLS silently. Many MTAs use "opportunistic TLS" — they encrypt if possible but do not verify certificates. This protects against passive eavesdropping but not active attacks. Use MTA-STS or DANE to require authenticated connections.
Misconfiguring certificates on MX hosts. If your MX record points to mx.yourdomain.com but your certificate only covers yourdomain.com, senders that verify certificates will fail. Ensure your certificate's SAN includes your MX hostname.
Ignoring certificate chain validation. Verifying the hostname is necessary but not sufficient. The full certificate chain must validate to a trusted root CA. Expired or self-signed certificates will still fail even if the name matches.
Not renewing certificates before expiry. Expired certificates on your MX servers cause TLS handshake failures. Senders enforcing MTA-STS will refuse to deliver mail to a server with an expired certificate.

Deliverability Impact

MTA-STS enforcement depends on this. When a receiving domain publishes an MTA-STS policy, sending servers must verify the MX certificate per RFC 7817. If your MX certificate is misconfigured, senders enforcing MTA-STS (including Gmail) will not deliver mail to you.
Google and Microsoft verify certificates. Major providers increasingly perform certificate verification for outbound mail. Misconfigured certificates on your receiving infrastructure can cause delivery failures from these senders.
DANE/TLSA provides the strongest guarantee. DANE ties the certificate to DNSSEC, removing the CA trust dependency entirely. Combined with RFC 7817 identity checks, it provides authenticated, encrypted email transport.
TLS reporting reveals problems. SMTP TLS Reporting (RFC 8460) sends you reports when senders experience TLS failures connecting to your servers, including certificate verification failures. Publish a TLSRPT record and monitor these reports.