← RFC Reference

TLS and Email Security

How TLS protects email in transit — STARTTLS, implicit TLS, MTA-STS, DANE, TLSRPT — and why opportunistic encryption is not enough.

The Problem: SMTP Was Born Unencrypted

SMTP was designed in 1982 (RFC 821) as a plaintext protocol. Every command, every response, every header, every byte of message content traveled across the network unencrypted. Passwords sent via AUTH were effectively shouted across the room.

TLS (Transport Layer Security) was layered onto SMTP to fix this. But unlike HTTPS — where TLS is mandatory and universal — email TLS has evolved through a series of imperfect compromises that are still being resolved today.

STARTTLS: Opportunistic Encryption

STARTTLS (RFC 3207) upgrades an existing plaintext SMTP connection to TLS. The process:

Key points about STARTTLS:

• The connection starts plaintext and upgrades mid-stream. The initial EHLO and STARTTLS commands themselves are unencrypted.
• After STARTTLS completes, the client must send EHLO again. The server may advertise different capabilities over the encrypted connection.
• STARTTLS is used on port 25 (server-to-server) and port 587 (client submission).
• STARTTLS on port 25 is opportunistic by default: if the server does not advertise STARTTLS, or if the TLS handshake fails, most sending MTAs fall back to plaintext and send the message anyway.

The STARTTLS stripping attack

Because STARTTLS begins in plaintext, a man-in-the-middle attacker can modify the server's EHLO response to remove the STARTTLS advertisement. The sending MTA thinks the server does not support TLS and proceeds in plaintext. The attacker can then read and modify the message.

This is not a theoretical attack. It has been observed in the wild, particularly in countries that conduct mass surveillance. Opportunistic STARTTLS provides no protection against active attackers — it only protects against passive eavesdropping.

Implicit TLS: Encryption from the First Byte

RFC 8314 recommends implicit TLS for mail submission (client to server). Instead of starting plaintext and upgrading, the connection is TLS from the very first byte.

• Port 465: Implicit TLS for message submission. The client opens a TLS connection immediately. No STARTTLS step. No opportunity for stripping.
• Port 993: Implicit TLS for IMAP.
• Port 995: Implicit TLS for POP3.

RFC 8314 explicitly declares that plaintext SMTP submission on port 587 without TLS is obsolete, and that implicit TLS on port 465 is the preferred submission path. In practice, port 587 with STARTTLS remains widely deployed, but the direction is clear: implicit TLS is the future.

However, implicit TLS only applies to the submission path (your client to your server). Server-to-server delivery on port 25 still uses STARTTLS because there is no universally deployed mechanism for servers to know in advance that the recipient's MX supports implicit TLS. This is where MTA-STS and DANE come in.

MTA-STS: Policy-Based TLS Enforcement

MTA-STS (RFC 8461) solves the STARTTLS stripping problem by allowing a domain to publish a policy that says "always require TLS when delivering mail to my MX servers." The policy is published via HTTPS (not DNS), which provides its own authentication through the web PKI.

How MTA-STS works

1. The domain publishes a DNS TXT record signaling that an MTA-STS policy exists:

2. The sending MTA fetches the policy over HTTPS from a well-known URL:

3. The policy file specifies the mode and valid MX hostnames:

4. The sending MTA caches the policy (for max_age seconds) and enforces it on all future connections. It will:
   • Require STARTTLS on port 25 (never fall back to plaintext)
   • Validate the server's TLS certificate against the MX hostname
   • Only deliver to the MX hostnames listed in the policy

MTA-STS modes:

• enforce: Require TLS and valid certificates. Fail delivery if TLS cannot be established.
• testing: Attempt TLS but deliver anyway if it fails. Send failure reports via TLSRPT.
• none: No policy. Used to explicitly revoke a previous policy.

MTA-STS has the same trust model as the web: it relies on Certificate Authorities. If a CA is compromised, an attacker could forge a certificate for your MX. DANE provides an alternative trust model based on DNSSEC.

DANE: DNS-Based Certificate Verification

DANE (RFC 7672) uses DNSSEC-signed TLSA records to associate a TLS certificate (or its public key) directly with an MX hostname. This removes the dependency on Certificate Authorities.

The TLSA record fields:

• 3 (Certificate Usage) — Domain-issued certificate (no CA required)
• 1 (Selector) — Match against the subject public key
• 1 (Matching Type) — SHA-256 hash of the public key
• a]b2c3d4e5f6... — The hash value

How DANE works for email

1. The sending MTA resolves the MX record and gets mx1.example.com.
2. It queries for a TLSA record at _25._tcp.mx1.example.com.
3. If a DNSSEC-validated TLSA record exists, the MTA knows:
   • TLS is mandatory (no fallback to plaintext).
   • The server's certificate must match the TLSA record.
4. The MTA connects, performs STARTTLS, and validates the server's certificate against the TLSA record rather than (or in addition to) the CA system.

DANE's strength: it does not depend on Certificate Authorities. Its weakness: it requires DNSSEC, which many domains have not deployed. Without DNSSEC, DANE records cannot be trusted (an attacker who can manipulate DNS can manipulate TLSA records too).

DANE vs. MTA-STS

Both mechanisms can coexist. A domain can publish both MTA-STS and DANE policies. Senders that support DANE use it; those that do not can fall back to MTA-STS.

TLSRPT: TLS Reporting

SMTP TLS Reporting (RFC 8460) lets a domain receive reports about TLS connection failures from sending servers. It is the TLS equivalent of DMARC aggregate reports.

Sending MTAs that support TLSRPT will send daily JSON reports detailing:

• Total connection attempts to your MX
• Successful TLS negotiations
• Failed TLS negotiations (and why: certificate errors, STARTTLS failures, MTA-STS policy violations)
• Connections that fell back to plaintext

TLSRPT is essential during MTA-STS deployment. Start with mode: testing in your MTA-STS policy and monitor TLSRPT reports. When you see zero (or acceptable) failures, move to mode: enforce.

Certificate Management

TLS requires certificates, and certificate management is one of the most common operational failure points in email security.

Certificates for mail servers

• The certificate must match the MX hostname. If your MX record points to mx1.example.com, the certificate must be valid for mx1.example.com. A certificate for example.com or www.example.com will not work.
• Use certificates from a public CA. Let's Encrypt works perfectly for mail servers. Self-signed certificates will cause TLS validation failures for senders that check (which includes anyone using MTA-STS or DANE).
• Automate renewal. Certificate expiry is the most common TLS failure. Use ACME (Let's Encrypt) or your CA's automation to ensure certificates are renewed well before expiry.
• Include all MX hostnames. If you have multiple MX servers, each needs a valid certificate for its hostname. A SAN (Subject Alternative Name) certificate can cover multiple hostnames.

Certificate errors and their impact

This table reveals the core problem with opportunistic TLS: without MTA-STS or DANE, most certificate errors are silently ignored. The connection is encrypted, but not authenticated — you might be encrypting your message to an attacker's server.

The State of Email Encryption

Email encryption exists at two levels, and they are often confused:

Transport encryption (TLS)

TLS encrypts the connection between servers. It protects against eavesdropping while the message is in transit between any two points. Once the message arrives at the destination server, it is decrypted and stored in plaintext (typically). TLS protects the transport, not the message.

Transport encryption is hop-by-hop: the message is encrypted between your server and the next server, then decrypted, then re-encrypted for the next hop. Each intermediate server has access to the plaintext message.

End-to-end encryption (S/MIME, PGP)

End-to-end encryption (S/MIME per RFC 8551, or PGP/OpenPGP) encrypts the message content itself, so only the intended recipient with the correct private key can decrypt it. The message remains encrypted at rest on the server and during every transit hop.

End-to-end encryption adoption in email remains extremely low. The reasons are practical:

• Key management is hard. Both sender and recipient must manage cryptographic keys. There is no universal key directory.
• Usability is poor. Most email clients make encryption cumbersome. The average user cannot (and should not need to) manage PGP keys.
• Server-side features break. If the server cannot read the message, it cannot index it, search it, filter spam, or apply rules.
• Interoperability is limited. S/MIME and PGP are not compatible with each other.

For most email, transport-level TLS is the practical encryption layer. End-to-end encryption remains niche, used in high-security environments where the operational burden is justified.

Require TLS (RFC 8689)

RFC 8689 defines a per-message mechanism to require TLS. The sender adds a REQUIRETLS SMTP extension to individual messages:

When REQUIRETLS is set:

• The receiving server must use TLS to deliver or relay the message. If TLS is not available at the next hop, the message is bounced rather than sent in plaintext.
• TLS certificate verification must succeed.
• All downstream hops must also support and honor REQUIRETLS.

REQUIRETLS is message-level — you can use it for sensitive messages without requiring TLS for all mail. However, adoption is still limited. Most servers do not support it yet.

Practical Deployment Guide

Here is a recommended sequence for deploying TLS for your domain's email:

Step 1: Ensure TLS on your MX servers

Get valid certificates from a public CA for every MX hostname. Automate renewal. Test that STARTTLS works correctly from external senders.

Step 2: Deploy TLSRPT

Publish a _smtp._tls TXT record to receive TLS failure reports. Start monitoring before you enforce anything.

Step 3: Deploy MTA-STS in testing mode

Publish the DNS TXT record and the HTTPS policy file with mode: testing. Monitor TLSRPT reports for failures. Investigate and fix any issues.

Step 4: Move MTA-STS to enforce mode

Once TLSRPT reports show clean results, change the mode to enforce. Increment the id in the DNS TXT record to signal the policy change.

Step 5: Consider DANE (if you use DNSSEC)

If your domain is DNSSEC-signed, publish TLSA records for your MX servers. This provides an additional layer of certificate pinning independent of the CA system.

What Can Go Wrong

Expired certificate causes delivery failures

Your MX server's certificate expires. Without MTA-STS, most senders ignore the error and deliver anyway (insecure but functional). With MTA-STS in enforce mode, senders that validate certificates will defer delivery. Messages queue on the sender's side. If you do not fix it within the sender's retry window (typically 4–5 days), messages bounce. The fix: automate certificate renewal with Let's Encrypt / ACME.

MTA-STS policy mismatch after MX change

You add a new MX server (mx3.example.com) but forget to add it to your MTA-STS policy file. Senders that have cached the old policy reject connections to the new MX because it is not in the policy's allowed list. The fix: always update the MTA-STS policy file before adding new MX records, and increment the policy id in DNS.

STARTTLS stripping undetected

Without MTA-STS or DANE, an attacker strips STARTTLS from the server's EHLO response. Messages are delivered in plaintext. Neither sender nor receiver is aware. The fix: deploy MTA-STS to make TLS stripping detectable and preventable.

DANE TLSA record becomes stale

You rotate your TLS certificate but forget to update the TLSA record in DNS. Senders that validate DANE see a mismatch and refuse to deliver. The fix: always update TLSA records before deploying a new certificate. Use automation that coordinates certificate deployment with DNS updates.

Key Takeaways

• Opportunistic STARTTLS is better than nothing, but provides no protection against active attacks. An attacker who can modify network traffic can strip TLS from the connection.
• MTA-STS prevents TLS stripping by publishing a policy that senders cache and enforce. It relies on the web PKI (Certificate Authorities).
• DANE prevents TLS stripping by publishing certificate data in DNSSEC-signed DNS records. It does not depend on CAs but requires DNSSEC.
• Deploy both if possible. MTA-STS has broader sender support; DANE is stronger but requires DNSSEC. Together they cover the most ground.
• TLSRPT is essential for visibility. Without it, you have no idea how often TLS fails for inbound mail to your domain.
• Certificate management is an operational concern. Automate renewal. Match certificates to MX hostnames. Coordinate certificate rotation with DANE TLSA updates.
• TLS protects transport, not content. For end-to-end message encryption, you need S/MIME or PGP — but the usability and adoption barriers remain high.
• Implicit TLS (port 465) is the future for submission. For server-to-server delivery, STARTTLS with MTA-STS/DANE enforcement is the current best practice.

Further Reading

Related RFCs