Check for 'v=spf1 -all' SPF records as a way to reject more bad domains

Author: JoshData

README.md

@@ -106,7 +106,7 @@ addresses are allowed when passing a `bytes`) and:
 
 When an email address is not valid, `validate_email` raises either an
 `EmailSyntaxError` if the form of the address is invalid or an
-`EmailUndeliverableError` if the domain name fails the DNS check. Both
+`EmailUndeliverableError` if the domain name fails DNS checks. Both
 exception classes are subclasses of `EmailNotValidError`, which in turn
 is a subclass of `ValueError`.
 
@@ -120,14 +120,17 @@ they will probably give you grief if you're using email for login. (See
 later in the document about that.)
 
 The validator checks that the domain name in the email address has a
-(non-null) MX DNS record indicating that it is configured for email.
+DNS MX record (except a NULL MX record) indicating that it can receive
+email and that it does not have a reject-all SPF record (`v=spf1 -all`)
+which would indicate that it cannot send email.
+(A/AAAA-record MX fallback is also checked.)
 There is nothing to be gained by trying to actually contact an SMTP
 server, so that's not done here. For privacy, security, and practicality
 reasons servers are good at not giving away whether an address is
 deliverable or not: email addresses that appear to accept mail at first
 can bounce mail after a delay, and bounced mail may indicate a temporary
 failure of a good email address (sometimes an intentional failure, like
-greylisting). (A/AAAA-record fallback is also checked.)
+greylisting).
 
 ### Options
 
@@ -138,7 +141,7 @@ The `validate_email` function also accepts the following keyword arguments
     require the
     [SMTPUTF8](https://tools.ietf.org/html/rfc6531) extension.
 
-`check_deliverability=True`: Set to `False` to skip the domain name MX DNS record check.
+`check_deliverability=True`: Set to `False` to skip the domain name DNS record checks.
 
 `allow_empty_local=False`: Set to `True` to allow an empty local part (i.e.
     `@example.com`), e.g. for validating Postfix aliases.
@@ -323,9 +326,7 @@ ValidatedEmail(
   ascii_email='test@joshdata.me',
   ascii_local_part='test',
   ascii_domain='joshdata.me',
-  smtputf8=False,
-  mx=[(10, 'box.occams.info')],
-  mx_fallback_type=None)
+  smtputf8=False)
 ```
 
 For the fictitious address `example@ツ.life`, which has an
@@ -392,6 +393,7 @@ are:
 | `smtputf8` | A boolean indicating that the [SMTPUTF8](https://tools.ietf.org/html/rfc6531) feature of your mail relay will be required to transmit messages to this address because the local part of the address has non-ASCII characters (the local part cannot be IDNA-encoded). If `allow_smtputf8=False` is passed as an argument, this flag will always be false because an exception is raised if it would have been true. |
 | `mx` | A list of (priority, domain) tuples of MX records specified in the DNS for the domain (see [RFC 5321 section 5](https://tools.ietf.org/html/rfc5321#section-5)). May be `None` if the deliverability check could not be completed because of a temporary issue like a timeout. |
 | `mx_fallback_type` | `None` if an `MX` record is found. If no MX records are actually specified in DNS and instead are inferred, through an obsolete mechanism, from A or AAAA records, the value is the type of DNS record used instead (`A` or `AAAA`). May be `None` if the deliverability check could not be completed because of a temporary issue like a timeout. |
+| `spf` | Any SPF record found while checking deliverability. |
 
 Assumptions
 -----------
@@ -401,10 +403,12 @@ strictly conform to the standards. Many email address forms are obsolete
 or likely to cause trouble:
 
 * The validator assumes the email address is intended to be
-  deliverable on the public Internet. The domain part
-  of the email address must be a resolvable domain name.
-  [Special Use Domain Names](https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml)
-  and their subdomains are always considered invalid (except see
+  usable on the public Internet. The domain part
+  of the email address must be a resolvable domain name
+  (without NULL MX or SPF -all DNS records) if deliverability
+  checks are turned on.
+  Most [Special Use Domain Names](https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml)
+  and their subdomains are considered invalid (except see
   the `test_environment` parameter above).
 * The "quoted string" form of the local part of the email address (RFC
   5321 4.1.2) is not permitted --- no one uses this anymore anyway.

email_validator/__init__.py

@@ -356,9 +356,8 @@ def validate_email(
         deliverability_info = validate_email_deliverability(
             ret["domain"], ret["domain_i18n"], timeout, dns_resolver
         )
-        if "mx" in deliverability_info:
-            ret.mx = deliverability_info["mx"]
-            ret.mx_fallback_type = deliverability_info["mx-fallback"]
+        for key, value in deliverability_info.items():
+            setattr(ret, key, value)
 
     return ret
 
@@ -588,6 +587,8 @@ def validate_email_deliverability(domain, domain_i18n, timeout=DEFAULT_TIMEOUT,
         dns_resolver = dns.resolver.get_default_resolver()
         dns_resolver.lifetime = timeout
 
+    deliverability_info = {}
+
     def dns_resolver_resolve_shim(domain, record):
         try:
             # dns.resolver.Resolver.resolve is new to dnspython 2.x.
@@ -611,39 +612,61 @@ def dns_resolver_resolve_shim(domain, record):
             raise dns.exception.Timeout()
 
         try:
-            # Try resolving for MX records and get them in sorted priority order
-            # as (priority, qname) pairs.
+            # Try resolving for MX records.
             response = dns_resolver_resolve_shim(domain, "MX")
+
+            # For reporting, put them in priority order and remove the trailing dot in the qnames.
             mtas = sorted([(r.preference, str(r.exchange).rstrip('.')) for r in response])
-            mx_fallback = None
 
-            # Do not permit delivery if there is only a "null MX" record (whose value is
-            # (0, ".") but we've stripped trailing dots, so the 'exchange' is just "").
+            # Remove "null MX" records from the list (their value is (0, ".") but we've stripped
+            # trailing dots, so the 'exchange' is just ""). If there was only a null MX record,
+            # email is not deliverable.
             mtas = [(preference, exchange) for preference, exchange in mtas
                     if exchange != ""]
             if len(mtas) == 0:
                 raise EmailUndeliverableError("The domain name %s does not accept email." % domain_i18n)
 
+            deliverability_info["mx"] = mtas
+            deliverability_info["mx_fallback_type"] = None
+
         except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
 
             # If there was no MX record, fall back to an A record.
             try:
                 response = dns_resolver_resolve_shim(domain, "A")
-                mtas = [(0, str(r)) for r in response]
-                mx_fallback = "A"
+                deliverability_info["mx"] = [(0, str(r)) for r in response]
+                deliverability_info["mx_fallback_type"] = "A"
             except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
 
                 # If there was no A record, fall back to an AAAA record.
                 try:
                     response = dns_resolver_resolve_shim(domain, "AAAA")
-                    mtas = [(0, str(r)) for r in response]
-                    mx_fallback = "AAAA"
+                    deliverability_info["mx"] = [(0, str(r)) for r in response]
+                    deliverability_info["mx_fallback_type"] = "AAAA"
                 except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
 
                     # If there was no MX, A, or AAAA record, then mail to
                     # this domain is not deliverable.
                     raise EmailUndeliverableError("The domain name %s does not exist." % domain_i18n)
 
+        try:
+            # Check for a SPF reject all ("v=spf1 -all") record which indicates
+            # no emails are sent from this domain, which like a NULL MX record
+            # would indicate that the domain is not used for email.
+            response = dns_resolver_resolve_shim(domain, "TXT")
+            for rec in response:
+                value = b"".join(rec.strings)
+                if value.startswith(b"v=spf1 "):
+                    deliverability_info["spf"] = value.decode("ascii", errors='replace')
+                    if value == b"v=spf1 -all":
+                        raise EmailUndeliverableError("The domain name %s does not send email." % domain_i18n)
+        except dns.resolver.NoAnswer:
+            # No TXT records means there is no SPF policy, so we cannot take any action.
+            pass
+        except (dns.resolver.NoNameservers, dns.resolver.NXDOMAIN):
+            # Failure to resolve at this step will be ignored.
+            pass
+
     except dns.exception.Timeout:
         # A timeout could occur for various reasons, so don't treat it as a failure.
         return {
@@ -660,10 +683,7 @@ def dns_resolver_resolve_shim(domain, record):
             "There was an error while checking if the domain name in the email address is deliverable: " + str(e)
         )
 
-    return {
-        "mx": mtas,
-        "mx-fallback": mx_fallback,
-    }
+    return deliverability_info
 
 
 def main():

tests/test_main.py

@@ -329,8 +329,8 @@ def test_dict_accessor():
 
 def test_deliverability_found():
     response = validate_email_deliverability('gmail.com', 'gmail.com')
-    assert response.keys() == {'mx', 'mx-fallback'}
-    assert response['mx-fallback'] is None
+    assert response.keys() == {'mx', 'mx_fallback_type', 'spf'}
+    assert response['mx_fallback_type'] is None
     assert len(response['mx']) > 1
     assert len(response['mx'][0]) == 2
     assert isinstance(response['mx'][0][0], int)