Some README improvements, remove references to obsoleted RFCs, add a test for an obsoleted quoted-string syntax

Author: JoshData

README.md

@@ -14,22 +14,23 @@ Key features:
 
 * Checks that an email address has the correct syntax --- good for
   registration/login forms or other uses related to identifying users.
-  By default, rejects obsolete email address syntax that you'd find unexpected.
 * Gives friendly English error messages when validation fails that you
   can display to end-users.
 * Checks deliverability (optional): Does the domain name resolve?
   (You can override the default DNS resolver to add query caching.)
 * Supports internationalized domain names and internationalized local parts.
-  Blocks unsafe characters for your safety.
+* Rejects addresses with unsafe Unicode characters, obsolete email address
+  syntax that you'd find unexpected, special use domain names like
+  `@localhost`, and domains without a dot by default. This is an
+  opinionated library!
 * Normalizes email addresses (important for internationalized
   and quoted-string addresses! see below).
 * Python type annotations are used.
 
-This library does NOT permit obsolete forms of email addresses by default,
-so if you need strict validation against the email specs exactly, use
-[pyIsEmail](https://github.com/michaelherold/pyIsEmail) or try
-[flanker](https://github.com/mailgun/flanker) if you are parsing the
-"To:" line of an email.
+This is an opinionated library. You should definitely also consider using
+the less-opinionated [pyIsEmail](https://github.com/michaelherold/pyIsEmail) and
+[flanker](https://github.com/mailgun/flanker) if they are better for your
+use case.
 
 [![Build Status](https://github.com/JoshData/python-email-validator/actions/workflows/test_and_build.yaml/badge.svg)](https://github.com/JoshData/python-email-validator/actions/workflows/test_and_build.yaml)
 
@@ -57,21 +58,23 @@ account in your application, you might do this:
 ```python
 from email_validator import validate_email, EmailNotValidError
 
-email = "my+address@mydomain.tld"
-is_new_account = True # False for login pages
+email = "my+address@example.org"
 
 try:
-  # Check that the email address is valid.
-  validation = validate_email(email, check_deliverability=is_new_account)
 
-  # Take the normalized form of the email address
-  # for all logic beyond this point (especially
-  # before going to a database query where equality
-  # may not take into account Unicode normalization).  
+  # Check that the email address is valid. Turn on check_deliverability
+  # for first-time validations like on account creation pages (but not
+  # login pages).
+  validation = validate_email(email, check_deliverability=False)
+
+  # After this point, use only the normalized form of the email address,
+  # especially before going to a database query.
   email = validation.email
+
 except EmailNotValidError as e:
-  # Email is not valid.
-  # The exception message is human-readable.
+
+  # The exception message is human-readable explanation of why it's
+  # not a valid (or deliverable) email address.
   print(str(e))
 ```
 
@@ -108,12 +111,15 @@ that no one uses anymore even though they are still valid and deliverable, since
 they will probably give you grief if you're using email for login. (See
 later in the document about how to allow some obsolete forms.)
 
-The validator checks that the domain name in the email address has a
-DNS MX record (except a NULL MX record) indicating that it can receive
-email (or a fallback A-record, see below).
-There is nothing to be gained by trying to actually contact an SMTP
-server, so that's not done here. For privacy, security, and practicality
-reasons servers are good at not giving away whether an address is
+The validator optionally checks that the domain name in the email address has
+a DNS MX record indicating that it can receive email. (Except a Null MX record.
+If there is no MX record, a fallback A/AAAA-record is permitted, unless
+a reject-all SPF record is present.) DNS is slow and sometimes unavailable or
+unreliable, so consider whether these checks are useful for your use case and
+turn them off if they aren't.
+There is nothing to be gained by trying to actually contact an SMTP server, so
+that's not done here. For privacy, security, and practicality reasons, servers
+are good at not giving away whether an address is
 deliverable or not: email addresses that appear to accept mail at first
 can bounce mail after a delay, and bounced mail may indicate a temporary
 failure of a good email address (sometimes an intentional failure, like
@@ -124,11 +130,11 @@ greylisting).
 The `validate_email` function also accepts the following keyword arguments
 (defaults are as shown below):
 
-`check_deliverability=True`: If true, a DNS query is made to check that a non-null MX record is present for the domain-part of the email address (or if not, an A/AAAA record as an MX fallback can be present but in that case a reject-all SPF record must not be present). Set to `False` to skip this DNS-based check. DNS is slow and sometimes unavailable, so consider whether these checks are useful for your use case. It is recommended to pass `False` when performing validation for login pages (but not account creation pages) since re-validation of a previously validated domain in your database by querying DNS at every login is probably undesirable. You can also set `email_validator.CHECK_DELIVERABILITY` to `False` to turn this off for all calls by default.
+`check_deliverability=True`: If true, DNS queries are made to check that the domain name in the email address (the part after the @-sign) can receive mail, as described above. Set to `False` to skip this DNS-based check. It is recommended to pass `False` when performing validation for login pages (but not account creation pages) since re-validation of a previously validated domain in your database by querying DNS at every login is probably undesirable. You can also set `email_validator.CHECK_DELIVERABILITY` to `False` to turn this off for all calls by default.
 
-`dns_resolver=None`: Pass an instance of [dns.resolver.Resolver](https://dnspython.readthedocs.io/en/latest/resolver-class.html) to control the DNS resolver including setting a timeout and [a cache](https://dnspython.readthedocs.io/en/latest/resolver-caching.html). The `caching_resolver` function shown above is a helper function to construct a dns.resolver.Resolver with a [LRUCache](https://dnspython.readthedocs.io/en/latest/resolver-caching.html#dns.resolver.LRUCache). Reuse the same resolver instance across calls to `validate_email` to make use of the cache.
+`dns_resolver=None`: Pass an instance of [dns.resolver.Resolver](https://dnspython.readthedocs.io/en/latest/resolver-class.html) to control the DNS resolver including setting a timeout and [a cache](https://dnspython.readthedocs.io/en/latest/resolver-caching.html). The `caching_resolver` function shown below is a helper function to construct a dns.resolver.Resolver with a [LRUCache](https://dnspython.readthedocs.io/en/latest/resolver-caching.html#dns.resolver.LRUCache). Reuse the same resolver instance across calls to `validate_email` to make use of the cache.
 
-`test_environment=False`: DNS-based deliverability checks are disabled and  `test` and `subdomain.test` domain names are permitted (see below). You can also set `email_validator.TEST_ENVIRONMENT` to `True` to turn it on for all calls by default.
+`test_environment=False`: If `True`, DNS-based deliverability checks are disabled and  `test` and `**.test` domain names are permitted (see below). You can also set `email_validator.TEST_ENVIRONMENT` to `True` to turn it on for all calls by default.
 
 `allow_smtputf8=True`: Set to `False` to prohibit internationalized addresses that would
     require the
@@ -160,15 +166,15 @@ while True:
 This library rejects email addresess that use the [Special Use Domain Names](https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml) `invalid`, `localhost`, `test`, and some others by raising `EmailSyntaxError`. This is to protect your system from abuse: You probably don't want a user to be able to cause an email to be sent to `localhost`. However, in your non-production test environments you may want to use `@test` or `@myname.test` email addresses. There are three ways you can allow this:
 
 1. Add `test_environment=True` to the call to `validate_email` (see above).
-2. Set `email_validator.TEST_ENVIRONMENT` to `True`.
-3. Remove the special-use domain name that you want to use from `email_validator.SPECIAL_USE_DOMAIN_NAMES`:
+2. Set `email_validator.TEST_ENVIRONMENT` to `True` globally.
+3. Remove the special-use domain name that you want to use from `email_validator.SPECIAL_USE_DOMAIN_NAMES`, e.g.:
 
 ```python
 import email_validator
 email_validator.SPECIAL_USE_DOMAIN_NAMES.remove("test")
 ```
 
-It is tempting to use `@example.com/net/org` in tests. These domains are reserved to IANA for use in documentation so there is no risk of accidentally emailing someone at those domains. But beware that this library will reject these domain names if DNS-based deliverability checks are not disabled because these domains do not resolve to domains that accept email. In tests, consider using your own domain name or `@test` or `@myname.test` instead.
+It is tempting to use `@example.com/net/org` in tests. They are *not* in this library's `SPECIAL_USE_DOMAIN_NAMES` list so you can, but shouldn't, use them. These domains are reserved to IANA for use in documentation so there is no risk of accidentally emailing someone at those domains. But beware that this library will nevertheless reject these domain names if DNS-based deliverability checks are not disabled because these domains do not resolve to domains that accept email. In tests, consider using your own domain name or `@test` or `@myname.test` instead.
 
 Internationalized email addresses
 ---------------------------------
@@ -255,17 +261,23 @@ change the user's login information without telling them.)
 Normalization
 -------------
 
+### Unicode Normalization
+
 The use of Unicode in email addresses introduced a normalization
 problem. Different Unicode strings can look identical and have the same
 semantic meaning to the user. The `email` field returned on successful
 validation provides the correctly normalized form of the given email
-address:
+address.
+
+For example, the CJK fullwidth Latin letters are considered semantically
+equivalent in domain names to their ASCII counterparts. This library
+normalizes them to their ASCII counterparts:
 
 ```python
 valid = validate_email("me@Ｄｏｍａｉｎ.com")
-email = valid.ascii_email
-print(email)
-# prints: me@domain.com
+print(valid.email)
+print(valid.ascii_email)
+# prints "me@domain.com" twice
 ```
 
 Because an end-user might type their email address in different (but
@@ -292,6 +304,8 @@ and conversion from Punycode to Unicode characters.
 3.1](https://tools.ietf.org/html/rfc6532#section-3.1) and [RFC 5895
 (IDNA 2008) section 2](http://www.ietf.org/rfc/rfc5895.txt).)
 
+### Other Normalization
+
 Normalization is also applied to quoted-string local parts and domain
 literal IPv6 addresses if you have allowed them by the `allow_quoted_local`
 and `allow_domain_literal` options. In quoted-string local parts, unnecessary

email_validator/rfc_constants.py

@@ -2,16 +2,13 @@
 
 import re
 
-# Based on RFC 2822 section 3.2.4 / RFC 5322 section 3.2.3, these
-# characters are permitted in email addresses (not taking into
-# account internationalization):
+# Based on RFC 5322 3.2.3, these characters are permitted in email
+# addresses (not taking into account internationalization) separated by dots:
 ATEXT = r'a-zA-Z0-9_!#\$%&\'\*\+\-/=\?\^`\{\|\}~'
 ATEXT_RE = re.compile('[.' + ATEXT + ']')  # ATEXT plus dots
-
-# A "dot atom text", per RFC 2822 3.2.4:
 DOT_ATOM_TEXT = re.compile('[' + ATEXT + ']+(?:\\.[' + ATEXT + r']+)*\Z')
 
-# RFC 6531 section 3.3 extends the allowed characters in internationalized
+# RFC 6531 3.3 extends the allowed characters in internationalized
 # addresses to also include three specific ranges of UTF8 defined in
 # RFC 3629 section 4, which appear to be the Unicode code points from
 # U+0080 to U+10FFFF.
@@ -20,7 +17,7 @@
 DOT_ATOM_TEXT_INTL = re.compile('[' + ATEXT_INTL + ']+(?:\\.[' + ATEXT_INTL + r']+)*\Z')
 
 # The domain part of the email address, after IDNA (ASCII) encoding,
-# must also satisfy the requirements of RFC 952/RFC 1123 Section 2.1 which
+# must also satisfy the requirements of RFC 952/RFC 1123 2.1 which
 # restrict the allowed characters of hostnames further.
 ATEXT_HOSTNAME_INTL = re.compile(r"[a-zA-Z0-9\-\." + "\u0080-\U0010FFFF" + "]")
 HOSTNAME_LABEL = r'(?:(?:[a-zA-Z0-9][a-zA-Z0-9\-]*)?[a-zA-Z0-9])'
@@ -30,7 +27,7 @@
 # Domain literal (RFC 5322 3.4.1)
 DOMAIN_LITERAL_CHARS = re.compile(r"[\u0021-\u00FA\u005E-\u007E]")
 
-# Quoted-string local part (RFC 5321 4.1.2, internationalized by RFC 6531 section 3.3)
+# Quoted-string local part (RFC 5321 4.1.2, internationalized by RFC 6531 3.3)
 # The permitted characters in a quoted string are the characters in the range
 # 32-126, except that quotes and (literal) backslashes can only appear when escaped
 # by a backslash. When internationalized, UTF8 strings are also permitted except

email_validator/syntax.py

@@ -64,11 +64,11 @@ def validate_email_local_part(local: str, allow_smtputf8: bool = True, allow_emp
 
     # Check the local part against the non-internationalized regular expression.
     # Most email addresses match this regex so it's probably fastest to check this first.
-    # (RFC 2822 3.2.4)
+    # (RFC 5322 3.2.3)
     # All local parts matching the dot-atom rule are also valid as a quoted string
     # so if it was originally quoted (quoted_local_part is True) and this regex matches,
     # it's ok.
-    # (RFC 5321 4.1.2).
+    # (RFC 5321 4.1.2 / RFC 5322 3.2.4).
     m = DOT_ATOM_TEXT.match(local)
     if m:
         # It's valid. And since it's just the permitted ASCII characters,
@@ -95,7 +95,7 @@ def validate_email_local_part(local: str, allow_smtputf8: bool = True, allow_emp
         if not allow_smtputf8:
             # Check for invalid characters against the non-internationalized
             # permitted character set.
-            # (RFC 2822 Section 3.2.4 / RFC 5322 Section 3.2.3)
+            # (RFC 5322 3.2.3)
             bad_chars = set(
                 safe_character_display(c)
                 for c in local
@@ -184,7 +184,7 @@ def validate_email_local_part(local: str, allow_smtputf8: bool = True, allow_emp
     # don't apply in those cases.)
 
     # Check for invalid characters.
-    # (RFC 2822 Section 3.2.4 / RFC 5322 Section 3.2.3, plus RFC 6531 section 3.3)
+    # (RFC 5322 3.2.3, plus RFC 6531 3.3)
     bad_chars = set(
         safe_character_display(c)
         for c in local
@@ -194,7 +194,7 @@ def validate_email_local_part(local: str, allow_smtputf8: bool = True, allow_emp
         raise EmailSyntaxError("The email address contains invalid characters before the @-sign: " + ", ".join(sorted(bad_chars)) + ".")
 
     # Check for dot errors imposted by the dot-atom rule.
-    # (RFC 2822 3.2.4)
+    # (RFC 5322 3.2.3)
     check_dot_atom(local, 'An email address cannot start with a {}.', 'An email address cannot have a {} immediately before the @-sign.', is_hostname=False)
 
     # All of the reasons should already have been checked, but just in case
@@ -255,7 +255,7 @@ def check_unsafe_chars(s, allow_space=False):
 
 
 def check_dot_atom(label, start_descr, end_descr, is_hostname):
-    # RFC 2822 3.2.4
+    # RFC 5322 3.2.3
     if label.endswith("."):
         raise EmailSyntaxError(end_descr.format("period"))
     if label.startswith("."):
@@ -308,7 +308,7 @@ def validate_email_domain_name(domain, test_environment=False, globally_delivera
     # Check that before we do IDNA encoding because the IDNA library gives
     # unfriendly errors for these cases, but after UTS-46 normalization because
     # it can insert periods and hyphens (from fullwidth characters).
-    # (RFC 952, RFC 2822 3.2.4)
+    # (RFC 952, RFC 1123 2.1, RFC 5322 3.2.3)
     check_dot_atom(domain, 'An email address cannot have a {} immediately after the @-sign.', 'An email address cannot end with a {}.', is_hostname=True)
 
     # Check for RFC 5890's invalid R-LDH labels, which are labels that start

tests/test_syntax.py

@@ -306,6 +306,7 @@ def test_domain_literal():
         ('my\n@example.com', 'The email address contains invalid characters before the @-sign: U+000A.'),
         ('test@\n', 'The part after the @-sign contains invalid characters: U+000A.'),
         ('bad"quotes"@example.com', 'The email address contains invalid characters before the @-sign: \'"\'.'),
+        ('obsolete."quoted".atom@example.com', 'The email address contains invalid characters before the @-sign: \'"\'.'),
         ('11111111112222222222333333333344444444445555555555666666666677777@example.com', 'The email address is too long before the @-sign (1 character too many).'),
         ('111111111122222222223333333333444444444455555555556666666666777777@example.com', 'The email address is too long before the @-sign (2 characters too many).'),
         ('me@1111111111222222222233333333334444444444555555555.6666666666777777777788888888889999999999000000000.1111111111222222222233333333334444444444555555555.6666666666777777777788888888889999999999000000000.111111111122222222223333333333444444444455555555556.com', 'The email address is too long (4 characters too many).'),