email.headerregistry.Address blocks Unicode local part addr_spec accepted elsewhere

[dracos]

BPO	36893
Nosy	@warsaw, @bitdancer, @dracos

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-05-12.11:06:18.673>
labels = ['expert-email']
title = 'email.headerregistry.Address blocks Unicode local part addr_spec accepted elsewhere'
updated_at = <Date 2019-05-12.13:10:20.478>
user = 'https://github.com/dracos'

bugs.python.org fields:

activity = <Date 2019-05-12.13:10:20.478>
actor = 'r.david.murray'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['email']
creation = <Date 2019-05-12.11:06:18.673>
creator = 'dracos'
dependencies = []
files = []
hgrepos = []
issue_num = 36893
keywords = []
message_count = 2.0
messages = ['342254', '342256']
nosy_count = 3.0
nosy_names = ['barry', 'r.david.murray', 'dracos']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue36893'
versions = ['Python 3.6']

Linked PRs

• gh-81074: Allow non-ASCII addr_spec in email.headerregistry.Address #122477

[dracos]

The parser for passing an addr_spec to email.headerregistry.Address does not allow non-ASCII local parts, but the rest of the email package handles them fine, either straight (with explicit references to RFC6532 and SMTPUTF8), or encoding as expected. Apologies if I've misunderstood something.

>>> from email.message import EmailMessage
>>> msg = EmailMessage()
>>> msg['To'] = 'Matthéw <aé@example.com>'
>>> msg.as_string()
'To: =?utf-8?q?Matth=C3=A9w?= <=?utf-8?q?a=C3=A9?=@example.com>\n\n'
>>> msg['To'].addresses[0]
Address(display_name='Matthéw', username='aé', domain='example.com')
>>> msg['To'].addresses[0].addr_spec
'aé@example.com'
>>> email.headerregistry.Address(addr_spec=msg['To'].addresses[0].addr_spec)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/headerregistry.py", line 48, in __init__
    raise a_s.all_defects[0]
email.errors.NonASCIILocalPartDefect: local-part contains non-ASCII characters)
>>>

[bitdancer]

In order to legitimately have a non-ascii localpart, you *must* be using RFC6532 and RFC6531. In the email package you do this by using policy=SMTPUTF8, or setting utf8=True in your custom Policy. In smtplib you do this by specifying smtputf8 in the mail_options list to sendmail, or passing a message with a policy that has utf8=True to send_message.

I notice in answering this report that this is not really documented clearly. The information is there, but only if you already know how the RFCs work. Some variation of the text above should be added to the smtplib documentation, and an example of using SMTPUTF8 should be added to the email examples chapter.

However, you are correct, there are couple of bugs here.

The rendering done by as_string (and as_bytes) is the best that we can do without raising an error...but we should probably be raising an error if the rendering policy does not have utf8=True and we don't have an "original source line" from parsing a message (which is the case here), rather than using the incorrect RFC2047 encoding.

The second bug, the one you are reporting, is that we apparently missed the constructor of Address when we were adding RFC6532 support. If you look at the comment above that code, it is purposefully trying to raise an error if the addr_spec is invalid and it was provided by the *application* (as opposed to email.Parser). But with RFC6532 support, it should be valid to have a local part that has non-ascii in an Address, and the error, as I noted above, should be raised only at serialization time and when we don't have an original source string. So that raise should be modified to explicitly ignore the NonASCIILocalPartDefect. (Really, Address should take a policy argument. That's a bigger change, but it would be the "right way" to fix this.)

Raising the error on serialization could cause some breakage if existing programs are "getting away" with specifying non-ascii local parts but not doing it via addr_spec. It is breakage that should happen, I think, but we may want to only do it in a feature release.