RFC 6531 (SMTPUTF8) support in smtplib

[zvyn]

BPO	22027
Nosy	@warsaw, @pitrou, @bitdancer, @soltysh, @zvyn
Files	smtplib_smtputf8_issue21725-dependent.patch smtplib_smtputf8_issue21725-dependentV2.patch: fixed warnings; adjusted Doc according to berkerpeksag's comment smtplib_smtputf8V3.patch smtplib_smtputf8v4.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-05-16.18:05:28.417>
created_at = <Date 2014-07-21.21:29:00.766>
labels = ['type-feature', 'expert-email']
title = 'RFC 6531 (SMTPUTF8) support in smtplib'
updated_at = <Date 2015-05-16.18:05:28.415>
user = 'https://github.com/zvyn'

bugs.python.org fields:

activity = <Date 2015-05-16.18:05:28.415>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2015-05-16.18:05:28.417>
closer = 'r.david.murray'
components = ['email']
creation = <Date 2014-07-21.21:29:00.766>
creator = 'zvyn'
dependencies = []
files = ['36015', '36324', '36350', '39368']
hgrepos = []
issue_num = 22027
keywords = ['patch']
message_count = 5.0
messages = ['223610', '225207', '243147', '243344', '243346']
nosy_count = 7.0
nosy_names = ['barry', 'pitrou', 'r.david.murray', 'jesstess', 'python-dev', 'maciej.szulik', 'zvyn']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22027'
versions = ['Python 3.5']

[zvyn]

This patch is related to and depends on bpo-21725. I put it here for review, it can be applied (maybe with small changes) as soon as bpo-21725 is fixed.

[zvyn]

After blindly raising errors for 'SMTPUTF8' without 'BODY=.*' I found out that 'SMTPUTF8' alone is actually perfectly fine (for using international names in addresses but sending ASCII data). Which means I did a mistake in the smtpd patch :(
The attached 3rd version of the smtplib patch implements your suggestions, including re-factoring of the mail option parsing. One test is commented out until smtpd is fixed. I'll do that as -part of- bpo-21795 tomorrow.

[bitdancer]

Coming back to this, I noticed a number of things wrong with the logic that I'm not sure how I missed before. I had to rewrite pretty much all of the logic. The good news is the patch is now smaller :)

The new tests are failing, because they are wrong: the msg argument to sendmail/mail *must* be a byte string if smtputf8 is in effect, since an email message may contain multiple encodings as well as other binary data. (Well, the msg argument could be an ascii string, but what would be the point?)

I'm uploading this without fixing the tests because I'm out of time for now and may not get back to it before the weekend. Perhaps someone else will have time to work on the tests before then.

[python-dev]

New changeset 6b0e4c87bf9e by R David Murray in branch 'default':
bpo-22027: Add RFC6531 support to smtplib.
https://hg.python.org/cpython/rev/6b0e4c87bf9e

[bitdancer]

Rewrote the tests to keep all the existing tests working against an unmodified SimSMTPServer, so that we know backward compatibility is maintained. And as I said, I changed the tests to reflect that a non-ascii body must be provided as binary data, which means I deleted two tests.

(Note: in thinking about this, I vaguely remember some conversations where we thought it might make sense that if someone provided a message string that when encoded to utf-8 then conforms to the RFC, it would be OK to auto-encode it as utf-8, but I'm just not comfortable with that idea. It provides insufficient value compared to the danger of blindly encoding something that is not RFC conformant...if someone does build a such a unicode message, they should have to convert it themselves before passing it to smtplib, as a sign they know what they are doing :)

Thanks for your work Milan.