email.policy.SMTP.fold() mangles long headers

[chrisarraicom]

BPO	30532
Nosy	@warsaw, @bitdancer, @jhillacre
PRs	bpo-30532: email.policy.SMTP.fold() mangles long headers #1892 [3.6] bpo-30532: Fix whitespace folding in certain cases #2591 [3.5] bpo-30532: Fix whitespace folding in certain cases #2592
Files	test_long_mime_headers.py: Test case for folding long email headers.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-07-06.21:30:10.140>
created_at = <Date 2017-05-31.22:08:53.091>
labels = ['type-bug']
title = 'email.policy.SMTP.fold() mangles long headers'
updated_at = <Date 2017-07-06.21:30:10.139>
user = 'https://bugs.python.org/chrisarraicom'

bugs.python.org fields:

activity = <Date 2017-07-06.21:30:10.139>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2017-07-06.21:30:10.140>
closer = 'r.david.murray'
components = []
creation = <Date 2017-05-31.22:08:53.091>
creator = 'chris@arrai.com'
dependencies = []
files = ['46917']
hgrepos = []
issue_num = 30532
keywords = []
message_count = 10.0
messages = ['294875', '294878', '295209', '295210', '296584', '296732', '296948', '297843', '297844', '297845']
nosy_count = 4.0
nosy_names = ['barry', 'r.david.murray', 'jhillacre', 'chris@arrai.com']
pr_nums = ['1892', '2591', '2592']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30532'
versions = ['Python 3.5']

[chrisarraicom]

When email.policy.SMTP.fold() attempts to wrap a line in which a chunk that does not contain a space is longer than the maximum line length, two things go wrong:

• The second line does not begin with a space, creating an invalid header.
• The space before the long chunk is stripped, causing the line break to occur in an invalid place.

In the attached test case, email.policy.SMTP.fold() is called on a line of the style:
Content-Disposition: attachment; filename="<filename>"

<filename> does not contain any spaces and increases in length with each pass. Everything works correctly when 'filename="<filename>"' fits onto one line. However, once it is longer than the maximum line length, the following happens:

• the space after the semicolon is dropped
• the line break often splits up the 'filename' keyword
• the second line loses the initial whitespace

[jhillacre]

I added a pull request that causes Chris's test to pass. I can confirm that the Chris's was not pass for me on master branch.

[jhillacre]

Rebased the github PR on latest master to get the windows build passing CI.

Is there anything I need to do to keep the PR moving?

[bitdancer]

Ping the issue again next week if I don't get to it this weekend.

[jhillacre]

Ping the issue again next week if I don't get to it this weekend.

I am a week late, but here is a ping.

[bitdancer]

Just as well. I had no time last weekend. I should have time this Sunday, though.

[bitdancer]

New changeset b350c22 by R. David Murray (Joel Hillacre) in branch 'master':
bpo-30532: Fix whitespace folding in certain cases
b350c22

[bitdancer]

New changeset c60d2f5 by R. David Murray (Joel Hillacre) in branch '3.6':
bpo-30532: Fix whitespace folding in certain cases (bpo-2591)
c60d2f5

[bitdancer]

New changeset 3bbdf99 by R. David Murray (Joel Hillacre) in branch '3.5':
bpo-30532: Fix whitespace folding in certain cases (bpo-2592)
3bbdf99

[bitdancer]

Thanks, Joel!