2to3 should rewrite execfile() to open in 'rb' mode

[warsaw]

BPO	16965
Nosy	@warsaw, @benjaminp, @serhiy-storchaka, @ZackerySpytz, @miss-islington
PRs	bpo-16965: 2to3 now rewrites execfile() to open with 'rb' #8569 [3.7] bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569) #9838 [3.6] bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569) #9839 [2.7] bpo-16965: 2to3 now rewrites execfile() to open with rb. (GH-8569) #9890

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-10-16.05:47:18.516>
created_at = <Date 2013-01-14.18:24:06.537>
labels = ['3.8', 'type-bug', 'expert-2to3']
title = "2to3 should rewrite execfile() to open in 'rb' mode"
updated_at = <Date 2018-10-16.05:47:18.515>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2018-10-16.05:47:18.515>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2018-10-16.05:47:18.516>
closer = 'serhiy.storchaka'
components = ['2to3 (2.x to 3.x conversion tool)']
creation = <Date 2013-01-14.18:24:06.537>
creator = 'barry'
dependencies = []
files = []
hgrepos = []
issue_num = 16965
keywords = ['patch']
message_count = 7.0
messages = ['179964', '223226', '327567', '327645', '327646', '327647', '327809']
nosy_count = 6.0
nosy_names = ['barry', 'benjamin.peterson', 'tshepang', 'serhiy.storchaka', 'ZackerySpytz', 'miss-islington']
pr_nums = ['8569', '9838', '9839', '9890']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue16965'
versions = ['Python 3.8']

[warsaw]

In this distribute bug I describe a problem when pip installing mimeparse, which has a setup.py with a BOM. distribute uses execfile() which gets fixed in Python 3 to use a combination of compile(open()) as the fixer.

https://bitbucket.org/tarek/distribute/issue/349/dont-rely-on-2to3-to-properly-rewrite

I think the problem is that the fixer opens the file in text mode, which leaves the BOM as a bogus first character, causing the SyntaxError. Instead, if the fixer opened the file in binary mode, then the BOM would be handled correctly by compile().

[BreamoreBoy]

Can we have a response to this please.

[serhiy-storchaka]

exec(compile(open("fn", "rb").read(), "fn", 'exec')) will emit a resource warning because the open file is not closed explicitly.

But this is a different issue.

[serhiy-storchaka]

New changeset d4d6013 by Serhiy Storchaka (Zackery Spytz) in branch 'master':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
d4d6013

[miss-islington]

New changeset 2238139 by Miss Islington (bot) in branch '3.7':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
2238139

[miss-islington]

New changeset 950fa40 by Miss Islington (bot) in branch '3.6':
bpo-16965: 2to3 now rewrites execfile() to open with 'rb'. (GH-8569)
950fa40

[serhiy-storchaka]

New changeset f5e00f4 by Serhiy Storchaka (Zackery Spytz) in branch '2.7':
[2.7] bpo-16965: 2to3 now rewrites execfile() to open with rb. (GH-8569) (GH-9890)
f5e00f4