fileinput.FileInput.readline() always returns str object at the end even if in 'rb' mode

[RyosukeIto]

BPO	25510
Nosy	@bitdancer, @serhiy-storchaka
Files	fileinput.py.diff: A patch for fileinput.py to fix this issue test_fileinput.py.diff: A patch for test_fileinput.py (parent: https://hg.python.org/cpython/rev/f7db966c9fee)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-11-01.14:50:32.362>
created_at = <Date 2015-10-29.14:12:19.720>
labels = ['type-bug', 'library']
title = "fileinput.FileInput.readline() always returns str object at the end even if in 'rb' mode"
updated_at = <Date 2015-11-01.21:02:14.130>
user = 'https://bugs.python.org/RyosukeIto'

bugs.python.org fields:

activity = <Date 2015-11-01.21:02:14.130>
actor = 'r.david.murray'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-11-01.14:50:32.362>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2015-10-29.14:12:19.720>
creator = 'Ryosuke Ito'
dependencies = []
files = ['40895', '40918']
hgrepos = []
issue_num = 25510
keywords = ['patch']
message_count = 8.0
messages = ['253675', '253679', '253680', '253844', '253848', '253849', '253850', '253875']
nosy_count = 4.0
nosy_names = ['r.david.murray', 'python-dev', 'serhiy.storchaka', 'Ryosuke Ito']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25510'
versions = ['Python 3.4', 'Python 3.5', 'Python 3.6']

[RyosukeIto]

In Python3, fileinput.FileInput.readline() always returns str object at the end,
even if in 'rb' mode.
Here's a test code.

import fileinput

fi = fileinput.input('test_fileinput.py', mode='rb')

while True:
    line = fi.readline()
    assert isinstance(line, bytes)
    if not len(line):
        break

It fails in Python3.2 to 3.5.

I wrote a patch for this.
With it, the test above passes.

[serhiy-storchaka]

Would be nice to add a test in Lib/test/test_fileinput.py.

[bitdancer]

We need to think carefully about backward compatibility here.

[RyosukeIto]

Add a test for fileinput.FileInput.readline() in 'rb' mode.

I'm sure this test fails on Python 3.5 and fixed in the patch(fileinput.py.diff).

[serhiy-storchaka]

LGTM.

I don't think we should care about breaking backward compatibility here. Most code tests end-of-files with "not line" or "len(line) == 0", and this work with fixed fileinput as well as with broken. That is why we have received this bug report now, after many years since releasing 3.0.

[python-dev]

New changeset e27c589e1c6a by Serhiy Storchaka in branch '3.4':
Issue bpo-25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/e27c589e1c6a

New changeset 321b34824020 by Serhiy Storchaka in branch '3.5':
Issue bpo-25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/321b34824020

New changeset 4f0e293e6eb0 by Serhiy Storchaka in branch 'default':
Issue bpo-25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/4f0e293e6eb0

[serhiy-storchaka]

Thank you Ryosuke for your contribution.

[bitdancer]

Ah, I misunderstood the bug report. Yes, this should be OK (I hope :).