os.listdir(bytes) gives a list of bytes, but os.listdir(buffer) gives a list of unicodes

[arigo]

BPO	30879
Nosy	@arigo, @alex, @serhiy-storchaka
PRs	bpo-30879: os.listdir() and os.scandir() now emit bytes names when #2634 [3.6] bpo-30879: os.listdir() and os.scandir() now emit bytes names when (GH-2634) #2656 [3.5] bpo-30879: os.listdir() and os.scandir() now emit bytes names when (GH-2634) #2657

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-07-11.04:44:12.866>
created_at = <Date 2017-07-08.16:27:45.508>
labels = ['3.7', 'type-bug', 'library']
title = 'os.listdir(bytes) gives a list of bytes, but os.listdir(buffer) gives a list of unicodes'
updated_at = <Date 2017-07-11.04:44:12.866>
user = 'https://github.com/arigo'

bugs.python.org fields:

activity = <Date 2017-07-11.04:44:12.866>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-07-11.04:44:12.866>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2017-07-08.16:27:45.508>
creator = 'arigo'
dependencies = []
files = []
hgrepos = []
issue_num = 30879
keywords = []
message_count = 7.0
messages = ['297960', '297962', '297963', '297984', '298111', '298116', '298118']
nosy_count = 3.0
nosy_names = ['arigo', 'alex', 'serhiy.storchaka']
pr_nums = ['2634', '2656', '2657']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30879'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[arigo]

The os functions generally accept any buffer-supporting object as file names, and interpret it as if bytes() had been called on it. However, os.listdir(x) uses the type of x to know if it should return a list of bytes or a list of unicodes---and the condition seems to be isinstance(x, bytes). So we get this kind of inconsistent behaviour:

>>> os.listdir(b".")
[b'python', b'Include', b'python-config.py', ...]

>>> os.listdir(bytearray(b"."))
['python', 'Include', 'python-config.py', ...]

[serhiy-storchaka]

Good catch Armin! PR 2634 fixes this inconsistency.

Maybe it is worth to deprecate support of other bytes-like object except bytes. open(), os.fspath() and os.path functions don't support them.

[arigo]

I've also been pointed to https://bugs.python.org/issue26800 .

[serhiy-storchaka]

Oh, I forgot that this feature already is deprecated!

[serhiy-storchaka]

New changeset 1180e5a by Serhiy Storchaka in branch 'master':
bpo-30879: os.listdir() and os.scandir() now emit bytes names when (bpo-2634)
1180e5a

[serhiy-storchaka]

New changeset ecfe4f6 by Serhiy Storchaka in branch '3.6':
[3.6] bpo-30879: os.listdir() and os.scandir() now emit bytes names when (GH-2634) (bpo-2656)
ecfe4f6

[serhiy-storchaka]

New changeset 7527c32 by Serhiy Storchaka in branch '3.5':
[3.5] bpo-30879: os.listdir() and os.scandir() now emit bytes names when (GH-2634) (bpo-2657)
7527c32