Bytes paths now are supported in os.scandir() on Windows

[serhiy-storchaka]

BPO	27998
Nosy	@gvanrossum, @pfmoore, @vstinner, @tjguk, @ned-deily, @benhoyt, @elprans, @zware, @serhiy-storchaka, @1st1, @eryksun, @zooba
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Dependencies	bpo-27781: Change sys.getfilesystemencoding() on Windows to UTF-8
Files	cleanup_scandir_bytes.patch direntry_bytes_path.patch issue_27998_01.patch doc-scandir-bytes.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-11-20.06:26:46.122>
created_at = <Date 2016-09-07.08:21:10.630>
labels = ['type-bug', '3.7', 'OS-windows', 'docs']
title = 'Bytes paths now are supported in os.scandir() on Windows'
updated_at = <Date 2017-03-31.16:36:24.155>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:24.155>
actor = 'dstufft'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-11-20.06:26:46.122>
closer = 'serhiy.storchaka'
components = ['Documentation', 'Windows']
creation = <Date 2016-09-07.08:21:10.630>
creator = 'serhiy.storchaka'
dependencies = ['27781']
files = ['44810', '45009', '45017', '45469']
hgrepos = []
issue_num = 27998
keywords = ['patch']
message_count = 26.0
messages = ['274778', '274779', '274782', '274812', '274846', '274848', '274850', '274851', '277363', '277397', '278071', '278077', '278085', '278147', '278150', '278285', '278286', '278306', '278308', '278310', '278311', '278312', '280689', '281244', '281249', '281250']
nosy_count = 14.0
nosy_names = ['gvanrossum', 'paul.moore', 'vstinner', 'tim.golden', 'ned.deily', 'benhoyt', 'Elvis.Pranskevichus', 'docs@python', 'python-dev', 'zach.ware', 'serhiy.storchaka', 'yselivanov', 'eryksun', 'steve.dower']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27998'
versions = ['Python 3.6', 'Python 3.7']

[serhiy-storchaka]

Bytes paths are deprecated on Windows and their support was not added in os.scandir() for purpose. But this makes harder converting filesystem walking functions to use os.scandir(). We have to add a special case for bytes paths on Windows in every such function (os.walk(), glob.iglob()). This is cumbersome inefficient and errorprone code that slows down common case due to additional checks for special case. It would be better to add a support of bytes paths directly in os.scandir().

[pfmoore]

In the light of Steve Dower's work to "un-deprecate" bytes paths, I agree this should be added.

[eryksun]

This is implemented in Steve's latest patch in bpo-27781. For example:

    >>> sys.platform
    'win32'
    >>> os.mkdir('test')
    >>> f = open('test/\U00010000.txt', 'w')
    >>> next(os.scandir(b'test')).name
    b'\xf0\x90\x80\x80.txt'
    >>> next(os.scandir(b'test')).path
    b'test\\\xf0\x90\x80\x80.txt'

[zooba]

Heh, totally by accident as well.

[zooba]

Guido just said that we should make sure that os.scandir() continues to not support bytes, even after the patch for bpo-27781 goes in, so I'm reopening this.

[serhiy-storchaka]

But why? The code of os and glob modules could be simpler if add the support.