os.readlink produces wrong result on windows

[simonmackenzie]

BPO	42957
Nosy	@pfmoore, @tjguk, @ethanfurman, @zware, @eryksun, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-01-22.19:44:43.542>
created_at = <Date 2021-01-18.19:13:09.085>
labels = ['type-bug', '3.8', '3.9', '3.10', 'invalid', 'library', 'OS-windows']
title = 'os.readlink produces wrong result on windows'
updated_at = <Date 2021-04-27.20:06:34.457>
user = 'https://bugs.python.org/simonmackenzie'

bugs.python.org fields:

activity = <Date 2021-04-27.20:06:34.457>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-22.19:44:43.542>
closer = 'steve.dower'
components = ['Library (Lib)', 'Windows']
creation = <Date 2021-01-18.19:13:09.085>
creator = 'simon mackenzie'
dependencies = []
files = []
hgrepos = []
issue_num = 42957
keywords = []
message_count = 6.0
messages = ['385218', '385232', '385285', '385287', '385308', '385510']
nosy_count = 7.0
nosy_names = ['paul.moore', 'tim.golden', 'ethan.furman', 'zach.ware', 'eryksun', 'steve.dower', 'simon mackenzie']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue42957'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[simonmackenzie]

os.readlink gives wrong result on python 3.8 onwards for windows

os.readlink("c:/users/simon/v1")
'\\\\?\\d:\\v1'

Should read d:\\v1

[eryksun]

Symlinks and mountpoints (aka junctions) contain two forms of the target path. There's a path that's intended for display in the shell, and there's the actual substitute path to which the link resolves. os.readlink() was changed to return the substitute path because the display form is not mandated by filesystem protocols (it's sometimes missing, especially for junctions) and not reliable (e.g. the display path may be a long path or contain reserved names such that it's not valid without the \\?\ prefix). It was decided to keep the C implementation of os.readlink() simple. Whether to retain the \\?\ prefix was shifted to high-level functions that consume the result of os.readlink(), such as os.path.realpath().

There was a previous issue related to this, in that the shutil module copies symlinks via os.readlink() and os.symlink(), which thus copies only the substitute path now. The issue was closed as not a bug, but had it been resolved with new functionality, I would have preferred to do so with a low-level function to copy a reparse point, not by reverting the behavior of os.readlink(). I also see no reason against adding an option to readlink() to return the display path instead of the substitute path, or to just remove the prefix. But I'd vote against making it the default behavior.

[simonmackenzie]

For most people the expectation would be that it returns a path in the same
format as any other path. Furthermore it seems odd to change the default
behaviour after years when it worked as expected. I never heard of this
substitute path before and it does not work in some circumstances e.g.
docker does not recognise it.

Note also that os.path.realpath(os.path.readlink("v1")) still returns
\\\\?\\d:\\v1. There needs to be some way of getting to the path that
everyone actually uses.

On Mon, 18 Jan 2021 at 21:25, Eryk Sun <report@bugs.python.org> wrote:

Eryk Sun <eryksun@gmail.com> added the comment:

Symlinks and mountpoints (aka junctions) contain two forms of the target
path. There's a path that's intended for display in the shell, and there's
the actual substitute path to which the link resolves. os.readlink() was
changed to return the substitute path because the display form is not
mandated by filesystem protocols (it's sometimes missing, especially for
junctions) and not reliable (e.g. the display path may be a long path or
contain reserved names such that it's not valid without the \\?\ prefix).
It was decided to keep the C implementation of os.readlink() simple.
Whether to retain the \\?\ prefix was shifted to high-level functions that
consume the result of os.readlink(), such as os.path.realpath().

There was a previous issue related to this, in that the shutil module
copies symlinks via os.readlink() and os.symlink(), which thus copies only
the substitute path now. The issue was closed as not a bug, but had it been
resolved with new functionality, I would have preferred to do so with a
low-level function to copy a reparse point, not by reverting the behavior
of os.readlink(). I also see no reason against adding an option to
readlink() to return the display path instead of the substitute path, or to
just remove the prefix. But I'd vote against making it the default behavior.

----------
components: +Library (Lib)
nosy: +eryksun
versions: +Python 3.10

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue42957\>

---