samefile and sameopenfile fail for WebDAV mapped drives

[eryksun]

BPO	30480
Nosy	@arigo, @pfmoore, @giampaolo, @tjguk, @zware, @eryksun, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-02-19.20:57:59.484>
created_at = <Date 2017-05-26.08:50:27.348>
labels = ['3.8', 'type-bug', '3.7', '3.9', 'OS-windows']
title = 'samefile and sameopenfile fail for WebDAV mapped drives'
updated_at = <Date 2021-02-19.20:57:59.483>
user = 'https://github.com/eryksun'

bugs.python.org fields:

activity = <Date 2021-02-19.20:57:59.483>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-02-19.20:57:59.484>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2017-05-26.08:50:27.348>
creator = 'eryksun'
dependencies = []
files = []
hgrepos = []
issue_num = 30480
keywords = []
message_count = 4.0
messages = ['294538', '295730', '383405', '387354']
nosy_count = 8.0
nosy_names = ['arigo', 'paul.moore', 'giampaolo.rodola', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'steven.geerts']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30480'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9']

[eryksun]

os.path.samefile (used by shutil copyfile and move), os.path.sameopenfile, and pathlib.samefile rely on os.path.samestat (from genericpath.py), which compares corresponding st_dev and st_ino values. POSIX assures that the tuple (st_dev, st_ino) is unique. Windows makes no such assurance for the combination of the volume serial number (st_dev) and file index number (st_ino). The volume serial number isn't required to be unique or even non-zero ([MS-FSCC] 2.5.9), and the file index number may also be zero if the file system doesn't support it ([MS-FSCC] 2.4.20).

[MS-FSCC]: File System Control Codes
https://msdn.microsoft.com/en-us/library/cc231987

A WebDAV network drive exemplifies both cases. The volume serial number is zero, as is the file index number for every file and directory. Thus samestat() is true for every comparison on the drive, whether for a file or directory, and also when comparing files mounted on different WebDAV drives.

This isn't a common problem, but I think it's severe enough to warrant a separate Windows implementation of samefile and sameopenfile. If the corresponding st_dev and st_ino values are equal and either is zero, it should compare the final paths via _getfinalpathname. The latter would need to be modified to support passing a file descriptor for use with sameopenfile.

[arigo]

Another example of this misbehaviour: there are cases where os.stat() will internally fail to obtain the whole stat info (in some case related to permissions) and silently fall back to the same behaviour as Python 2.7. In particular, it will return a result with st_dev == st_ino == 0. Of course, os.path.samefile() will then consider all such files as "the same one", which is nonsense.

[stevengeerts]

still an issue with Python 3.8 and 3.9.
A copy between 2 files on webdave mapped drives can't be done, although src and dst are different.

[zooba]

Same as bpo-33935, but since the newer issue has proposed solutions on it already, I'm preferring to keep that one open.