pathlib.resolve(strict=False) only includes first child

[mshuffett]

BPO	30177
Nosy	@pitrou, @larryhastings, @ned-deily, @zware, @serhiy-storchaka, @eryksun, @zooba, @seirl, @zopieux, @richardcooper
PRs	bpo-30177: Fix misleading description of path.resolve() #1649 bpo-30177: pathlib: include the full path in resolve(strict=False) #1893 [3.6] bpo-30177: pathlib: include the full path in resolve(strict=False) #1985 [3.6] bpo-30177: add NEWS entry #2134 bpo-30177: add NEWS entry #2135

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-06-07.17:20:32.608>
created_at = <Date 2017-04-26.16:19:45.264>
labels = ['3.7', 'type-bug', 'library']
title = 'pathlib.resolve(strict=False) only includes first child'
updated_at = <Date 2017-06-12.16:40:16.197>
user = 'https://bugs.python.org/mshuffett'

bugs.python.org fields:

activity = <Date 2017-06-12.16:40:16.197>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2017-06-07.17:20:32.608>
closer = 'steve.dower'
components = ['Library (Lib)']
creation = <Date 2017-04-26.16:19:45.264>
creator = 'mshuffett'
dependencies = []
files = []
hgrepos = []
issue_num = 30177
keywords = []
message_count = 26.0
messages = ['292369', '292623', '294647', '294855', '294857', '294859', '294861', '294862', '294863', '294864', '294877', '294879', '294880', '294883', '294889', '294890', '294972', '295206', '295304', '295345', '295346', '295349', '295360', '295361', '295801', '295802']
nosy_count = 11.0
nosy_names = ['pitrou', 'larry', 'ned.deily', 'zach.ware', 'serhiy.storchaka', 'eryksun', 'steve.dower', 'antoine.pietri', 'mshuffett', 'zopieux', 'richardc']
pr_nums = ['1649', '1893', '1985', '2134', '2135']
priority = 'critical'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30177'
versions = ['Python 3.6', 'Python 3.7']

[mshuffett]

According to the documentation https://docs.python.org/3/library/pathlib.html#pathlib.Path.resolve
If strict is False, the path is resolved as far as possible and any remainder is appended without checking whether it exists.

The current behavior is not consistent with this, and only appends the first remainder.

For example:
If we have an empty '/tmp' directory
Path('/tmp/foo').resolve() and Path('/tmp/foo/bar').resolve() both result in Path('/tmp/foo') but Path('/tmp/foo/bar').resolve() should result in Path('/tmp/foo/bar')

[seirl]

I can reproduce this bug. This behavior is really confusing.

[richardcooper]

Pull Request (PR 1649) treats this as a documentation problem. I would argue that the documentation is correct and this is a bug in the code.

The strict flag was added as a result of bpo-19717. The decision on what to do when strict=False seems to come in https://mail.python.org/pipermail/python-ideas/2016-September/042203.html where Guido says:

"I would prefer it if Path.resolve() resolved symlinks until it hits
something that doesn't exist and then just keep the rest of the path
unchanged."

The documented behaviour also seems much more useful than the current behaviour.

[zooba]

This is definitely a bug if it really behaves as described.

PRs for fixing the functionality are welcome - I'm inclined to say fix it in 3.6 as well, but it might be too big a behavioural change if people are already working around it.

[seirl]

In Windows/Python 3.6, the behavior matches the documented one, so this could actually be an important silent bug when executing a code wrote for Windows in another OS. I'd argue to fix it in 3.6 too.

[seirl]

By the way, I'm pasting what I said on the PR here:

For what it's worth, this bug was the cause of an important downtime in our organization, because someone needed to normalize a path that was later passed to a .mkdir(parents=True), and it was, in some cases, silently doing the mkdir of a wrong path instead of creating all the parents.

[pitrou]

This is definitely a bug and I think the fixes should be backported.

[zooba]

Ah, didn't catch that it doesn't occur on Windows - that explains why I've never seen it before.

Yes, definitely fix and backport. Adding the RMs in case they want to delay any upcoming releases to get the fix.

[zopieux]

I agree bpo-30177 is not a suitable fix for this issue, as it fixes the doc instead of fixing the actual underlying bug in the function behavior.

This bug can lead to files being added to the wrong (parent) directory, which is quite critical.

[ned-deily]

We have less than 2 weeks until cutoff for 3.6.2 so it would be great if someone would take this on.

[seirl]

I'm on it.

[zooba]

The initial fix should be easy:

--- a/Lib/pathlib.py
+++ b/Lib/pathlib.py
@@ -329,8 +329,6 @@ class _PosixFlavour(_Flavour):
                     if e.errno != EINVAL:
                         if strict:
                             raise
-                        else:
-                            return newpath
                     # Not a symlink
                     path = newpath

However, the trick is going to be in the tests, which are shared between POSIX and Windows - and are apparently passing on Windows right now. I'm not entirely sure why that is, but it may not be as simple here as "works on POSIX".