subprocess._execute_child doesn't accept a single PathLike argument for args

[rowillia]

BPO	31961
Nosy	@gpshead, @ned-deily, @bitdancer, @njsmith, @serhiy-storchaka, @eryksun, @Phaqui, @rowillia, @csabella
PRs	bpo-31961: subprocess.run() now accepts path-like args #4329 Revert "bpo-31961: subprocess now accepts path-like args (GH-4329)" #5912 bpo-31961: Fix support of path-like executables in subprocess. #5914 [3.7] Revert "bpo-31961: subprocess now accepts path-like args" (GH-5912) #5931

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2019-08-02.22:39:24.649>
created_at = <Date 2017-11-06.17:27:25.843>
labels = ['3.8', 'type-feature', 'library']
title = "subprocess._execute_child doesn't accept a single PathLike argument for args"
updated_at = <Date 2019-08-02.22:39:24.649>
user = 'https://github.com/rowillia'

bugs.python.org fields:

activity = <Date 2019-08-02.22:39:24.649>
actor = 'steve.dower'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2019-08-02.22:39:24.649>
closer = 'steve.dower'
components = ['Library (Lib)']
creation = <Date 2017-11-06.17:27:25.843>
creator = 'Roy Williams'
dependencies = []
files = []
hgrepos = []
issue_num = 31961
keywords = ['patch']
message_count = 34.0
messages = ['305663', '305669', '305710', '305802', '311247', '311250', '311604', '311611', '311639', '311645', '311675', '311683', '311695', '311697', '311698', '311715', '311730', '311733', '311734', '311737', '311775', '311948', '312859', '312958', '312960', '312962', '312966', '313033', '313035', '313038', '330496', '331062', '342778', '343814']
nosy_count = 9.0
nosy_names = ['gregory.p.smith', 'ned.deily', 'r.david.murray', 'njs', 'serhiy.storchaka', 'eryksun', 'Phaqui', 'Roy Williams', 'cheryl.sabella']
pr_nums = ['4329', '5912', '5914', '5931']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31961'
versions = ['Python 3.8']

[rowillia]

Repro:

from pathlib import Path
import subprocess

subprocess.run([Path('/bin/ls')])  # Works Fine
subprocess.run(Path('/bin/ls'))  # Fails

The problem seems to originate from here:
https://github.com/python/cpython/blob/master/Lib/subprocess.py#L1237-L1240

This file auspiciously avoids importing pathlib, which I'm guessing is somewhat intentional? Would the correct fix be to check for the existence of a __fspath__ attribute as per https://www.python.org/dev/peps/pep-0519/ ?

[rowillia]

Ignore my comment re: pathlib, it looks like PathLike is defined in os and not pathlib.

[phaqui]

I was able to make a test that reproduces your code, and expectedly fails. Also implemented a fix for it. See a temporary diff here: https://pastebin.com/C9JWkg0i

However, there is also a specific MS Windows version of _execute_child() (a phrase only computer-folks can say out loud without feeling very...wrong...). It uses a different method to extract and parse arguments, and it should be corrected and tested under windows, before I submit a PR for this.

Also, my test messes up the formatting. Instead of saying "....ok", they both say "... total 0\nok". I have no idea why.

[phaqui]

While researching this, I discovered that on MS Windows

>> subprocess.run([pathlike_object, additional_arguments])

did not run like it did on Posix. My PR includes this problem and it's fix.

[gpshead]

New changeset dd42cb7 by Gregory P. Smith (Anders Lorentsen) in branch 'master':
bpo-31961: subprocess now accepts path-like args (GH-4329)
dd42cb7

[gpshead]

Thanks for the PR! I accepted it as this makes sense as a feature.

<meme> pathlib all the things! </meme>

I think the code be polished up a bit to not rely on catching TypeError but this gets the feature in before the 3.7 feature freeze (just). If further changes are needed or bugs are found, they can be addressed during the 3.7 betas. :)

[serhiy-storchaka]

This made tests failing on Windows. See bpo-32764.