Python flag/envvar to not put current directory on sys.path (but don't ignore PYTHONPATH)

hroncok · 2020-05-20T14:14:30.314Z

Hello. There are currently two similar ways to invoke pytest: either via the pytest executable or python -m pytest. It has a documented difference: calling via python will also add the current directory to sys.path.

As I understand things, calling tools that are specifically tied to a Python version/installation/venv via python -m is The Right Thing To Do™. In a multi-Python environment with system installed pytest (etc.), the distributors don’t need to worry about creating /usr/bin/pytest-3.8, /usr/bin/pytest-3.9 etc. when the users can run python3.8 -m pytest. IIRC the pyvenv command was deprecated and removed for the same reasons. (This does not apply for tools where the Python version is implementation detail, such as pygmentize.)

In Fedora, we’d like to advise packagers to invoke pytest via python -m pytest. However, we don’t like the current working directory added to sys.path. We set our own $PYTHONPATH to <path_to_the_buildroot>/usr/lib(64)?)/pythonX.Y/site-packages so we actually test the content we will ship.

Would it be possible to have a Python flag that prevents adding the current working directory to sys.path? The -I flag does that, but at the same time, all PYTHON* environment variables are ignored, too. That prevent us from doing the two following things simultaneously:

set and use custom $PYTHONPATH
ignore the current working directory

In the language of precmdline_parse_cmdline I’d like to be able to have cmdline->isolated = 1 and cmdline->use_environment = 1.

vstinner · 2020-05-20T14:26:06.100Z

Another option is to no longer add the current directory to sys.path by default. It was discussed a few times.

In 2017, my attempt when Perl did exactly that: https://mail.python.org/pipermail/python-ideas/2017-June/045842.html
2009, python-dev: https://mail.python.org/pipermail/python-dev/2009-August/091360.html
2009, CVE-2008-5983 python: untrusted python modules search path: https://bugs.python.org/issue5753
My notes on sys.path: https://python-security.readthedocs.io/security.html#module-search-path-sys-path
2018, Avoid adding an empty directory to sys.path when running a module with -m: https://bugs.python.org/issue33053
2012: https://bugs.python.org/issue16202
etc.

eric.snow · 2020-05-22T16:41:02.831Z

Also see https://bugs.python.org/issue13475 (“Add ‘–mainpath’/’–nomainpath’ command line options to override sys.path[0] initialisation”).

hroncok · 2022-02-23T11:35:37.929Z

Another use case for this flag. Our programs are installed into /usr/bin. We want them to not add /usr/bin to sys.path so we would add this flag to our shebang in Fedora. Otherwise having randomly named files in /usr/bin can interfere with Python programs, as described e.g. in bad magic number in 'six' · Issue #359 · benjaminp/six · GitHub

vstinner · 2022-02-24T02:05:21.286Z

hroncok:

Another use case for this flag. Our programs are installed into /usr/bin . We want them to not add /usr/bin to sys.path so we would add this flag to our shebang in Fedora. Otherwise having randomly named files in /usr/bin can interfere with Python programs, as described e.g. in bad magic number in ‘six’ · Issue #359 · benjaminp/six · GitHub

Oh right. I wrote gh-57684: Add -P cmdline option and PYTHONSAFEPATH env var by vstinner · Pull Request #31542 · python/cpython · GitHub to add -P option to Python, so you can add -P to the shebang to avoid the issue. Somewhere, it was proposed that setuptools should use such option since it generates scripts from entry points.

CAM-Gerlach · 2022-02-24T04:12:50.626Z

Thanks, Victor! This is a very useful change and one I and others have been looking forward to for quite a while. I’m happy to help on the docs, if you’d like a hand proofreading or contributing remaining changes.