Add python.ini file for embedded/applocal installs

[zooba]

BPO	23955
Nosy	@pfmoore, @ronaldoussoren, @ncoghlan, @tjguk, @ned-deily, @merwok, @takluyver, @zware, @zooba
Files	23955_1.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2015-05-22.22:21:14.326>
created_at = <Date 2015-04-14.19:16:30.917>
labels = ['interpreter-core', 'type-feature', 'OS-windows']
title = 'Add python.ini file for embedded/applocal installs'
updated_at = <Date 2015-05-22.22:21:14.325>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2015-05-22.22:21:14.325>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2015-05-22.22:21:14.326>
closer = 'steve.dower'
components = ['Interpreter Core', 'Windows']
creation = <Date 2015-04-14.19:16:30.917>
creator = 'steve.dower'
dependencies = []
files = ['39276']
hgrepos = []
issue_num = 23955
keywords = ['patch']
message_count = 18.0
messages = ['240989', '241343', '241422', '241424', '241670', '242002', '242020', '242038', '242042', '242044', '242100', '242125', '242127', '242450', '242589', '242865', '243857', '243859']
nosy_count = 11.0
nosy_names = ['richard', 'paul.moore', 'ronaldoussoren', 'ncoghlan', 'tim.golden', 'ned.deily', 'eric.araujo', 'python-dev', 'takluyver', 'zach.ware', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23955'
versions = ['Python 3.5']

[zooba]

When Python starts running we (getpathp.c) looks in the registry for its library directory. To avoid this, you can set PYTHONPATH and PYTHONHOME environment variables, but this isn't always easy if you've copied Python into your app's directory.

We should look for a python35.ini alongside python35.dll when initializing and extract values from it. Possibly this could just be an [Environment] section with key-value pairs to set before loading.

This would ensure that app-local Python installations never look for a global install in the registry. (As an aside, for the 3.5 beta we'll be shipping an app-local "installer", which will make this situation easier to end up in, but it also has value for xcopy installs.)

[merwok]

Hm, could you reuse the venv config file with use-site-packages=true?

[zooba]

I could reuse the venv config file but it would need a new property to forcibly use the 'home' value regardless of whether '$home\Lib\os.py' exists (in case the stdlib is zipped) and to make it relative to the file's path rather than the user's current directory.

How about adding an 'applocal=True' property to pyvenv.cfg that basically implies -I, PYTHONHOME=os.dirname(pyvenv_cfg), and skips the registry lookup on Windows completely (which there's currently no way to achieve)?

[zooba]

Arguably we should be making 'home' in pyvenv.cfg relative to the pyvenv.cfg file's directory anyway... AFAIK it's always generated as an absolute path, but I see no good reason to prevent people from manually configuring a relative path here.

[takluyver]

Relative paths would be nice for Pynsist - I would prefer to create the config file at build time and then install it along with the other files. If it needed absolute paths, then the installer would have to write the config file after the user selects the installation directory. That's doable, but it's easier to write and test the build steps than the install steps.

[ncoghlan]

I'd actually like a python.ini file defined as a Python 3.5 feature in order to provide a better backporting story for the PEP-476 certificate changes in downstream long term support releases of Python 2.7.

So +1 in principle, but I'd like to see this written up as a PEP (the SSL configuration setting doesn't need to be in the PEP, but the optional existence of the config file and reading it at startup does).

[pfmoore]

As described. this seems to be a Windows-only feature (it's replacing a registry lookup with a config file alongside the Python DLL).

So I'm not sure I understand Nick's comment - Nick, are you suggesting this should be a cross-platform feature? If so, what's the equivalent of "alongside the Python DLL", and what would the ini file be used for on Unix, where there's no registry lookup to override?

[ncoghlan]

If we're adding a static startup configuration file, then yes, I believe that should be a cross-platform feature, not something Windows specific, as it's a major change to our configuration philosophy. (I think it's a *necessary* change, but it's also a significant one)

Even if it started out as Windows only, we should have a forward looking plan for how it relates to the startup sequence changes proposed in PEP-432.

[zooba]

I'm leaning towards clarifying/fixing the behavior of the venv config, since it's most of the way there for what I care about. Adding another option for applocal may be needed, but possibly not, in which case defining how relative paths are handled would be sufficient.

Any other startup file I'd want to define as setting environment variables only, to avoid having multiple ways to set properties. So for most cases a shell script would suffice (doesn't help people linking to libpython directly, but they can call SetPythonHome themselves).

[ncoghlan]

Treating "local Python" as a special-case of venv sounds like a good approach, then (especially as it currently seems plausible we'll end up going for environment variable based approach to the downstream PEP-476 backport)

[zooba]

Having looked at the implementation, the easiest way to do this will be to add an "applocal = true" option to pyvenv.cfg, which would behave identically to setting %PYTHONHOME% to the directory containing the config before launch.

I wanted to support relative paths for "home = " but the path handling is so far from sufficient that it wouldn't be secure. As it is, I've fixed a lot of potential buffer overruns in getpathp.c already, but I don't want to implement true relative paths, so I'd prefer to require "home = " to be absolute and add a second property that effectively means "home = .".

Any thoughts/comments/preferences?

[takluyver]

Would that option be the only thing that needs to be set to make Python app-local? I'm not familiar with what lives in pyvenv.cfg.