gh-106718: Respect PyConfig.stdlib_dir in getpath.

[yilei]

• Issue: PyConfig.stdlib_dir does not affect sys._stdlib_dir #106718

[yilei]

(I'm looking at the test_embed failure)

[yilei]

Tests passed!

[zooba]

stdlib_dir isn't set by this point unless it was in the config, so we can just check that it's empty and set it if so.

Later on there are more checks that may overwrite stdlib_dir without checking. Those also ought to check that it's empty.

stdlib_dir_was_set_in_config shouldn't be needed, I don't think. But it might be (if e.g. we need one of the later stdlib_dir assignments to overwrite an earlier one if it were set from a build directory, but that shouldn't be the case).

[yilei]

Thanks for the review!

Before making a change, a question:

I added stdlib_dir_was_set_in_config for this case:

cpython/Modules/getpath.py

Line 549 in a0773b8

stdlib_dir = None

as it was unconditionally setting stdlib_dir to None. At this point, stdlib_dir might be already set either in the config or some conditions above. So stdlib_dir_was_set_in_config is needed I think.

And given stdlib_dir_was_set_in_config is needed here, do you prefer using stdlib_dir_was_set_in_config everywhere, or check if stdlib_dir is empty in some cases?

[zooba]

You'll need to determine whether setting home should override other parts of the config. Historically, I believe that's what it has done, and so arguably it should continue to do it. Or maybe it should change? But that will need to be a deliberate (and documented) change of behaviour.

Any changes made to this script are incredibly fragile. Hopefully if this change goes in then we'll have all of the 3.13 prereleases to find out who it broke, but it would be best to figure out which change is least breaking, and be very explicit about which scenarios will change. Even then, we're basically guaranteed to miss some things.

[Yhg1s]

I think it makes sense for more specific settings to override more general ones. 'home' sets a bunch of things, one of which is stdlib_dir, so setting stdlib_dir explicitly should override home. (If the user doesn't want stdlib_dir to override home, they can just not set stdlib_dir if they're setting home, after all).

As for stdlib_dir_was_set_in_config, can't you just test if stdlib_dir is None instead?

[yilei]

Thanks, I reworded the changelog to say "When PyConfig.stdlib_dir is explicitly set, it's now respected and won't be overridden by PyConfig.home."

For stdlib_dir_was_set_in_config, I can test if stdlib_dir is None here. But below when home is set, it will override the stdlib_dir found by the build directory here. And I don't want to change this behavior so stdlib_dir_was_set_in_config is necessary there (similarly the two places below). Since there is stdlib_dir_was_set_in_config, I'm using it everywhere. Let me know if you prefer testing test if stdlib_dir is None here.

[Yhg1s]

I think it makes sense for more specific settings to override more general ones. 'home' sets a bunch of things, one of which is stdlib_dir, so setting stdlib_dir explicitly should override home. (If the user doesn't want stdlib_dir to override home, they can just not set stdlib_dir if they're setting home, after all).

As for stdlib_dir_was_set_in_config, can't you just test if stdlib_dir is None instead?

[yilei]

Any further comments? (The Windows x64 failure seems to be a flake / timeout.)

[zooba]

Done. We've got plenty of prereleases to find out what goes wrong from here (hopefully nothing, but I've hoped that for every single other getpath change and have been wrong every time 😄 )