Release Windows Store app containing Python

[zooba]

BPO	34977
Nosy	@pfmoore, @vsajip, @vstinner, @tjguk, @jkloth, @ned-deily, @zware, @zooba, @ethanhs, @miss-islington, @tirkarthi
PRs	bpo-34977: Add Windows App Store package #10245 [3.7] bpo-34977: Add Windows App Store package (GH-10245) #10677 Revert "bpo-34977: Add Windows App Store package (GH-10245)" #11019 [3.7] Revert "[3.7] bpo-34977: Add Windows App Store package (GH-10245)" #11021 bpo-34977: Add Windows App Store package #11027 [3.7] bpo-34977: Add Windows App Store package (GH-11027) #11028 bpo-34977: Use venv redirector instead of original python.exe on Windows #11029 [3.7] bpo-34977: Use venv redirector instead of original python.exe on Windows (GH-11029) #11030 bpo-34977: Remove unused preprocessor definition #11092 [3.7] bpo-34977: Remove unused preprocessor definition (GH-11092) #11093 bpo-32907: pathlib: Fix test_resolve_common failure on Windows #5812

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2018-12-12.23:45:02.538>
created_at = <Date 2018-10-13.22:43:47.118>
labels = ['3.8', 'type-feature', '3.7', 'OS-windows']
title = 'Release Windows Store app containing Python'
updated_at = <Date 2019-10-18.00:21:36.712>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2019-10-18.00:21:36.712>
actor = 'izbyshev'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2018-12-12.23:45:02.538>
closer = 'steve.dower'
components = ['Windows']
creation = <Date 2018-10-13.22:43:47.118>
creator = 'steve.dower'
dependencies = []
files = []
hgrepos = []
issue_num = 34977
keywords = ['patch']
message_count = 42.0
messages = ['327679', '327691', '328146', '328147', '330166', '330359', '330452', '330643', '331265', '331266', '331267', '331300', '331301', '331304', '331310', '331311', '331316', '331321', '331322', '331328', '331350', '331352', '331356', '331357', '331358', '331363', '331368', '331377', '331387', '331517', '331518', '331519', '331520', '331521', '331530', '331534', '331553', '331560', '331566', '331571', '331572', '331573']
nosy_count = 11.0
nosy_names = ['paul.moore', 'vinay.sajip', 'vstinner', 'tim.golden', 'jkloth', 'ned.deily', 'zach.ware', 'steve.dower', 'ethan smith', 'miss-islington', 'xtreak']
pr_nums = ['10245', '10677', '11019', '11021', '11027', '11028', '11029', '11030', '11092', '11093', '5812']
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue34977'
versions = ['Python 3.7', 'Python 3.8']

[zooba]

The Windows Store can now install unrestricted apps, which means we can provide the Python interpreter in there.

Advantages:

• far more reliable installation
• can have specific executables on PATH
• automatic updates
• other apps can act as extensions (with permission), so things like Mu could also become store apps in the future

Disadvantages:

• only --user installs for pip will work
• harder to find/modify installed files

I think for a (very significant) subset of our users, this will be a much better experience than downloading our current installer. It also has the advantage of making "Python" appear in searches in the start menu, which will link directly to a one-click "Install" button.

The biggest issue is likely to be pip not installing with --user by default, as well as subsequent issues with apps that require users to navigate to their %AppData% directories manually (since these will be redirected to different locations).

But until we get a package available for testing, it will be hard to figure out what needs fixing. Given this is just distribution and not a new platform, I plan to enable it for Python 3.7 (probably 3.7.2, but it may be possible to bundle 3.7.1 as-is for testing).

[tirkarthi]

+1 for reliable installation. There were some cases (bpo-33220, bpo-30944) where the installer was detected as a Virus by some vendors which would be reduced since this is installed from the App store as a verified app. I hope this will also reduce the scenarios where users need to resort to bug tracker with log files when there are installation failures [1] providing a more smooth one click install.

There is a long standing open issue [2] to make pip install --user as a default which I hope gets resolved and there is an issue with using pip install --user inside a virtualenv which seems to block the earlier issue [3]

Thanks a lot for this!

[1] https://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=windows+msi+install&submit=search&status=-1%2C1%2C2%2C3

[2] pypa/pip#1668

[3] pypa/pip#5702

[zooba]

I'm 99% sure that with bpo-34725 I'll be able to make "pip --user" the default when using this install, as well as making the commands globally available (i.e. "python[3[.8]]", "pip[3[.8]]", "idle[3[.8]]" will work everywhere).

[zooba]

(And I'll deal with that 1% at the PyPA sprints next weekend. We are due for a new pip insertion anyway, so I guess that'll come for 3.7.2.)

[zooba]

For people watching the bug, this is ready for review.

Most of the change is adding a "layout" script (roughly the equivalent of "make install" for Windows, but with more flexibility to generate different structures, precompile stuff, etc.)

About the only CPython change is to venv. I added a new launcher variation that looks for pyvenv.cfg and runs the python.exe listed in that, so now we don't have to copy any DLLs into the venv at all (it's needed because the old one would try to LoadLibrary DLLs inside the store package, which is not allowed, but it also fixes my #1 complaint about venv which is that upgrading the base Python breaks all venvs).

There are also a few tweaks to the startup process, including extending the existing __PYVENV_LAUNCHER__ env variable from macOS to Windows as well, though it's handled in a different location. And a couple of minor tweaks to tests - issues discovered now that it's pretty easy to generate a "real" layout to run tests in.

I can break this up into a couple of PRs if that's preferred, though it doesn't really gain anything. If you're not interested in the new layout script, stop reading the PR after you finish launcher.c :)

[zooba]

If anyone would like to try this out early, here's some instructions for a build I just made.

Enable sideloading apps on your machine (see https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development - may already be done, sideload or developer mode should do)

In a regular (non-admin) PowerShell prompt in a temporary directory, run these commands:

iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34540/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix

(The first may take a while as it downloads approx. 80MB. I'm intending to leave the test suite out of the final package, which should bring it down closer to 40MB.)

Now play with it:
$> python.exe
$> python3.exe
$> python3.7.exe
$> pip.exe
$> pip3.exe
$> pip3.7.exe
$> idle.exe
$> idle3.exe
$> idle3.7.exe
(Or press Win+S and search for new entries)

To remove it:

Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)

Right now, the py.exe launcher can't find it, nor can any tools that rely on enumerating the registry - directly requesting the reg keys will return the right values though. VS Code seems to pick it up just fine though, I guess because it's on PATH.

[zooba]

For my own reference, some issues I found installing this package on a clean machine:

• requires Developer Mode (until I get the sccd file signed - just sideloading is not sufficient)
• requires vcruntime140.dll to be included in the package (workaround is to install the redistributable from https://visualstudio.microsoft.com/downloads/#title-39324)
• venv needs to treat .pdb files as binary

[zooba]

Updated testing instructions - this one should work with just sideloading permissions I think.

iwr "https://dev.azure.com/Python/8e426817-76c0-4b99-ba9e-a48a1e4bd5db/_apis/build/builds/34760/artifacts?artifactName=appx&api-version=5.0-preview.5&%24format=zip" -outfile python37.zip
Expand-Archive .\python37.zip .
Add-AppxPackage appx\python.msix

To remove:
Remove-AppxPackage (Get-AppxPackage *Python.3.7)
(delete the downloaded files)

[zooba]

New changeset 468a15a by Steve Dower in branch 'master':
bpo-34977: Add Windows App Store package (GH-10245)
468a15a

[zooba]

New changeset 2532091 by Steve Dower in branch '3.7':
[3.7] bpo-34977: Add Windows App Store package (GH-10245)
2532091