Update PCBuild for VS 2015

[zooba]

BPO	22919
Nosy	@pitrou, @vstinner, @tjguk, @jkloth, @tpn, @zware, @zooba
Files	projects_highlights.diff projects_full.diff fixups.diff: tcl/tk/OpenSSL fixups cpython_93607_to_93616.diff python3.diff round4.diff round5complete.patch round6incremental.patch round7complete.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2015-01-17.23:11:30.791>
created_at = <Date 2014-11-22.21:40:07.139>
labels = ['type-feature', 'OS-windows', 'build']
title = 'Update PCBuild for VS 2015'
updated_at = <Date 2015-11-06.18:08:00.514>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2015-11-06.18:08:00.514>
actor = 'steve.dower'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2015-01-17.23:11:30.791>
closer = 'steve.dower'
components = ['Build', 'Windows']
creation = <Date 2014-11-22.21:40:07.139>
creator = 'steve.dower'
dependencies = []
files = ['37246', '37247', '37248', '37251', '37265', '37270', '37374', '37392', '37405']
hgrepos = []
issue_num = 22919
keywords = ['patch', 'needs review']
message_count = 31.0
messages = ['231531', '231533', '231541', '231547', '231593', '231604', '231631', '231634', '231635', '231878', '232237', '232239', '232248', '232249', '232344', '232410', '232570', '232571', '233637', '233641', '233643', '233644', '233648', '233652', '233671', '233945', '233946', '233969', '234202', '254205', '254206']
nosy_count = 11.0
nosy_names = ['pitrou', 'vstinner', 'tim.golden', 'jkloth', 'trent', 'BreamoreBoy', 'python-dev', 'zach.ware', 'mdengler', 'steve.dower', 'Kevin.Phillips']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22919'
versions = ['Python 3.5']

[zooba]

I've basically finished the core of the work to refresh the PCBuild project files and support building with VS 2015, and I believe it's ready to merge in.

Though the title says VS 2015, builds will work fine with VS 2010 and VS 2013 (and probably VS 2012, but I didn't try it). Even so, I've copied the old project files into PC/VS10.0, though I'd be happy to just forget them.

I suspect Tools/msi/msi.py will stop working with this change. My replacement installer is not ready yet. If anyone's particularly concerned about msi.py then I can try and restore it before checking this in, but I'm 99.9% certain it won't be used for 3.5, so I don't see the point.

I've attached two patches, one with all the changes and the other with the "highlights" - the diff between the diffs are most of the project files and the added/deleted files. I imagine practically everyone will have a better time viewing the changes in their own hg tools, so the changes can be pulled from my sandbox with "hg pull https://hg.python.org/sandbox/steve.dower -b Projects" or viewed at https://hg.python.org/sandbox/steve.dower/shortlog/d08b456124e5. (I'll rebase these as a single commit in default when it goes in.)

A few changes that may deserve some more discussion:

• Builds for 32-bit now go to PCBuild\win32 instead of PCBuild (the batch file updates should make this transparent)

• The project that used to trigger the OpenSSL build is now two projects (libeay, ssleay) that do the build directly, so they may need updating whenever we update the OpenSSL version or change _ssl/_hashlib. I'm obviously happy to track these. (The advantages are proper incremental builds, better debugging, better optimisation - definitely PGO if we turn that back on.)

• The posixmodule.c update for Py_Verify_fd may become redundant - the VC14 CRT will include a per-thread function to change the invalid parameter handling, so when that's available we should be able to switch away from this trickery completely.

• I changed some GetVersion calls (which no longer behave properly as of Windows 8.1) to use functions from VersionHelpers.h in VS 2013 and later.

• build.bat now builds in parallel by default, with '-M' to disable.

• PCBuild/readme.txt has been updated as if VS 2015 has already been released. This is not yet true, but I doubt anyone who notices will be particularly confused or upset.

[zooba]

Oh, one other thing I just thought of: the current release of nmake (in VS 2015) has a regression that will make TCL and TK fail to build, but I have a workaround for their makefile. OpenSSL also had a problem with VC14 but their fix isn't in the build we've got on svn.python.org.

If possible, I'd like to get the workarounds into our sources for those before this is checked in, so that the buildbots (currently there's one that I know of with VS 2015 Preview installed) can pick up those fixes and won't fail because of these.

The nmake issue should be fixed for VS 2015 RC and the OpenSSL issue is already fixed (in their 1.1 branch, IIRC). I've attached a patch listing the changes needed.

[zooba]

Fixed and pushed a minor issue with debug builds - I wasn't setting the 'g' suffix on the tcltk libs correctly.

[zooba]

Pushed some changes based on Zach's feedback and added a new complete patch (minus the deletion of the PC/VS9.0 and PC/VS10.0 folders).

[zooba]

Added some fixes to the python3.dll build (and xxlimited test project) which I noticed thanks to Antoine's feedback.

[zware]

Another round of review posted on Rietveld. I'll try to get the VS2015 preview downloaded today and give some more feedback after using it.

Just one inconsequential note on python3.diff: the 'lib' command in python3dll.vcxproj could use a '/nologo' flag.

[zware]

After a debug build with VS2015, my biggest concern is this:

..\PC\bdist_wininst\install.rc(19): fatal error RC1015: cannot open include file 'afxres.h'. [P:\ath\to\cpython\PCbuild\bdist_wininst.vcxproj]

There's also a slew of new warnings, mostly C4456 and C4457 with a few C4459s thrown in. I think we can safely disable those three (which are all about name shadowing), perhaps with a XXX comment alongside to the effect of 'it might be nice to not disable these warnings someday'.

Interestingly, there's also this warning:

C:\Program Files (x86)\Windows Kits\8.1\Include\um\shlobj.h(1054): warning C4091: 'typedef ': ignored on left of 'tagGPFIDL_FLAGS' when no variable is declared [P:\ath\to\cpython\PCbuild\pylauncher.vcxproj]

And a matching one for pywlauncher.vcxproj.

[zooba]

After a debug build with VS2015, my biggest concern is this:

..\PC\bdist_wininst\install.rc(19): fatal error RC1015: cannot open
include file 'afxres.h'. [P:\ath\to\cpython\PCbuild\bdist_wininst.vcxproj]

I thought I had a workaround for this, but I'll have to check it out. Oddly enough, all of my builds have been fine (I think I'm mostly doing release builds - does that make a difference?)

There's also a slew of new warnings, mostly C4456 and C4457 with a
few C4459s thrown in. I think we can safely disable those three
(which are all about name shadowing), perhaps with a XXX comment
alongside to the effect of 'it might be nice to not disable these
warnings someday'.

I'm torn about that. Ideally I'd like to suppress the existing ones and still have new ones appear, but that probably requires going through and fixing them. Maybe leaving the warnings there will encourage someone to do it? :)

Interestingly, there's also this warning:

C:\Program Files (x86)\Windows Kits\8.1\Include\um\shlobj.h(1054):
warning C4091: 'typedef ': ignored on left of 'tagGPFIDL_FLAGS' when
no variable is declared [P:\ath\to\cpython\PCbuild\pylauncher.vcxproj]

And a matching one for pywlauncher.vcxproj.

This is a known issue with the Windows SDK - they need to release a new version to handle the changes in the compiler. It doesn't affect the Python build, though it is currently breaking some of my installer work :( I'm confident it will be fixed soon - they're getting a lot of reports about it.

[zooba]

Added changes from Zach's last review.

Highlight: I deleted the make_versioninfo project and build step :)

[zooba]

> After a debug build with VS2015, my biggest concern is this:
>
> ..\PC\bdist_wininst\install.rc(19): fatal error RC1015: cannot open
> include file 'afxres.h'. [P:\ath\to\cpython\PCbuild\bdist_wininst.vcxproj]

I thought I had a workaround for this, but I'll have to check it out.
Oddly enough, all of my builds have been fine (I think I'm mostly
doing release builds - does that make a difference?)

I haven't been able to reproduce this. bdist_wininst isn't selected to build for debug in VS, but it builds fine for me anyway. Was your repo dirty? (It didn't make a difference for me, but I have had files occasionally not update properly when switching branches.)

[zooba]

Does anyone else want to provide any feedback? Or should I go ahead and get it in (after rebasing/retesting/etc.)? The feedback so far has been really helpful.