Pre-build OpenSSL and Tcl/Tk for Windows

[zooba]

BPO	30916
Nosy	@gpshead, @pfmoore, @vstinner, @tiran, @tjguk, @alex, @zware, @zooba, @dstufft
PRs	[bpo-30916] Pre-build OpenSSL and Tcl/Tk for Windows #2688 bpo-9566: Fix some Windows x64 compiler warnings #2492

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zooba'
closed_at = <Date 2017-07-26.16:01:28.946>
created_at = <Date 2017-07-12.22:37:45.570>
labels = ['type-feature', '3.7', 'OS-windows', 'build']
title = 'Pre-build OpenSSL and Tcl/Tk for Windows'
updated_at = <Date 2017-07-26.22:17:58.996>
user = 'https://github.com/zooba'

bugs.python.org fields:

activity = <Date 2017-07-26.22:17:58.996>
actor = 'Segev Finer'
assignee = 'steve.dower'
closed = True
closed_date = <Date 2017-07-26.16:01:28.946>
closer = 'steve.dower'
components = ['Build', 'Windows']
creation = <Date 2017-07-12.22:37:45.570>
creator = 'steve.dower'
dependencies = []
files = []
hgrepos = []
issue_num = 30916
keywords = []
message_count = 7.0
messages = ['298249', '298269', '298378', '298467', '298468', '298470', '299250']
nosy_count = 10.0
nosy_names = ['gregory.p.smith', 'paul.moore', 'janssen', 'vstinner', 'christian.heimes', 'tim.golden', 'alex', 'zach.ware', 'steve.dower', 'dstufft']
pr_nums = ['2688', '2492']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue30916'
versions = ['Python 3.7']

[zooba]

To save time and improve reliability while building CPython, we should pre-build and sign OpenSSL and Tcl/Tk binaries and make it a binary dependency.

Our slightly-patched sources will still be available, and it will be relatively easy for people to download and rebuild them, but for most cases (particularly CI/buildbots) we will be able to reuse prebuilt binaries.

Since they need to be signed, this basically means it's my responsibility to update the GitHub repo whenever we update Tcl, Tk, Tix or OpenSSL versions. As is usual whenever I get a job, I promise to find a way to mostly automate it :)

[zooba]

@zach - FYI, this will make it unnecessary to run prepare_ssl.bat when checking in updated sources. Rather, this script now requires Perl to do the entire build, so we don't have to have any difference between our source bundle and the original OpenSSL one.

[zooba]

Zach opted out of reviewing right now, so I went ahead and pushed my builds to the GitHub repos so that AppVeyor can run.

It built and ran fine (which I'm not surprised about), but it was only 1-2 minutes faster on AppVeyor (which I _am_ surprised about).

Tcl/tk/tix take longer than that to build on my dev machine, as does OpenSSL, so I'm not sure how AppVeyor was getting them to build so quickly. But in any case, it is definitely faster now, and I suspect more for our devs than our CI. OpenSSL can also be more easily upgraded, and the entire library is now available via ctypes, so all up I think it's a win (+ssl and hashlib experts in case they disagree).

Anyone want to take a look at the change before I merge?

[zooba]

New changeset 68d663c by Steve Dower in branch 'master':
[bpo-30916] Pre-build OpenSSL and Tcl/Tk for Windows (bpo-2688)
68d663c

[vstinner]

Failure:
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/750

[zooba]

The buildbot failure blamed on this issue is actually due to bpo-30450 - it's the first time we've added a new external, and so the inability to download Python via Powershell on Windows 7 is the problem, not this.

[zooba]

That buildbot has recovered with other changes, so I'm closing this.