Incorporate Tcl/Tk/Tix into the Windows build process

[jkloth]

BPO	15968
Nosy	@loewis, @terryjreedy, @vstinner, @tjguk, @jkloth, @zware, @serhiy-storchaka
Files	029d1cdf6422.diff issue15968_update.diff: Updated patch issue15968.v2.diff issue15968_tix.svndiff: Patch to tix-8.4.3.x issue15968.v3.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zware'
closed_at = <Date 2014-07-16.20:33:10.982>
created_at = <Date 2012-09-18.19:36:23.100>
labels = ['type-feature', 'OS-windows', 'build']
title = 'Incorporate Tcl/Tk/Tix into the Windows build process'
updated_at = <Date 2014-07-16.20:33:10.980>
user = 'https://github.com/jkloth'

bugs.python.org fields:

activity = <Date 2014-07-16.20:33:10.980>
actor = 'zach.ware'
assignee = 'zach.ware'
closed = True
closed_date = <Date 2014-07-16.20:33:10.982>
closer = 'zach.ware'
components = ['Build', 'Windows']
creation = <Date 2012-09-18.19:36:23.100>
creator = 'jkloth'
dependencies = []
files = ['27660', '30588', '32997', '32998', '34377']
hgrepos = ['149']
issue_num = 15968
keywords = ['patch']
message_count = 20.0
messages = ['170668', '191153', '192907', '205326', '213275', '213281', '213283', '213284', '213325', '213403', '214140', '214436', '214437', '214445', '214785', '215081', '215086', '215715', '222964', '223268']
nosy_count = 10.0
nosy_names = ['loewis', 'terry.reedy', 'vstinner', 'tim.golden', 'jkloth', 'jeremy.kloth', 'BreamoreBoy', 'python-dev', 'zach.ware', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue15968'
versions = ['Python 3.5']

[jkloth]

This patch incorporates Tcl/Tk/Tix into the MSVC build in the same fashion as OpenSSL has been done.

Highlights:

• A new project, tcltk, is added that simply calls the Python script build_tkinter.py to build the externals.
• New helper module PCbuild/buildlib.py to parse and evaluate the MSBuild property files.
• build_tkinter.py is updated to use MSBuild properties for locating the sources.
• The tcltk output directory is now inside the PCbuild directory to enable different Tcl/Tk/Tix versions for different Python versions

[zware]

The current patch doesn't apply cleanly anymore and contained an accidental reversion of a commit, so I took the liberty of creating an updated version. I'll also be leaving comments on Rietveld.

[zware]

I'm not sure that moving the tcltk output dir into PCbuild is the right way to go about making different versions of Tcl/Tk available to different Python versions. For one, because it's not true if you're working out of a single checkout for multiple versions. I think I'd rather see 'tcltk'/'tcltk64' stay where they are, but grow extensions to denote different builds--'tcltk-Win32-py34' and 'tcltk-x64-py35', for example. Also, PCbuild/rt.bat needs to be updated if tcltk[64] is changed.

I can't comment much on buildlib.py as I haven't had the time to learn everything I'd need to to understand it all, but it seems to work well doing what it is meant to do. However, it does seem like substantial overkill for what the comment at the top says it is for :P

Barring the issues I've raised so far, I really like the general idea and direction of the patch. I really hope this can make it to inclusion before too long.

[zware]

Here's a new patch based on Jeremy's that addresses all of my review comments and includes the update to Tcl/Tk 8.6.1.

Also, I'll be attaching a patch to tix-8.4.3.x which allows it to be built in debug configuration and removes many warnings about unrecognized command line options.

[zware]

I started looking at this again recently, and discovered that building from within Visual Studio doesn't work with the proposed patches.

So, here's a different approach that adds 'Makefile' projects for each of Tcl, Tk, and Tix instead of using build_tkinter.py. The "NMakeBuildCommandLine" for each new project is actually a short, fairly straightforward script that first checks for the necessary components in the install location ($(tcltkDir)) and bails out with a success message if they're already there, or builds if they're not. The Tix project still requires http://bugs.python.org/file32998/issue15968_tix.svndiff to function properly on Debug builds (which means this patch isn't quite right, the proper Tix version number will be 8.4.3.4).

There are a couple of changes that are made trivial by the main change here, namely copying the Tcl and Tk DLLs into the output dir where _tkinter can find them, which means PCbuild/rt.bat doesn't need to mess with PATH (nor do individuals need to change PATH when testing Tkinter manually). To simplify the new projects, I also simplified $(tcltkDir)/$(tcltk64Dir) and $(tcltk[64]Lib)/$(tcltk[64]LibDebug): there is now only $(tcltkDir) and $(tcltkLib). x64.props redefines $(tcltkDir) to be $(externalsDir)\tcltk64, and $(tcltkLib) is defined in terms of $(tcltkDir) and $(TclDebugExt).

I've not tested every possible configuration, but I can confirm that Debug and Release configurations work as expected on both Win32 and x64 platforms, whether built by command line (msbuild) or Visual Studio. There is an issue with PGInstrument/PGUpdate builds on Win32 (and probably x64, untested) where $(OutDir) is not set properly for the Tcl and Tk projects, and thus their DLLs are copied into $(SolutionDir) instead of the expected output dir. There are other (unrelated to tkinter) issues with building PGI and PGO, though, so I don't consider that problem a blocker for this issue.

One thing I'm not certain about, should this go into maintenance branches (namely 3.4, possibly 3.3), or just default (after 3.4 is branched)?

[loewis]

I wouldn't change the maintenance branches. I expect that the buildbots will break when the patch is applied, since they will fail to fetch the tix sources (unless I'm missing something).

Nevertheless, I like the approach, hoping that you'll be available to fix urgent issues that arise once it is applied.

[tjguk]

I haven't looked at the patch, but +1 to anything which brings
Tcl/Tk/Tix support into a state of default usability. Thanks for picking
this up, Zachary.

[zware]

Martin: I'll wait until after 3.4 is branched to commit to default, and make sure I have some time to pick up the pieces when I do :). The buildbots should be fine, though; external-common.bat is patched to pull in the tix sources as well.

issue15968_tix.svndiff will need to be applied to http://svn.python.org/projects/external/tix-8.4.3.x and be tagged as tix-8.4.3.4 before changes are made to the cpython repo, though, but I don't have access to svn.python.org (nor enough experience with svn to be comfortable making the change). Could you possibly do that for me?

[jeremykloth]

I'm curious as to the issues that arose in getting 'build_tkinter.py'
to work within the Visual Studio IDE, as that is what I used to
develop the patches to start with. I would like to look over the new
changes, but will not have any time to do so until 3-15-2014 at the
earliest. Hopefully that is still enough time.

[zware]

The problem with the previous approach and building inside Visual Studio was that makefile.vc in both Tcl and Tk first checks for one of "MSDEVDIR", "MSVCDIR", "VCINSTALLDIR", "MSSDK" or "WINDOWSSDKDIR" being set, which Visual Studio doesn't set. That issue could have been fixed basically the same way I fixed it in the new approach (set "MSDEVDIR" to a dummy value, probably in build_tkinter.py), but the new approach strikes me as being simpler and cleaner. Not to mention that without the dependency python.vcxproj, tcl/tk/tix can be built in parallel with other projects (like pythoncore, the longest-running Python project) using the msbuild /m switch.

I welcome having more eyes on it :). I definitely won't be committing before 3.4.0 final (scheduled for the 16th), and it may be a week or better after that before I have good opportunity.

[zware]

If there are no objections before then, I'll split out the Tix parts of this patch and commit the rest this weekend. I'll open a new issue for adding Tix to the mix.

[python-dev]

New changeset c2e2dc6c8769 by Zachary Ware in branch 'default':
Issue bpo-15968: Incorporated Tcl, Tk, and Tix builds into the Windows build
http://hg.python.org/cpython/rev/c2e2dc6c8769