Bring Doc/make.bat as close to Doc/Makefile as possible

[zware]

BPO	17386
Nosy	@terryjreedy, @tiran, @tjguk, @ezio-melotti, @briancurtin, @zware
Files	win_doc_make.diff: Doc/make.bat overhaul win_doc_make_minipatch.diff: Fix to %PYTHON% assignment win_doc_make.v2.diff: Version 2 win_doc_make.v3.diff: Version 3 win_doc_make.v4.diff: Version 4 win_doc_make.v5.diff: Version 5 issue17386.diff: New approach issue17386.v2.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/zware'
closed_at = <Date 2014-04-29.14:47:55.734>
created_at = <Date 2013-03-08.22:33:14.274>
labels = ['type-feature', 'OS-windows', 'build', 'docs']
title = 'Bring Doc/make.bat as close to Doc/Makefile as possible'
updated_at = <Date 2014-04-29.14:47:55.733>
user = 'https://github.com/zware'

bugs.python.org fields:

activity = <Date 2014-04-29.14:47:55.733>
actor = 'python-dev'
assignee = 'zach.ware'
closed = True
closed_date = <Date 2014-04-29.14:47:55.734>
closer = 'python-dev'
components = ['Build', 'Documentation', 'Windows']
creation = <Date 2013-03-08.22:33:14.274>
creator = 'zach.ware'
dependencies = []
files = ['29353', '29358', '29418', '29792', '29876', '29993', '35010', '35079']
hgrepos = []
issue_num = 17386
keywords = ['patch']
message_count = 16.0
messages = ['183768', '183844', '183848', '184116', '184126', '184127', '184237', '186036', '186677', '186768', '186785', '187033', '187666', '217076', '217419', '217520']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'christian.heimes', 'tim.golden', 'ezio.melotti', 'brian.curtin', 'docs@python', 'python-dev', 'zach.ware']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue17386'
versions = ['Python 3.4', 'Python 3.5']

[zware]

I have found Doc/make.bat to be very useful, but I have hit a few of its limitations now and then. Thus, I have made extensive modifications to bring it much closer to the way Doc/Makefile works. Among the things changed in the attached patch:

• Use pushd/popd to make sure everything is done in the Doc directory.

• Allow variables to be set on the command line, e.g.:
  make html "PYTHON2=python"

• Set PYTHON2 and PYTHON3 variables separately; the Sphinx version and associated tools we're currently using want Python 2, which defaults to "py -2". Other tools, like Tools/scripts/serve.py (the make serve target), expect Python 3; setting the default PYTHON3 searches for python(_d).exe in PCbuild(\amd64), and falls back to "py -3" if not found.

• Set the HTML Help Workshop default path to %ProgramFiles(x86)%\..., if that variable is defined. (If I understand correctly, that variable is only defined on x64 installations, and in cases where it is not defined on x64, %ProgramFiles% silently copies %ProgramFiles(x86)%.)

• Convert target choice to a single for loop, rather than several lines of if. Anything not in the list falls through to the help message.

• Expand the help message to match the message given by Doc/Makefile

• Add the "clean" target, which does the same as Makefile

• Reimplement "update" to match Makefile (call clean, call checkout. I do wonder if maybe it should be done the other way around, though; reimplement Makfile's "update" target to simply update the svn checkouts, rather than re-pull them)

• Add separate labels for each target. The old method of sending everything to :build was very clever and kept the file short, but limited flexibility. It's also nice to get a message at the end (matching Makefile), which is much easier to do with separate labels.

• Add support for the PAPER environment variable in LaTeX building.

• Add coverage, doctest, pydoc-topics, dist, check, and serve targets. "dist" cannot be completed in the same way with default tools on Windows, so all it does is build html, text, latex (a4 and letter), and epub and copy them into the dist directory.

• checkout checks for the existence of the checked out packages before checking out

• build calls checkout before building, so you can go right to "make html" instead of having to do "make checkout && make html"

• Add support for SOURCES and SPHINXOPTS environment variables.

Everything seems to work for me, but I've only been able to test on Windows 7 (32 and 64 bit). I don't believe there's anything in there that doesn't exist in XP, though. Also, since this version of the file has a lot of labels, this patch does somewhat rely on resolution of bpo-17202. It should probably work without any changes to .hgeol, but that can't be guaranteed.

There are a couple of drawbacks to the patched version; it's a much longer file, and it does lose a good chunk of the elegance of the original solution. If anyone has any suggestions for regaining some of that elegance without losing any of the functionality, I am of course all ears :). In particular, I'm not a huge fan of having to have "pushd ." at the beginning of every label, but the only other solutions I came up with involved setting and unsetting a "DISALLOW_POPD" variable in several places, and a condition on the popd at :end, or forgetting the pushd/popd thing entirely.

I'm looking forward to hearing what anyone else thinks of this, positive or negative :)

[terryjreedy]

+1 I would worry more about performance than elegance.

While this is debated, can you provide a mini-patch for at least 3.3+ that corrects the bug of trying to run sphinx with python 3, so that anything but 'make checkout' fails? I would apply that immediately.

[zware]

Sure can. This one should do it, at least for anyone with 3.3 installed.

[terryjreedy]

2.7 make.bat, which currently downloads sphinx v0.6.7 but not at least one needed file, does not work. See bpo-17412. I asked there whether simply updating the sphinx version, which seems to work, is the right fix. I believe this is currently the only different in make.bat between Python versions.

[python-dev]

New changeset 9b45873e5a68 by Terry Jan Reedy in branch '3.2':
Issue bpo-17386 make.bat must run with Python 2 until Sphinx runs with Python 3.
http://hg.python.org/cpython/rev/9b45873e5a68

New changeset e45db319e590 by Terry Jan Reedy in branch '3.3':
Merge with 3.2: Issue bpo-17386
http://hg.python.org/cpython/rev/e45db319e590

New changeset 99e72eae35bb by Terry Jan Reedy in branch 'default':
Merge with 3.3: Issue bpo-17386
http://hg.python.org/cpython/rev/99e72eae35bb

[terryjreedy]

I pushed the minipatch but did not add a news item as it may get superseded and will not affect anyone currently setting PYTHON.

[zware]

Applying the minipatch makes the big patch fail to apply, so here's an update. Also changed in this version of the patch:

• README.txt is updated. Not all the changes directly relate to this patch, but there were some needed updates in there anyway. If desired, I can split those into a separate issue, but all the changes are fairly minor.

• Added *VERSION variables for each of the tools for easy overriding and easier-to-find, single place to update.

Terry, you're right about make.bat being very similar between versions. About the only other difference I saw was the removal of the line that prints the Python version in use, and the version of Pygments used. I'd be happy to make patches for backporting this as far as desired (assuming, of course, that it is accepted and is desirable to backport :-).

[zware]

Any comments? Christian, I'd especially like to hear from you since it looks like you were the original author of Doc/make.bat (bpo-1472).

[zware]

Here's a new patch to address Ezio's review comments, including more updates to Doc/README.txt, the addition of an 'htmlview' target to make.bat, and documentation of the 'htmlview' target in the Makefile usage message.

[ezio-melotti]

Terry, do you want to test and commit this?
Should it be backported to 3.3 (and possibly 2.7) too?