'make touch' does not work with git clones of the source repository

[vlee]

BPO	23404
Nosy	@brettcannon, @ncoghlan, @vstinner, @larryhastings, @benjaminp, @ned-deily, @ezio-melotti, @berkerpeksag, @vadmium, @zware, @koobs, @willingc, @MojoVampire, @yan12125
PRs	bpo-23404: make touch becomes make regen-all #1405 [3.6] bpo-23404: make touch becomes make regen-all (#1405) #1461 bpo-30273: Update sysconfig #1464 bpo-23404: make touch becomes make regen-all (#1405) (#1461) #1465 [2.7] bpo-23404: make touch becomes make regen-all #1466 bpo-23404: Document "make regen-all" in What's New in Python 3.6 #2127 bpo-23404: make regen-all What's New entry #2128 bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 #7620 [3.7] bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 (GH-7620) #7621 [3.6] bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 #7622 [3.5] bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 #7623 [2.7] bpo-23404: Update/sync What's New files for 3.6/3.5/2.7 #7624
Files	0001-Issue-23404-Do-not-use-Python-2.7-constructs-in-Pars.patch: Do not use Python 2.7 constructs in Parser/asdl.py. iss23404devguide.patch boot-flag.patch boot-flag.patch: Based on Martin's version, with a minor fix boot-flag.py3.5.patch: Needs _freeze_importlib line merged boot-flag.py2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-06-11.09:04:17.337>
created_at = <Date 2015-02-06.22:46:05.940>
labels = ['3.8', 'type-feature', '3.7', 'build']
title = "'make touch' does not work with git clones of the source repository"
updated_at = <Date 2018-06-11.09:04:17.335>
user = 'https://bugs.python.org/vlee'

bugs.python.org fields:

activity = <Date 2018-06-11.09:04:17.335>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2018-06-11.09:04:17.337>
closer = 'ned.deily'
components = ['Build']
creation = <Date 2015-02-06.22:46:05.940>
creator = 'vlee'
dependencies = []
files = ['38033', '40044', '44627', '46620', '46772', '46773']
hgrepos = []
issue_num = 23404
keywords = ['patch', 'buildbot']
message_count = 53.0
messages = ['235505', '235508', '235509', '235510', '235511', '235513', '235514', '235515', '235516', '247510', '247574', '247575', '275285', '276252', '287493', '287580', '291069', '292067', '292068', '292092', '292126', '292143', '292170', '292194', '292207', '292822', '292823', '292829', '292855', '292858', '292860', '292913', '292916', '293002', '293018', '293026', '293030', '293033', '293034', '293094', '295647', '295764', '295767', '295769', '295770', '299181', '303572', '303624', '319282', '319283', '319285', '319286', '319287']
nosy_count = 16.0
nosy_names = ['brett.cannon', 'ncoghlan', 'vstinner', 'larry', 'benjamin.peterson', 'ned.deily', 'ezio.melotti', 'python-dev', 'berker.peksag', 'martin.panter', 'zach.ware', 'koobs', 'willingc', 'josh.r', 'vlee', 'yan12125']
pr_nums = ['1405', '1461', '1464', '1465', '1466', '2127', '2128', '7620', '7621', '7622', '7623', '7624']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23404'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.7', 'Python 3.8']

[vlee]

Is there a minimum Python requirement to build Python?

Python 3.5 does not build with Python 2.6. Python 3.4, Python 3.3, and Python 2.7 build with Python 2.6 so this is recent change in build requirements.

For example, this build failure occurs on CentOS 6.

python ./Parser/asdl_c.py -h Include ./Parser/Python.asdl
Traceback (most recent call last):
  File "./Parser/asdl_c.py", line 6, in <module>
    import asdl
  File "Parser/asdl.py", line 36
    builtin_types = {'identifier', 'string', 'bytes', 'int', 'object', 'singleton'}
                                 ^
SyntaxError: invalid syntax

[MojoVampire]

Looks like this revision, part of bpo-22823, changed it: https://hg.python.org/cpython/rev/4480506137ed

[MojoVampire]

To answer your question: Since asdl uses set literals (introduced in 2.7 and 3.0), that appears to put an effective minimum version requirement of 2.7 to build 3.5. Whether that was the intent is unclear (they were changing a lot of places from set([a, b, c]) to {a, b, c}; causing problems building Python was probably not the intent, but 2.6 is well out of support at this point).

[ned-deily]

Try using "make touch" before "make". Because hg does not preserve precise time stamps when creating working directories, some build steps are run unnecessarily after an initial checkout. 'make touch' updates the file time stamps so that these steps are skipped and a "bootstrap" Python is not needed.

[ned-deily]

https://docs.python.org/devguide/setup.html#avoiding-re-creating-auto-generated-files

[vlee]

I am building from a git copy of the source repository. "make touch" before "make" does not work for me.

$ make touch
cd .; \
	hg --config extensions.touch=Tools/hg/hgtouch.py touch -v
abort: There is no Mercurial repository here (.hg not found)!

I've attached a patch that addresses the Python 3.5 build issue for me. The patch contains two syntax construct changes to Parser/asdl.py. Please consider applying this patch upstream.

[ned-deily]

Unfortunately, those syntax differences aren't the only problems you could run into so making these syntax changes isn't really a general solution, IMO. 'make touch' is the documented and supported way to ensure that the unnecessary build steps are not run. But, as you discovered, 'make touch' only works with an hg copy of the repo; it was not designed for git clones. To fully support git clones, there would need to be a git equivalent of the hg scripts called by 'make touch'. This would be a worthwhile project for someone interested in git support to work on and would be considered for inclusion in the repo.