Unable to build Python out-of-tree when source tree is readonly.

[tpn]

BPO	15819
Nosy	@loewis, @doko42, @gpshead, @pitrou, @vstinner, @ned-deily, @tpn, @ericsnowcurrently, @vadmium, @zware, @koobs
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Dependencies	bpo-28066: [Patch] Fix the ability to cross compile Python when doing a rebuild of importlib.h
Files	readonly_src.patch hgtouch.diff unused-flags.patch: Unneeded code in Py 2

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tpn'
closed_at = <Date 2016-09-16.02:48:55.797>
created_at = <Date 2012-08-30.06:49:21.582>
labels = ['build']
title = 'Unable to build Python out-of-tree when source tree is\treadonly.'
updated_at = <Date 2017-03-31.16:36:17.495>
user = 'https://github.com/tpn'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:17.495>
actor = 'dstufft'
assignee = 'trent'
closed = True
closed_date = <Date 2016-09-16.02:48:55.797>
closer = 'martin.panter'
components = []
creation = <Date 2012-08-30.06:49:21.582>
creator = 'trent'
dependencies = ['28066']
files = ['27054', '28874', '44574']
hgrepos = []
issue_num = 15819
keywords = ['patch']
message_count = 44.0
messages = ['169430', '169431', '169432', '169433', '169434', '169435', '169437', '169444', '169458', '169461', '169462', '169463', '169480', '169484', '169486', '169489', '169490', '169534', '169535', '169541', '169542', '169544', '169545', '169546', '169552', '169558', '169569', '169595', '173038', '173041', '173042', '173046', '173144', '180788', '181177', '203348', '275751', '275768', '275910', '275948', '275995', '276260', '276265', '276667']
nosy_count = 15.0
nosy_names = ['loewis', 'doko', 'gregory.p.smith', 'pitrou', 'vstinner', 'ned.deily', 'trent', 'rpetrov', 'Arfrever', 'python-dev', 'sbt', 'eric.snow', 'martin.panter', 'zach.ware', 'koobs']
pr_nums = ['552']
priority = 'low'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue15819'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4']

[tpn]

This is somewhat related to Issue bpo-9860. You can't currently build Python out-of-tree when the source directory is readonly. This is because there are a few Parser/pgen/AST/python_asdl.py steps that try and write to $(srcdir).

The attached patch fixes this.

[loewis]

Instead of using "mkdir -p", it should use $(MKDIR_P) (which actually isn't normally "mkdir -p" :-) Other than that, the patch looks fine.

[tpn]

Hmmmmm. Now I'm confused. Is this what you meant?

-------------------

% hg diff
diff -r 019a2390b014 Makefile.pre.in
--- a/Makefile.pre.in   Tue Aug 21 23:41:43 2012 +0000
+++ b/Makefile.pre.in   Thu Aug 30 07:16:55 2012 +0000
@@ -43,6 +43,8 @@
 
 GNULD=         @GNULD@
 
+MKDIR_P=       @MKDIR_P@
+
 # Shell used by make (some versions default to the login shell, which is bad)
 SHELL=         /bin/sh
 
@@ -223,8 +225,8 @@
 
 ##########################################################################
 # Grammar
-GRAMMAR_H=     $(srcdir)/Include/graminit.h
-GRAMMAR_C=     $(srcdir)/Python/graminit.c
+GRAMMAR_H=     Include/graminit.h
+GRAMMAR_C=     Python/graminit.c
 GRAMMAR_INPUT= $(srcdir)/Grammar/Grammar
 
 
@@ -266,9 +268,9 @@
 
 ##########################################################################
 # AST
-AST_H_DIR=     $(srcdir)/Include
+AST_H_DIR=     Include
 AST_H=         $(AST_H_DIR)/Python-ast.h
-AST_C_DIR=     $(srcdir)/Python
+AST_C_DIR=     Python
 AST_C=         $(AST_C_DIR)/Python-ast.c
 AST_ASDL=      $(srcdir)/Parser/Python.asdl
 
@@ -605,9 +607,11 @@
 Parser/pgenmain.o:     $(srcdir)/Include/parsetok.h
 
 $(AST_H): $(AST_ASDL) $(ASDLGEN_FILES)
+       $(MKDIR_P) $(AST_H_DIR)
        $(ASDLGEN) -h $(AST_H_DIR) $(AST_ASDL)
 
 $(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
+       $(MKDIR_P) $(AST_C_DIR)
        $(ASDLGEN) -c $(AST_C_DIR) $(AST_ASDL)
 
 Python/compile.o Python/symtable.o Python/ast.o: $(GRAMMAR_H) $(AST_H)
diff -r 019a2390b014 configure
--- a/configure Tue Aug 21 23:41:43 2012 +0000
+++ b/configure Thu Aug 30 07:16:55 2012 +0000
@@ -658,6 +658,7 @@
 GNULD
 LINKCC
 LDVERSION
+MKDIR_P
 RUNSHARED
 INSTSONAME
 LDLIBRARYDIR
@@ -5262,6 +5263,7 @@
     HGBRANCH=""
 fi
 
+MKDIR_P="mkdir -p"
 
 DISABLE_ASDLGEN=""
 # Extract the first word of "python", so it can be a program name with args.

---

Given your "which actually isn't normally mkdir -p" comment... my patch doesn't seem right.

[loewis]

Am 30.08.12 09:19, schrieb Trent Nelson:

Hmmmmm. Now I'm confused. Is this what you meant?

No. Makefile.pre.in *already* has a definition of MKDIR_P, no need to
add one.

[tpn]

That's what I figured you meant initially... Until I couldn't find any MKDIR_P definitions, and couldn't get it working without the hack above.

I thought it might be an implicit make variable as well, but, not so much. (Tested with BSD make and gmake on FreeBSD.)

[akheron]

In 3.3, Makefile.pre.in already defines MKDIR_P, in 3.2 it doesn't.

[tpn]

Ah, I see the MKDIR_P changes in 3.x/configure as well.

Objections to backporting to 3.2?

Is 2.7 off the table?

(Building from a readonly source is handy for Snakebite; saves having a dozen machines needing to use their own hg clone.)

[loewis]

Am 30.08.12 09:47, schrieb Trent Nelson:

Objections to backporting to 3.2?

Is 2.7 off the table?

No, adding AC_PROG_MKDIR_P into all of them is fine.

[python-dev]

New changeset 62044cd5b19b by Trent Nelson in branch '3.2':
Issue bpo-15819: Fix out-of-tree builds from a readonly source.
http://hg.python.org/cpython/rev/62044cd5b19b

[python-dev]

New changeset af36536988fd by Trent Nelson in branch 'default':
Block 78809:62044cd5b19b (Issue bpo-15819) from 3.2.
http://hg.python.org/cpython/rev/af36536988fd

[python-dev]

New changeset ab6ab44921b2 by Trent Nelson in branch 'default':
Issue bpo-15819: Fix out-of-tree builds from a readonly source.
http://hg.python.org/cpython/rev/ab6ab44921b2

[python-dev]

New changeset 2dde5a7439fd by Trent Nelson in branch '2.7':
Issue bpo-15819: Fix out-of-tree builds from a readonly source.
http://hg.python.org/cpython/rev/2dde5a7439fd