sysconfig._parse_makefile doesn't expand ${} vars appearing before $() vars

[doko42]

BPO	24705
Nosy	@doko42, @tarekziade, @berkerpeksag, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/doko42'
closed_at = <Date 2016-01-28.14:59:40.461>
created_at = <Date 2015-07-24.12:13:00.571>
labels = ['type-bug', 'library']
title = "sysconfig._parse_makefile doesn't expand ${} vars appearing before $() vars"
updated_at = <Date 2016-01-28.14:59:40.459>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2016-01-28.14:59:40.459>
actor = 'berker.peksag'
assignee = 'doko'
closed = True
closed_date = <Date 2016-01-28.14:59:40.461>
closer = 'berker.peksag'
components = ['Library (Lib)']
creation = <Date 2015-07-24.12:13:00.571>
creator = 'doko'
dependencies = []
files = []
hgrepos = []
issue_num = 24705
keywords = []
message_count = 7.0
messages = ['247272', '247273', '247274', '247275', '258012', '259139', '259140']
nosy_count = 5.0
nosy_names = ['doko', 'tarek', 'python-dev', 'berker.peksag', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue24705'
versions = ['Python 3.5', 'Python 3.6']

[doko42]

LIPL has the value

${prefix}/lib/python3.5/config-$(VERSION)$(ABIFLAGS)-x86_64-linux-gnu

and the code relies to substitute parameters from the left to the right, but it prefers $() variables. the attached patch substitutes all variables from the left to the right.

diff -r d8229c26dd92 Lib/sysconfig.py
--- a/Lib/sysconfig.py	Fri Jul 24 09:05:59 2015 +0300
+++ b/Lib/sysconfig.py	Fri Jul 24 14:04:57 2015 +0200
@@ -260,7 +260,12 @@
     while len(variables) > 0:
         for name in tuple(variables):
             value = notdone[name]
-            m = _findvar1_rx.search(value) or _findvar2_rx.search(value)
+            m1 = _findvar1_rx.search(value)
+            m2 = _findvar2_rx.search(value)
+            if m1 and m2:
+                m = m1 if m1.start() < m2.start() else m2
+	    else:
+                m = m1 if m1 else m2
             if m is not None:
                 n = m.group(1)
                 found = True

[serhiy-storchaka]

Could you please provide an example where unpatched code fails but patched code work?

[doko42]

On 07/24/2015 03:14 PM, Serhiy Storchaka wrote:

Serhiy Storchaka added the comment:

Could you please provide an example where unpatched code fails but patched code work?

yes, see the substitution for the LIBPL macro, which leaves ${prefix} unexpanded.

[serhiy-storchaka]

Sorry, I don't understand your example. Could you please provide reproducible Python code? Or better a patch for Lib/test/test_sysconfig.py?

[python-dev]

New changeset ef84d21f5292 by doko in branch '3.5':

• Issue bpo-24705: Fix sysconfig._parse_makefile not expanding ${} vars
  https://hg.python.org/cpython/rev/ef84d21f5292

[python-dev]

New changeset dec734dfe2fe by Berker Peksag in branch '3.5':
Issue bpo-24705: Add a test case for ef84d21f5292
https://hg.python.org/cpython/rev/dec734dfe2fe

New changeset f9a18032cc22 by Berker Peksag in branch 'default':
Issue bpo-24705: Add a test case for ef84d21f5292
https://hg.python.org/cpython/rev/f9a18032cc22

[berkerpeksag]

I've added a test case for ef84d21f5292. I think this can be closed now. Please reopen if not.