rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)

[doko42]

BPO	23968
Nosy	@warsaw, @doko42, @ncoghlan, @vstinner, @larryhastings, @ned-deily, @freakboy3742, @xdegaye, @zware, @koobs, @tpetazzoni, @rkuska, @wscullin, @yan12125
Files	platdir.diff: patch platdir2.diff issue23968-android-regression.patch: Fix a regression after changeset 5c77488830bc

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-08-29.18:04:43.978>
created_at = <Date 2015-04-15.19:04:07.449>
labels = ['expert-installation', 'release-blocker']
title = 'rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)'
updated_at = <Date 2016-08-31.18:22:20.390>
user = 'https://github.com/doko42'

bugs.python.org fields:

activity = <Date 2016-08-31.18:22:20.390>
actor = 'yan12125'
assignee = 'none'
closed = True
closed_date = <Date 2016-08-29.18:04:43.978>
closer = 'doko'
components = ['Installation']
creation = <Date 2015-04-15.19:04:07.449>
creator = 'doko'
dependencies = []
files = ['39049', '43214', '44260']
hgrepos = []
issue_num = 23968
keywords = ['patch', 'needs review']
message_count = 32.0
messages = ['241143', '243874', '243896', '244362', '244411', '244605', '248423', '267342', '267352', '268528', '268530', '268536', '268829', '268833', '269237', '269240', '269360', '271636', '271690', '271691', '271692', '271693', '272790', '273179', '273182', '273185', '273875', '273876', '273903', '273904', '274042', '274056']
nosy_count = 17.0
nosy_names = ['barry', 'doko', 'ncoghlan', 'vstinner', 'larry', 'ned.deily', 'Arfrever', 'freakboy3742', 'xdegaye', 'python-dev', 'zach.ware', 'koobs', 'sYnfo', 'thomas-petazzoni', 'rkuska', 'wscullin', 'yan12125']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'needs patch'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue23968'
versions = ['Python 3.6']

[doko42]

Currently there is only one platform directory for all linux architectures, there are two reasons to have a separate directory for each architecture:

• the contents of the files in this directory differ
  (most prominent case are the RTLD_* constants on various
  architectures).

• The _sysconfigdata.py module is installed in the platdir.
  A cross build of the extensions in the standard library
  is done by running the python interpreter for the build
  machine with the platdir for the host machine. This
  currently works for cross builds targeting a different
  os, but not cross building from one linux architecture
  to another.

This would then remove Lib/plat-linux and introduce some Lib/plat-$(PLATFORM_TRIPLET) directories.

[doko42]

ping? I would like to get this into 3.5.

[ncoghlan]

+1 from me.

[sYnfo]

+1 from the Python team from Fedora, the patch looks good from downstream standpoint.

[warsaw]

I don't know what problems I might have run into previously, but it's working now and seems fine to me. +1

[Arfrever]

What Lib/plat-$(PLATFORM_TRIPLET) directories will be introduced?

This patch would make situation worse on rarely used architectures (these without Lib/plat-$(PLATFORM_TRIPLET) distributed in repository of CPython) if patch for bug bpo-12619 is not applied.

[wscullin]

This would likely improve life for folks with cross-compile toolchains.

[doko42]

attaching an updated patch. besides renaming of the platform directory, the following changes are made:

• it makes the name of the internal _sysconfigdata module dependent
  on the abiflags
• it moves the _sysconfigdata module into the platform directory.
  This address as well bpo-22724, it should address bpo-18083 as well,
  provided that some multiarch tuples for solaris are defined.
• it fixes regeneration of the Python modules from C headers for
  cross builds.
• it introduces a sys.implementation._multiarch attribute to construct
  the name of the platform directory without having the
  _sysconfigdata module available (which now is located in the platform
  dir)

One outstanding issue probably is the generation of the platform directory in the source tree, not in the build tree (but this is existing behaviour).

This should address bpo-12619 (this patch probably needs an update),

[ncoghlan]

This may be of interest for iOS and Android cross-compilation, so I've added Xavier de Gaye to the nosy list, while Russell Keith-Magee added himself.

[python-dev]

New changeset 78d2cb7f66b6 by doko in branch 'default':

• Issue bpo-23968: Rename the platform directory from plat-$(MACHDEP) to
  https://hg.python.org/cpython/rev/78d2cb7f66b6

[doko42]

now checked in, after 3.6.0a2, to watch behaviour on different platforms. Still needs doc updates.

[python-dev]

New changeset a062c60c9f7d by doko in branch 'default':

• Issue bpo-23968: Update distutils/sysconfig.py to look for the renamed
  https://hg.python.org/cpython/rev/a062c60c9f7d

[python-dev]

New changeset e2b09c00ee24 by Ned Deily in branch 'default':
Issue bpo-23968: Fix installs of the renamed config directory for OS X
https://hg.python.org/cpython/rev/e2b09c00ee24

[ned-deily]

There is a problem with the current implementation. When running ./configure using compilers other than a current gcc, for example, when using clang or older versions of gcc, the build variable MULTIARCH may not get set because those compilers do not support --print-multiarch:

configure.ac:744
MULTIARCH=$($CC --print-multiarch 2>/dev/null)

But, a bit later in ./configure, PLATFORM_TRIPLET gets set as a result of running conftest.c. The end result is that PLATFORM_TRIPET is non-empty but MULTIARCH is empty. Still later in configure.ac (around 4477:

if test x$PLATFORM_TRIPLET = x; then
LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
else
LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"
fi

So, the value of LIBPL is extended to include PLATFORM_TRIPLET regardless of whether MULTIARCH has a value.

This causes problems at runtime because get_makefile_filename() in Lib/sysconfig.py now uses the value of sys.implemntation's _multiarch attribute (whose value is compiled in sysmodule.c from MULTIARCH) to form the path to the config directory (the equivalent of LIBPL). In these cases, sys.implementation will have no_multiarch attribute so sysconfig looks for the config directory in the pre-3.6 location (e.g. .../lib/python3.6/config-3.6m) even though it was installed in the new location (.../lib/python3.6/config-3.6m-darwin). Fortunately, both test_sysconfig and test_distutils catch this:

======================================================================
FAIL: test_srcdir (test.test_sysconfig.TestSysConfig)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/py/dev/3x/root/uxd/lib/python3.6/test/test_sysconfig.py", line 346, in test_srcdir
    self.assertTrue(os.path.isdir(srcdir), srcdir)
AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm

======================================================================
FAIL: test_get_makefile_filename (test.test_sysconfig.MakefileTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/py/dev/3x/root/uxd/lib/python3.6/test/test_sysconfig.py", line 414, in test_get_makefile_filename
    self.assertTrue(os.path.isfile(makefile), makefile)
AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm/Makefile

======================================================================
FAIL: test_srcdir (distutils.tests.test_sysconfig.SysconfigTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/py/dev/3x/root/uxd/lib/python3.6/distutils/tests/test_sysconfig.py", line 61, in test_srcdir
    self.assertTrue(os.path.isdir(srcdir), srcdir)
AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm

---

I'm not sure what the best solution is.