[macOS] Build macOS installer with LTO and PGO optimizations

[vstinner]

BPO	41181
Nosy	@rhettinger, @ronaldoussoren, @vstinner, @ned-deily, @methane, @ambv
PRs	bpo-41181: macOS build script uses LTO and PGO #21256

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-01.16:20:28.758>
created_at = <Date 2020-07-01.10:32:21.690>
labels = ['OS-mac', '3.8', '3.9', '3.10', 'performance', 'build', 'invalid']
title = '[macOS] Build macOS installer with LTO and PGO optimizations'
updated_at = <Date 2020-11-01.16:37:44.044>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-11-01.16:37:44.044>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-01.16:20:28.758>
closer = 'ned.deily'
components = ['Build', 'macOS']
creation = <Date 2020-07-01.10:32:21.690>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 41181
keywords = ['patch']
message_count = 10.0
messages = ['372743', '372744', '372747', '372758', '372764', '372766', '372767', '372778', '372779', '380153']
nosy_count = 6.0
nosy_names = ['rhettinger', 'ronaldoussoren', 'vstinner', 'ned.deily', 'methane', 'lukasz.langa']
pr_nums = ['21256']
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue41181'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[vstinner]

Link Time Optimization (LTO) and Profile-Guided Optimization (PGO) have a major impact on Python performance: they make Python between 10% and 30% faster (coarse estimation).

Currently, macOS installers distributed on python.org are built with Clang 6.0 without LTO or PGO. I propose to enable LTO and PGO to make these binaries faster.

IMO we should build all new Python macOS installers with these optimizations.

Attached PR adds the flags.

Python 3.9.0b3 binary:

$ python3.9
Python 3.9.0b3 (v3.9.0b3:b484871ba7, Jun  9 2020, 16:05:25) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

configure options:

>>> import sysconfig; print(sysconfig.get_config_var('CONFIG_ARGS'))
'-C' '--enable-framework' '--enable-universalsdk=/' '--with-universal-archs=intel-64' '--with-computed-gotos' '--without-ensurepip' '--with-tcltk-includes=-I/tmp/_py/libraries/usr/local/include' '--with-tcltk-libs=-ltcl8.6 -ltk8.6' 'LDFLAGS=-g' 'CFLAGS=-g' 'CC=gcc'

Compiler flags:

>>> sysconfig.get_config_var('PY_CFLAGS') + sysconfig.get_config_var('PY_CFLAGS_NODIST')
'-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g-std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I/Users/sysadmin/build/v3.9.0b3/Include/internal'

Linker flags:

>>> sysconfig.get_config_var('PY_LDFLAGS') + sysconfig.get_config_var('PY_LDFLAGS_NODIST')
'-arch x86_64 -g'

[vstinner]

The performance issue was noticed by Raymond Hettinger who ran a microbenchmark on tuplegetter_descr_get(), comparison between Python 3.8 and Python 3.9:

https://mail.python.org/archives/list/python-dev@python.org/message/Q3YHYIKNUQH34FDEJRSLUP2MTYELFWY3/

INADA-san confirms that the performance regression was introduced by the commit 45ec5b9 (bpo-40170) which changes PyType_HasFeature() implementation to always call PyType_GetFlags() as a function rather than reading directly the PyTypeObject.tp_flags member.

https://mail.python.org/archives/list/python-dev@python.org/message/FOKJXG2SYMXCHYPGUZWVYMHLDR42BYFB/

On Fedora 32, there is no performance difference because binaries are built with GCC using LTO and PGO: the PyType_GetFlags() function call is inlined by GCC 10.

I built Python on macOS with clang 11.0.3 on macOS 10.15.4, and I confirm that LTO+PGO allows to inline the PyType_GetFlags() function call in tuplegetter_descr_get().

Using "./configure && make":
---

$ lldb ./python.exe
(lldb) disassemble --name tuplegetter_descr_get
(...)
python.exe[0x1001c46ad] <+29>:  callq  0x10009c720               ; PyType_GetFlags at typeobject.c:2338
python.exe[0x1001c46b2] <+34>:  testl  $0x4000000, %eax          ; imm = 0x4000000 
(...)

---

Using "./configure --with-lto --enable-optimizations && make":
---

$ lldb ./python.exe
(lldb) disassemble --name tuplegetter_descr_get
(...)
python.exe[0x1002a9542] <+18>:  movq   0x10(%rbx), %rdx
python.exe[0x1002a9546] <+22>:  movq   0x8(%rsi), %rax
python.exe[0x1002a954a] <+26>:  testb  $0x4, 0xab(%rax)
python.exe[0x1002a9551] <+33>:  je     0x1002a956f               ; <+63>
(...)

---

[ambv]

The installer is built on Mac OS X 10.9 so that it is forward compatible with all OS X and macOS versions. We cannot depend on PGO and LTO for it unless we start building the installer on 10.15. We cannot do this currently as those installers would not work with older macOS and OS X versions.

Since the Mac is switching to Apple Silicon, the plan is to start building a separate macOS 11+ installer. *That* could use PGO and LTO.