--enable-optimizations makes common build commands always need to compile from scratch

[zhangyangyu]

BPO	29243
Nosy	@brettcannon, @gpshead, @vstinner, @asvetlov, @methane, @zhangyangyu, @stratakis, @torsava
PRs	bpo-29243: Fix Makefile with respect to --enable-optimizations #1478 [3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) #1518 [3.5] bpo-29243: Fix Makefile with respect to --enable-optimizations … #1520 [2.7] bpo-29243: Fix Makefile with respect to --enable-optimizations … #1522

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-01-11.16:15:14.785>
labels = ['type-feature', '3.7', 'build']
title = '--enable-optimizations makes common build commands always need to compile from scratch'
updated_at = <Date 2017-12-20.20:50:35.331>
user = 'https://github.com/zhangyangyu'

bugs.python.org fields:

activity = <Date 2017-12-20.20:50:35.331>
actor = 'asvetlov'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2017-01-11.16:15:14.785>
creator = 'xiang.zhang'
dependencies = []
files = []
hgrepos = []
issue_num = 29243
keywords = []
message_count = 18.0
messages = ['285233', '285259', '286226', '290076', '293100', '293112', '293113', '293315', '293319', '293334', '293377', '293385', '293387', '293388', '293397', '293414', '293437', '308811']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'gregory.p.smith', 'vstinner', 'asvetlov', 'methane', 'xiang.zhang', 'cstratak', 'torsava']
pr_nums = ['1478', '1518', '1520', '1522']
priority = 'normal'
resolution = None
stage = 'commit review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29243'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6', 'Python 3.7']

[zhangyangyu]

In 3.6 we get --enable-optimizations. One thing I find annoyed with this flag is that it makes some build commands compile from scratch since it always clears the environment first. For example, with the commands listed at the top of Makefile:

./configure
make
make test
make install

It compiles 3 times and considering the optimized build needs more time every time, it lasts long.

I am not sure this is a problem and feel free to close it.

[brettcannon]

Since optimizations uses PGO which should do a fresh run, that's why there's the clearing of the state.

I'm personally not bothered by it as you really should only being doing an optimized build once when you do the first install and then you're done. But if or anyone else can come up with a way to not clear out the results while not accidentally using stale PGO-collected stats or builds then I don't see why we can't speed it up.

[methane]

That's why I want to enable only --with-lto

[zhangyangyu]

Another complaint from bpo-29889.

[torsava]

I've opened a PR#1478 that I believe fixes the issue.

[vstinner]

New changeset a1054c3 by Victor Stinner (torsava) in branch 'master':
bpo-29243: Fix Makefile with respect to --enable-optimizations (bpo-1478)
a1054c3

[vstinner]

I closed my issue bpo-29641 as a duplicate of this one.

[vstinner]

New changeset 03b8a37 by Victor Stinner (torsava) in branch '3.6':
[3.6] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) (bpo-1518)
03b8a37

[vstinner]

New changeset 8489409 by Victor Stinner (torsava) in branch '3.5':
[3.5] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) (bpo-1520)
8489409

[vstinner]

New changeset a473a73 by Victor Stinner (torsava) in branch '2.7':
[2.7] bpo-29243: Fix Makefile with respect to --enable-optimizations (GH-1478) (bpo-1522)
a473a73

[zhangyangyu]

So now we have to run make if we want to trigger the profile procedure since other make commands like make test won't trigger the procedure by itself, right?

[vstinner]

Xiang Zhang added the comment:

So now we have to run make if we want to trigger the profile procedure since other make commands like make test won't trigger the procedure by itself, right?

I don't think that the patch changes the behaviour of "make": it only
changes the behaviour of "make install" and "make test" (and a few
others).

To be honest, I don't know if Makefile knows when a rebuild is needed
and how it works. To run benchmarks, I always rebuild Python from
scratch, even from an empty directory!