gh-141770: Annotate anonymous mmap usage if "-X dev" is used

[corona10]

• Issue: Name arenas allocated by mmap in _PyMem_ArenaAlloc for debugging #141770

[corona10]

Annotation

Without annotation

I verified that this annotation actually works, and I believe it will be very helpful for debugging when needed.

• Ruby: Annotate anonymous mmap ruby/ruby#12119
• Go: https://go.dev/doc/go1.25#vma-names-on-linux

[corona10]

cc @picnixz

[corona10]

FYI, Fedora does not support yet even if the kernel version itself already support it.
See: https://bugzilla.redhat.com/show_bug.cgi?id=2302746

cc @vstinner @encukou

[SpecLad]

FWIW, it would be useful to have an option to enable this in release builds too.

[corona10]

@SpecLad Thanks for the feedback, I will create a separate PR :)

[corona10]

@SpecLad Just out of curiosity, passing the build flag through ./configure --enable-annotate-mmap would be fine to you?
I would like to follow Ruby's approach but disable this flag by default unless many disto actually want this flag.

[SpecLad]

To be clear, by "option" I really mean a runtime option. 🙂 It would be pretty annoying to have to rebuild Python just for this. If you compare with other implementations, none of them require a compile-time option:

• C (glibc) - off by default, can be enabled with environment variable;
• Go - on by default, can be disabled with environment variable;
• Ruby - on permanently (unless I missed something).

TBH, I don't see why Python shouldn't also permanently enable this, but I would settle for "off by default, enabled with environment variable or -X option".

[picnixz]

It's a bit annoying that this would only be supported on Linux. Do we have an option that isn't supported everywhere?

[picnixz]

Do we really need macros? it is just to avoid the (unsigned long) cast? If so, I would suggest having a static inline function (but you can have a macro for the no-op case to avoid an empty function) and leave the rest to the compiler. Or if you want to keep a macro, please align the \ (see PEP-7).

[vstinner]

I would also prefer a static inline function for cleaner code.

[vstinner]

I would also prefer a static inline function for cleaner code.

[corona10]

@vstinner @picnixz I 've applied all your suggestion.

[picnixz]

* Annotating anonymous mmap usage is now supported if Linux kernel
  supports :manpage:`PR_SET_VMA_ANON_NAME <PR_SET_VMA(2const)>`
  (Linux 5.17 or newer).

I don't know under which manpage entry it is, but if you locally run Sphinx, you should be able to find the correct one. That way, users would have a link to click on. If there isn't a manpage, nevermind.

[vstinner]

LGTM

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x (tier-1) has failed when building commit c4ccaf4.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/506/builds/11950) and take a look at the build logs.
4. Check if the failure is related to this commit (c4ccaf4) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/506/builds/11950

Failed tests:

• test_sys
• test_audit
• test_profiling
• test_external_inspection
• test_remote_pdb

Failed subtests:

• test_sys_remote_exec - test.test_audit.AuditTest.test_sys_remote_exec

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_audit.py", line 332, in test_sys_remote_exec
    self.fail(stderr)
    ~~~~~~~~~^^^^^^^^
AssertionError: Fatal Python error: _Py_CheckFunctionResult: a function returned a result with an exception set
Python runtime state: initialized
OSError: Cannot open ELF file '[anon:cpython:pymalloc]' for section 'PyRuntime' search: No such file or directory

[hugovk]

Please can you check the buildbot failure?

[hugovk]

I think it's a problem with the buildbot machine: it's timing out on other branches now, and the builds are taking nearly five hours, when they used to take around 50 minutes:

https://buildbot.python.org/#/workers/31

[vstinner]

I just sent an email to David Bolen (the buildbot owner) to ask him to have a look. Something seems to be wrong with this buildbot worker.

[vstinner]

This change introduced a regression: #142627.

[vstinner]

Note for myself: The feature is not available on Fedora, since Fedora builds the Linux kernel without the CONFIG_ANON_VMA_NAME option:

$ grep CONFIG_ANON_VMA_NAME /boot/config-$(uname -r)
# CONFIG_ANON_VMA_NAME is not set