RFC: Let's document (and enforce?) a minimum Python version for LLDB

DavidSpickett · 2024-10-24T10:56:08.973Z

Someone raised [lldb] Fix deprecation warning because of Py_SetPythonHome · Issue #113475 · llvm/llvm-project · GitHub recently and suggested it as a good first issue. The main reason I don’t think it is a good first issue is because we don’t document a minimum supported Python version and we don’t enforce one with CMake.

Instead we occasionally have a thread like this and Jonas tells us whether Mac OS is ready to upgrade yet

So let’s agree on a minimum Python version to document on Building - 🐛 LLDB.

I’ve done a survey of the versions used or mentioned:

LLVM itself requires a minimum of 3.8 to run tests or 3.0 without tests (with a note that 3.0 may not actually work). Python is optional for lldb, so this is only a reference point not something we have to follow.
Building - 🐛 LLDB mentions a python36 package install for NetBSD. Could not confirm what current NetBSD uses.
The latest FreeBSD port of Python is 3.11 (FreshPorts -- lang/python3: Meta-port for the Python interpreter 3.x) but I do not know if the system lldb is built with Python. Also FreeBSD can have multiple Python versions installed.
The buildbots:
- lldb-arm-ubuntu - 3.10
- lldb-aarch64-ubuntu - 3.10
- lldb-remote-linux-ubuntu - 3.12
- lldb-x86_64-debian - 3.11
- lldb-aarch64-windows - 3.11.9
Ubuntu system Python versions (python3-defaults package : Ubuntu):
- Bionic - 3.6 (standard support ended 31 May 2023)
- Focal - 3.8 (standard support ends April 2025)
- Jammy - 3.10 (standard support ends April 2027)
- Noble - 3.12 (standard support ends April 2029)
Mac OS? (@JDevlieghere)

With a minimum of 3.6. Though that is for NetBSD who may have moved on since and Ubuntu Bionic which is out of support.

Our code refers to versions 3.2, 3.3, 3.7, 3.9 and 3.13. If we have a set minimum then we can remove some of those #ifs.

I think it’s either 3.6 or 3.7 for our minimum version that is actually used somewhere.

omjavaid · 2024-10-24T11:55:20.821Z

FWIW
For LLVM Windows Release I plan to keep standard python per year on Windows. Move to 3.12.x from LLVM 20 onwards and will keep 3.12.x in LLVM 21 as well. Which actually means that we should now update LLVM buildbot to use python 3.12.x to make sure LLVM 20 runs smooth when its released.

We build LLVM Linux release builds on LTS - 1 (currently Jammy). I guess we dont have any known cases of LLDB being built on bionic anymore. So may be python 3.8 is safe bet for Linux.

jasonmolenda · 2024-10-24T16:21:26.994Z

On macOS, Python3 is included in the Xcode release. The past three years have been

2022, Xcode 14, Python 3.8 + 3.9
2023, Xcode 15, Python 3.9
2024, Xcode 16, Python 3.9

If people are locked on to older OSes that cannot update Xcode (Xcode runs on its matching macOS release, and the previous year’s macOS release, both the OS and Xcode are on an annual update schedule), they can install it from Homebrew packages, or build from source. I’m fine with a requirement of 3.9, as far as macOS is concerned.

JDevlieghere · 2024-10-25T03:29:33.779Z

Thanks for putting this together @DavidSpickett. I’m very supportive of documenting a minimum Python version. The transition from Python 2 to Python 3 was an “interesting” experience but luckily that’s fully behind us now.

As @jasonmolenda pointed out, the latest 3 Xcode releases have shipped with Python 3.9. My preference would be to match LLVM and require at least Python 3.8.

nikic · 2024-10-25T04:45:29.271Z

From a RHEL perspective, anything <= 3.9 is convenient for us, because that’s the system Python version on RHEL 9. On RHEL 8, it’s Python 3.6, but because the rest of LLVM requires a newer version, we end up using Python 3.12 instead anyway (which is less good for overall system integration, but not super important for lldb’s case).

I generally think it’s a good idea to have the same Python version requirement in lldb as for the rest of LLVM (so that would be Python 3.8 currently).

DavidSpickett · 2024-10-25T07:16:05.943Z

Thanks @nikic I didn’t think to check Redhat.

https://apt.llvm.org/ goes back to Focal (3.8) for Ubuntu and Debian 10 (3.7, Python - Debian Wiki). Is that correct @sylvestre?

Maybe you are also using a minimum of 3.8 to match llvm.

DavidSpickett · 2024-10-25T07:20:14.634Z

jasonmolenda:

they can install it from Homebrew packages

This applies to lldb and its dependencies. So they could use Homebrew to install a newer Python first. The earliest one is 3.8 (python@3.13 — Homebrew Formulae), which is convenient.

sylvestre · 2024-10-25T07:31:13.427Z

this is correct

tedwoodward · 2024-10-27T02:44:58.325Z

I’m building with and shipping Python 3.11.4 with my Hexagon and RISC-V toolchains. I’ll upgrade to 3.14 or 3.15 when 3.11 is end-of-life.

I chose 3.11 because it was the first release that officially supported ARM Windows, and we ship both Hexagon and RISC-V toolchains on ARM Windows.

labath · 2024-10-28T10:37:14.320Z

+1 for matching LLVM.

LLDB’s use case is somewhat… different from the rest of llvm, so I can imagine us moving forwards, if we have a good reason, but:
a) I’m not sure if have that right now
b) it definitely doesn’t make sense to require a lower version than llvm

DavidSpickett · 2024-10-28T10:48:02.847Z

Matching llvm makes sense, but I should emphasise that llvm requires 3.8 if and only if you enable tests:

github.com

llvm/llvm-project/blob/64148944c56f50a50383c79d5b6d074d8608f2a4/llvm/CMakeLists.txt#L971


      
            set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default ON)
          else()
            set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default OFF)
          endif()
          set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR ${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default} CACHE BOOL
            "Enable per-target runtimes directory")
          
          set(LLVM_PROFDATA_FILE "" CACHE FILEPATH
            "Profiling data file to use when compiling in order to improve runtime performance.")
          
          if(LLVM_INCLUDE_TESTS)
            # All LLVM Python files should be compatible down to this minimum version.
            set(LLVM_MINIMUM_PYTHON_VERSION 3.8)
          else()
            # FIXME: it is unknown if this is the actual minimum bound
            set(LLVM_MINIMUM_PYTHON_VERSION 3.0)
          endif()
          
          # Find python before including config-ix, since it needs to be able to search
          # for python modules.
          find_package(Python3 ${LLVM_MINIMUM_PYTHON_VERSION} REQUIRED

I don’t know that llvm has anything linking against Python like we do.

If 3.8 is an ok minimum for us anyway, then this detail is not important.

Edit: Getting Started with the LLVM System — LLVM 20.0.0git documentation as well but it makes the same point as the CMake does.

labath · 2024-10-28T14:45:20.654Z

If you disable tests, is python even used for anything in llvm? Some build time scripts perhaps? There’s certainly not many of them, so I can see how they could have a stricter (lower) version requirement.

And I think the same could be said for lldb: if we disable tests and linking with python, then our usage of python is very low, and if someone was motivated enough, then he could probably make that work with an earlier version without too much effort. So, given that llvm (at least the llvm subproject) does link against python, we could sort of say that we’re matching llvm (but maybe we also don’t have to – I think people build llvm in more constrained environments)

DavidSpickett · 2024-10-29T16:29:03.324Z

The minimum versions we can be sure of choosing:

Ubuntu (Focal) - 3.8
Windows on Arm - 3.11
Mac OS - 3.8
Redhat - 3.8

3.8 is the favourite right now.

I have sent out some emails to folks who can hopefully tell me what the BSDs use.

Windows x86 we are safe in assuming 3.8 will run there. We already have situations where people have to install specific Python’s there to use certain lldb builds (which is not ideal but out of scope for this thread).

Debian’s current LTS ships 3.9, but apt.llvm goes back to Debian 10 which is 3.7. @sylvestre , would a minimum of 3.8 be a problem for your Debian 10 builds? (I kinda asked this already but I didn’t phrase it properly the first time)

For enforcing the minimum, perhaps we start by warning for < 3.8 and recommending 3.8 in the documentation.

Once the 20 branch is taken, we make it a cmake error to have < 3.8, document it as the minimum and remove support code for anything older.

emaste · 2024-10-29T17:46:58.662Z

I sent this by email to @DavidSpickett but might as well add it here for posterity.

For the copy of LLDB in the FreeBSD base system we provide (only) lua scripting, as we already have lua available and do not want to maintain Python as contrib code.

In the ports tree we have Python versions from 3.8 through 3.11. Our 3.8 port is expiring at the end of the year:

DEPRECATED=     Upgrade to a newer Python version. 3.8 is in
maintenance status and gets security fixes only. End-of-Life: 2024-10.
See https://devguide.python.org/versions/

LLDB using 3.8 as a minimum requirement is fine for FreeBSD.

DavidSpickett · 2024-10-30T10:14:05.851Z

Thanks @emaste.

I also heard back from Brad Styles who said that:

OpenBSD uses Python 3.11 as the default. NetBSD uses Python 3.12 as the
default.

Once we have confirmation about llvm’s Debian 10 builds, I will go ahead with 3.8 as the minimum. Warning for 20, requirement for 21.

BradSmith · 2024-10-31T03:06:21.940Z

DavidSpickett:

I also heard back from Brad Styles who said that:

OpenBSD uses Python 3.11 as the default. NetBSD uses Python 3.12 as the
default.

There was some mix-up there. That is not my last name.

DavidSpickett · 2024-10-31T09:20:31.954Z

BradSmith:

There was some mix-up there. That is not my last name.

Sorry! Somehow I mixed up your first name and part of the domain name of your commit email.

DavidSpickett · 2024-11-04T14:53:08.577Z

DavidSpickett:

For enforcing the minimum, perhaps we start by warning for < 3.8 and recommending 3.8 in the documentation.

github.com/llvm/llvm-project

[lldb] Recommend Python 3.8 as the minimum Python version for LLDB

llvm:main ← DavidSpickett:lldb-python

opened 02:52PM - 04 Nov 24 UTC

DavidSpickett

+23 -16

See https://discourse.llvm.org/t/rfc-lets-document-and-enforce-a-minimum-python-…version-for-lldb/82731 for discussions. This matches LLVM's requirement to run tests. For LLDB 20 there will be a CMake warning telling builders that from LLDB 21 this will be a hard requirement. From LLDB 21, it will be an error to try to build with anything <= 3.8. So there are no code changes in this commit. Once the llvm 20 branch is created we can remove some < 3.8 support code. As always, if you disable Python support you will not get any new warnings or errors from this change.

DavidSpickett · 2025-01-28T11:49:40.838Z

[lldb] Make Python >= 3.8 required for LLDB 21 by DavidSpickett · Pull Request #124735 · llvm/llvm-project · GitHub is in review, in anticipation of the 20 branch happening soon.

DavidSpickett · 2025-01-29T10:06:13.513Z

That PR was landed. Python 3.8 is now the minimum requirement.