Fix cross compilation of Python ARM64 greenlet extension for Windows ARM64 using llvm-mingw#224
Closed
ader1990 wants to merge 1 commit intopython-greenlet:masterfrom
Closed
Fix cross compilation of Python ARM64 greenlet extension for Windows ARM64 using llvm-mingw#224ader1990 wants to merge 1 commit intopython-greenlet:masterfrom
ader1990 wants to merge 1 commit intopython-greenlet:masterfrom
Conversation
The patch adds platform implementation for win32 llvm reusing platform/switch_aarch64_gcc.h and fixes the stackref type to int64_t, as long is considered to be 32bit by llvm-mingw. The .pyd extension for greenlet can be manually built using: https://github.com/mstorsjo/llvm-mingw. How to set the env to build a working .pyd: * Install mingw64 or mingw32 * Download the llvm-mingw binaries and add them to path https://github.com/mstorsjo/llvm-mingw/releases * Download a Python 3.9 build for ARM64 and unzip in greenlet/src/greenlet https://github.com/ader1990/cpython-builder/actions/runs/377798403 How to build the .pyd using llvm-mingw: * cd greenlet/src/greenlet * aarch64-w64-mingw32-clang -c greenlet.c -o greenlet.o -I./cpython/include * aarch64-w64-mingw32-clang -shared -o _greenlet.cp39-win_arm64.pyd greenlet.o -L./cpython/libs -l:python3.lib -l:python39.lib Partially-fixes: python-greenlet#220
Contributor
|
Why does this need special build instructions? Why doesn't a simple I don't think I can ask users to both install some special binaries and then compile manually from a source checkout. |
Author
|
Hello @jamadden, This patch brings the ability to compile the dll for greenlet, I have to make another patch to be able to use the llvm compiler when using python setup.py. Using python setup.py is a little complicated, as https://github.com/mstorsjo/llvm-mingw requires mingw x86/x64 and cross compiles, but the python.exe is arm64. Currently, on Windows, the MSVC compiler is used and there is no platform implementation for ARCH64. The implementation should be similar to https://github.com/python-greenlet/greenlet/blob/master/src/greenlet/platform/switch_x64_masm.asm, but for ARCH64. That implementation is, to put it bluntly, unchartered territory, see #220. Thank you, |
ader1990
changed the title
ader1990
changed the title
Contributor
|
Thank you for the explanation. I will wait to merge a complete, native, solution. |
Author
This is a complete, native solution. I am working on a change so that it can leverage the llvm compiler for the build, so that there is no need for manual intervention to change the pyd. |
Contributor
|
A complete, native solution does not involve cross compilation or installing third-party software. It uses the same tools that building all other extensions does. |
netbsd-srcmastr
referenced
this pull request
in NetBSD/pkgsrc
Oct 29, 2023
Switch to wheel.mk. 3.0.1 (2023-10-25) ================== - Fix a potential crash on Python 3.8 at interpreter shutdown time. This was a regression from earlier 3.0.x releases. Reported by Matt Wozniski in `issue 376 <https://github.com/python-greenlet/greenlet/issues/376>`_. 3.0.0 (2023-10-02) ================== - No changes from 3.0rc3 aside from the version number. 3.0.0rc3 (2023-09-12) ===================== - Fix an intermittent error during process termination on some platforms (GCC/Linux/libstdc++). 3.0.0rc2 (2023-09-09) ===================== - Fix some potential bugs (assertion failures and memory leaks) in previously-untested error handling code. In some cases, this means that the process will execute a controlled ``abort()`` after severe trouble when previously the process might have continued for some time with a corrupt state. It is unlikely those errors occurred in practice. - Fix some assertion errors and potential bugs with re-entrant switches. - Fix a potential crash when certain compilers compile greenlet with high levels of optimization. The symptom would be that switching to a greenlet for the first time immediately crashes. - Fix a potential crash when the callable object passed to the greenlet constructor (or set as the ``greenlet.run`` attribute) has a destructor attached to it that switches. Typically, triggering this issue would require an unlikely subclass of ``greenlet.greenlet``. - Python 3.11+: Fix rare switching errors that could occur when a garbage collection was triggered during the middle of a switch, and Python-level code in ``__del__`` or weakref callbacks switched to a different greenlet and ultimately switched back to the original greenlet. This often manifested as a ``SystemError``: "switch returned NULL without an exception set." For context on the fixes, see `gevent issue #1985 <https://github.com/gevent/gevent/issues/1985>`_. 3.0.0rc1 (2023-09-01) ===================== - Windows wheels are linked statically to the C runtime in an effort to prevent import errors on systems without the correct C runtime installed. It's not clear if this will make the situation better or worse, so please share your experiences in `issue 346 <https://github.com/python-greenlet/greenlet/issues/346>`_. Note that this only applies to the binary wheels found on PyPI. Building greenlet from source defaults to the shared library. Set the environment variable ``GREENLET_STATIC_RUNTIME=1`` at build time to change that. - Build binary wheels for Python 3.12 on macOS. - Fix compiling greenlet on a debug build of CPython 3.12. There is `one known issue <https://github.com/python-greenlet/greenlet/issues/368>`_ that leads to an interpreter crash on debug builds. - Python 3.12: Fix walking the frame stack of suspended greenlets. Previously accessing ``glet.gr_frame.f_back`` would crash due to `changes in CPython's undocumented internal frame handling <https://github.com/python/cpython/commit/1e197e63e21f77b102ff2601a549dda4b6439455>`_. Platforms --------- - Now, greenlet *may* compile and work on Windows ARM64 using llvm-mingw, but this is untested and unsupported. See `PR <https://github.com/python-greenlet/greenlet/pull/224>`_ by Adrian Vladu. - Now, greenlet *may* compile and work on LoongArch64 Linux systems, but this is untested and unsupported. See `PR 257 <https://github.com/python-greenlet/greenlet/pull/257/files>`_ by merore. Known Issues ------------ - There may be (very) subtle issues with tracing on Python 3.12, which has redesigned the entire tracing infrastructure. 3.0.0a1 (2023-06-21) ==================== - Build binary wheels for S390x Linux. See `PR 358 <https://github.com/python-greenlet/greenlet/pull/358>`_ from Steven Silvester. - Fix a rare crash on shutdown seen in uWSGI deployments. See `issue 330 <https://github.com/python-greenlet/greenlet/issues/330>`_ and `PR 356 <https://github.com/python-greenlet/greenlet/pull/356>`_ from Andrew Wason. - Make the platform-specific low-level C/assembly snippets stop using the ``register`` storage class. Newer versions of standards remove this storage class, and it has been generally ignored by many compilers for some time. See `PR 347 <https://github.com/python-greenlet/greenlet/pull/347>`_ from Khem Raj. - Add initial support for Python 3.12. See `issue <https://github.com/python-greenlet/greenlet/issues/323>`_ and `PR <https://github.com/python-greenlet/greenlet/pull/327>`_; thanks go to (at least) Michael Droettboom, Andreas Motl, Thomas A Caswell, raphaelauv, Hugo van Kemenade, Mark Shannon, and Petr Viktorin. - Remove support for end-of-life Python versions, including Python 2.7, Python 3.5 and Python 3.6. - Require a compiler that supports ``noinline`` directives. See `issue 271 <https://github.com/python-greenlet/greenlet/issues/266>`_. - Require a compiler that supports C++11.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The patch adds platform implementation for win32 llvm reusing
platform/switch_aarch64_gcc.h and fixes the stackref type to
int64_t, as long is considered to be 32bit by llvm-mingw.
The .pyd extension for greenlet can be manually built using:
https://github.com/mstorsjo/llvm-mingw.
How to set the env to build a working .pyd:
https://github.com/mstorsjo/llvm-mingw/releases
https://github.com/ader1990/cpython-builder/actions/runs/377798403
How to build the .pyd using llvm-mingw:
Partially-fixes: #220