Kay Hayen: Nuitka Release 2.7
This is to inform you about the new stable release ofNuitka. It is the extremely compatible Python compiler,“download now”.
This release adds a ton of new features and corrections.
Bug Fixes
macOS:Correctly recognize self-dependencies of DLLs that include an architecture suffix onx86_64. (Fixed in 2.6.1 already.)
Standalone:Resolved an issue where.pyifiles associated with versioned extension module filenames were not detected. (Fixed in 2.6.1 already.)
Standalone:Fixed.pyifile parsing failures caused by single-line triple quotes. (Fixed in 2.6.1 already.)
Standalone:Corrected the__spec__.originpath for packages where__init__is an extension module.
While this primarily affected the module’sreprstring, code relying onoriginfor resource location could previously encounter errors. (Fixed in 2.6.1 already.)
Multidist:Ensured created binaries use the name they were launched with, rather than the path of the actual binary file.
This allows entry points invoked viasubprocesswith different process names to function correctly from a single binary distribution. (Fixed in 2.6.1 already.)
Windows:Fixed unusablesys.stdinwhen attaching to a console (--windows-console-mode=attach) without an active terminal, which previously led to errors when forking processes. (Fixed in 2.6.1 already.)
Modules:Prevented crashes in module mode when encountering potentially optimizableimportlibdistribution calls that are non-optimizable in this mode. (Fixed in 2.6.1 already.)
Python3:Resolved errors with newersetuptoolsversions caused by namespace packages not providing apath_finder. (Fixed in 2.6.1 already.)
Windows:Removed a UTF-8 comment from a C header file that could prevent MSVC from compiling correctly in certain system locales. (Fixed in 2.6.2 already.)
Standalone:Ensured that user-inhibited data files have their associated tags cleared to prevent confusion in plugins. (Fixed in 2.6.2 already.)
Standalone:Correctedpkgutil.iter_modulesto prevent incomplete module listings.
Previously, if a package directory existed, Python’s file finder could interfere, yielding incomplete results that excluded compiled modules. (Fixed in 2.6.2 already.)
Windows:Addressed a C compiler warning related to onefile compression on 32-bit Windows during bootstrap compilation.
Python3.12+:Resolved aSystemErrorwhen accessing themoduleattribute oftypevariables.
Metadata:Handled cases where reading record data from distribution files might fail because the files do not always exist. (Fixed in 2.6.3 already.)
Compatibility:Fixed dependency resolution for shared libraries that are symbolic links on non-Windows platforms.
Resolution failed if dependencies originated from the symlink target and that target was not in the system library path or nearby as another symlink. (Fixed in 2.6.3 already.)
Standalone:Prevented the use ofmsvcp140.dllfromPySide6(specificallyshiboken6) by other packages to avoid potential compatibility crashes. (Fixed in 2.6.3 already.)
Standalone:Fixed.pyifile parsing failures caused by comments appearing afterimportstatements. (Fixed in 2.6.3 already.)
UI:Corrected the output of DLL and EXE listings, which no longer provided the correct sub-folder information. (Fixed in 2.6.3 already.)
macOS:Restored the attachment of icons to application bundles. (Fixed in 2.6.3 already.)
Onefile:Resolved a C compiler warning on 32-bit Windows related to missing type conversion for the decompression buffer size. (Fixed in 2.6.3 already.)
Python3.12+:Ensured type aliases have a usablemoduleattribute containing the correct module name. (Fixed in 2.6.3 already.)
Python3.12+:Improved compatibility for type alias values.
Corrected the creation process for compound type aliases, resolving errors in libraries likepydanticwhen used in schemas. (Fixed in 2.6.5 already.)
Python3.12+:Implemented a workaround for extension modules failing to set the correct package context, which previously caused significant compatibility issues (e.g., namespace collisions).
This was particularly problematic when static libpython was unavailable (common on Windows and official macOS CPython). The workaround involves saving/restoring the module context and correcting potentially wrongly named sub-modules created during extension module loading.
This improves compatibility for packages likePySide6QtAds,onnx,mediapipe,paddleocr, and newerscipyversions, which were previously affected. (Fixed in 2.6.6 already.)
Plugins:Prevented a crash in the Nuitka Package Configuration helper functionget_datawhen falling back topkgutilfor data files not found naively. (Fixed in 2.6.6 already.)
Standalone:Corrected the rpath value used for finding dependent DLLs in sub-folders on non-Windows platforms.
It previously excluded the DLL’s own folder ($ORIGIN), sometimes preventing DLLs from loading. (Fixed in 2.6.7 already.)
Standalone:Preserved existing$ORIGIN-relative rpaths in DLLs on Linux.
Some PyPI packages rely on these existing paths to reference content in other packages; replacing them previously broke these setups. (Fixed in 2.6.7 already.)
Standalone:Now treats shared library dependencies specified with paths as implicit rpaths.
This fixes builds using Python distributions likePython Build Standalone(installed byuv), which may have an$ORIGIN-relativelibpythondependency that needs to be respected. (Fixed in 2.6.7 already.)
Python3:Ensured generators preserve external exceptions and restore their own upon resuming.
This fixes issues where generators used as context managers, handling an exception viayield, could prevent an outerwithstatement from correctly re-raising its own exception. (Fixed in 2.6.7 already.)
Android:Removed the Termuxrpathvalue pointing into its installation from standalone builds.
While ineffective after APK packaging due to Android security, this value should not have been present. (Fixed in 2.6.7 already.)
Python Build Standalone:Added the rpath tolibpythonby default in all modes forPython Build Standalonedistributions.
This resolves issues with uninstalledlibpythononLinux. (Fixed in 2.6.8 already.)
Standalone:Resolved incompatibility with older Linux distributions caused by using newerpatchelfoptions introduced in 2.6.8’s rpath changes. (Fixed in 2.6.9 already.)
Python3.9:Fixed errors in thespacyplugin when using olderimportlib.metadataversions. (Fixed in 2.6.8 already.)
Standalone:Preventedrequestspackage imports from being incorrectly treated as sub-packages. (Fixed in 2.6.8 already.)
Distutils on macOS:Improved integration for scanned extension modules where determining the correct architecture can be difficult. (Fixed in 2.6.8 already.)
Windows:Defineddotnetas a dependency to ensure all UI features requiring it are properly enabled.
Scons:Ensured the correctlink.exeexecutable is used for the MSVC backend, avoiding potential conflicts with linkers added to thePATH(e.g., bygit). (Fixed in 2.6.9 already.)
Scons:Avoided usingconfig.txtwithclcachewhen using MSVC.
This prevents potential race conditions during the first use where multipleclcachethreads might attempt to create the file simultaneously. (Fixed in 2.6.9 already.)
Standalone:Ensured extension modules are loaded during thecreate_modulephase of the Nuitka loader for better compatibility.
Loading them later duringexec_modulecaused issues with some extension modules, such as those created bymypy(used inblack).
Python3.13:Corrected the workaround for extension module package context issues, resolving errors that occurred when the module and package names were identical.
Module:Prevented stub generation attempts for namespace packages, which previously resulted in warnings as there is no source code to process.
Debian:Ensured consistent casing for the installer name used in Debian package metadata.
Poetry:Updated detection logic for newerpoetryversions to handle changes in installer name casing, which could previously impact system DLL usage determination.
Module:Improved stub generation (stubgen) for generics, handling of missingtypingimports, and other cases.
Plugins:Fixed potential corruption and crashes in thedill-compatplugin when handling functions with keyword defaults.
Standalone:Added support for newerpy-cpuinfoversions on non-Windows platforms.
Accelerated:Prevented Nuitka’ssys.path_hookfrom overriding standard Python path loader hooks, as it doesn’t yet support all their functionalities.
Python3.12.7+:Set additional unicode immortal attributes (including for non-attributes) to prevent triggering Python core assertions when enabled.
Compatibility:Ensured errors are properly fetched during class variable lookups.
Previously, an error exit could occur without an exception being set, leading to crashes when attempting to attach tracebacks.
Python3.13:Adapted dictionary value creation and copying to follow internal layout changes, preventing potential crashes and corruption caused by using obsolete Python 3.11/3.12 code.
Scons:Corrected the default LTO module count calculation to refer to the number of compiled modules.
Package:Ensured namespace parent modules are included in compiled packages.
These were previously missed due to the removal of reliance on--include-packagefor delayed namespace package handling.
macOS:Ensured data files included in application bundles are also signed.
Windows:Applied short path conversion to the directory part ofsys.argv[0]as well.
This prevents issues with tools called using this path that might not handle non-shortened (potentially unicode) paths correctly.
Package Support
Standalone:Included necessary data files for theblib2to3package. (Added in 2.6.1 already.)
Standalone:Added support for newernumbaversions. (Added in 2.6.2 already.)
Standalone:Added support for newerhuggingface_hubversions. (Added in 2.6.2 already.)
Anti-Bloat:Provided additionalnumpy.testingstubs required for proper execution of somesklearnmodules. (Fixed in 2.6.2 already.)
Standalone:Enhanced configuration forfontTools. Avoided configuring hidden dependencies now detected by parsing provided Python files like.pyifiles. (Fixed in 2.6.2 already.)
Standalone:Corrected plugin configuration for newerPySide6sqldriverson macOS. (Fixed in 2.6.3 already.)
Python3.12+:Introduced standalone support formediapipe, including a workaround for extension module sub-module creation issues. (Fixed in 2.6.3 already.)
Python3.12+:Introduced standalone support foronnx, including a workaround for extension module sub-module creation issues. (Fixed in 2.6.3 already.)
Standalone:Added support for newersqlglotversions. (Added in 2.6.5 already.)
Standalone:Includedassetdata files for thearcadepackage. (Added in 2.6.5 already.)
Standalone:Added implicit dependencies forsqlalchemy.orm. (Added in 2.6.5 already.)
macOS:Included additional frameworks required for PySide 6.8 web-engine support. (Added in 2.6.5 already.)
Standalone:Enhancedcv2support to handle potentially Python minor version-specific config files by allowing optional data file discovery in plugins. (Added in 2.6.6 already.)
Standalone:Added support for thescipysub-module loader mechanism.
By treating it as a lazy loader, implicit dependencies withinscipyare now correctly detected without requiring explicit configuration. (Added in 2.6.7 already.)
Standalone:Automatically include Django database engine modules. (Added in 2.6.7 already.)
Homebrew:Addedtk-intersupport for Python versions usingTcl/Tkversion 9.
Standalone:Included a missing data file for thejennpackage.
Standalone:Added support for newerscipy.optimize._cobylaversions. (Fixed in 2.6.8 already.)
Anaconda:Fixed issues with baremklusage (withoutnumpy).
Standalone:Included a missing data file for thecyclonedxpackage.
Compatibility:Enabled pickling of local compiled functions usingcloudpickleandray.cloudpickle.
Standalone:Added support formitmproxyon macOS.
Standalone:Included necessary data files forpython-docsandmne.
Standalone:Added support for newertogaversions, requiring handling of its lazy loader.
Standalone:Introduced support for theblackcode formatter package.
Standalone:Included metadata when thetravertinopackage is used.
Standalone:Significantly enhanced support for detecting dependencies derived fromdjangosettings.
Standalone:Added support for theduckdbpackage.
New Features
DLL Mode:Introduced a new experimental mode (--mode=dll) to create standalone DLL distributions.
While functional for many cases, documentation is currently limited, and features like multiprocessing require further work involving interaction with the launching binary.
This mode is intended to improve Windows GUI compatibility (tray icons, notifications) for onefile applications by utilizing an internal DLL structure.
Windows:Onefile mode now internally uses the new DLL mode by default, interacting with a DLL instead of an executable in temporary mode.
Use--onefile-no-dllto revert to the previous behavior if issues arise.
Windows:Added support for dependency analysis on Windows ARM builds usingpefile(as Dependency Walker lacks ARM support).
Android:Enabled module mode support when using Termux Python. (Added in 2.6.7 already.)
Compatibility:Added support forPython Build Standalonedistributions (e.g., as downloaded byuv).
Note that staticlibpythonis not supported with these distributions as the included static library is currently unusable. (Added in 2.6.7 already.)
Windows:Enabled taskbar grouping for compiled applications if product and company names are provided in the version information. (Added in 2.6.4 already.)
Windows:Automatically use icons provided via--windows-icon-from-icoforPySide6applications.
This eliminates the need to separately provide the icon as a PNG file, avoiding duplication.
Nuitka Package Configuration:Allowed using values fromconstantsandvariabledeclarations withinwhenconditions where feasible.
Reports:Clearly indicate if an included package is “vendored” (e.g., packages bundled withinsetuptools).
Compatibility:Added support for thesafe_path(-P) Python flag, preventing the use of the current directory in module searches.
Compatibility:Added support for thedont_write_bytecode(-B) Python flag, disabling the writing of.pycfiles at runtime (primarily for debugging purposes, as compiled code doesn’t generate them).
UI:Introduced a new experimental tool for scanning distribution metadata, producing output similar topiplist-v. Intended for debugging metadata scan results.
Plugins:Enhanced thedill-compatplugin to transfer__annotations__and__qualname__.
Added an option to control whether the plugin should also handlecloudpickleandray.cloudpickle.
AIX:Implemented initial enhancements towards enabling Nuitka usage on AIX, although further work is required.
Optimization
Optimized finalizer handling in compiled generators, coroutines, and asyncgens by avoiding slower C API calls introduced in 2.6, restoring performance for these objects.
Implemented a more compact encoding for empty strings in data blobs.
Instead of 2 bytes (unicode) + null terminator, a dedicated type indicator reduces this frequent value to a single byte.
Anti-Bloat
Avoided includingmatplotlibwhen used by thetqdmpackage. (Added in 2.6.2 already.)
Avoided includingmatplotlibwhen used by thescipypackage. (Added in 2.6.2 already.)
Avoided includingcythonwhen used by thefontToolspackage. (Added in 2.6.2 already.)
Avoided includingsparsewhen used by thescipypackage. (Added in 2.6.3 already.)
Avoided includingndonnxwhen used by thescipypackage. (Added in 2.6.3 already.)
Avoided includingsetuptoolsfor thejaxlibpackage.
Also prevented attempts to query the version fromjaxlibsource code using git. (Added in 2.6.3 already.)
Avoided includingyamlwhen used by thescipypackage. (Added in 2.6.4 already.)
Avoided includingcharset_normalizerfor thenumpypackage. (Added in 2.6.5 already.)
Avoided includinglxmlfor thepandaspackage. (Added in 2.6.5 already.)
Avoided includingPIL(Pillow) for thesklearnpackage. (Added in 2.6.5 already.)
Avoided includingnumbawhen used by thesmtpackage. (Added in 2.6.7 already.)
Avoided including more optionalpygamedependencies. (Added in 2.6.8 already.)
Avoided includingsetuptools,tomli, andtomllibfor theincrementalpackage.
Avoided includingIPythonwhen used by therichpackage vendored withinpip.
For reporting purposes, treated usage ofipywidgetsas equivalent to usingIPython.
Added support forassert_raiseswithin Nuitka’snumpy.testingstub.
Organizational
UI:Improved the output format for used command line options.
Filenames provided as positional arguments now use the report path format. Info traces now support an optional leader for intended value output, enhancing readability.
Reports:Saved and restored timing information for cached modules.
This eliminates timing differences based on whether a module was loaded from cache, reducing noise inNuitka-Watchcomparisons where cached module timings previously changed with every new compilation.
Actions:Added compilation report artifacts to all CI runs involving empty module compilations.
Debugging:Enabled the--editoption to find modules within.appbundles created on macOS. (Added in 2.6.1 already.)
User Manual:Updated the Nuitka-Action example; linking directly to its documentation might be preferable. (Changed in 2.6.1 already.)
Quality:Enforced ASCII encoding for all Nuitka C files to prevent accidental inclusion of non-ASCII characters.
Quality:Added syntax validation forglobal_replacementsresult values, similar to existing checks forreplacements.
Also added validation to ensureglobal_replacements_reandreplacements_reresult values are valid regular expressions.
Plugins:Ensured error messages for illegal module names in implicit imports correctly report the originating plugin name.
Quality:Enabled use ofclang-format-21if available and applied formatting changes specific to this newer version.
Quality:Suppressedpylintwarnings related tosetuptoolsusage when running with Python 3.12+.
UI:Disallowed mixed usage of Anaconda and Poetrywithoutan active Poetry virtual environment.
This avoids issues caused by a Poetry bug where it incorrectly sets theINSTALLERmetadata for Conda packages in this scenario, making reliable detection of Conda packages impossible.
macOS:Deprecated--macos-target-archin favor of the standard--target-archoption, with plans for eventual removal.
Release:Ensured usage of a compatiblesetuptoolsversion duringoscuploads.
UI:Improved the error message for invalid custom anti-bloat modes by listing the allowed values.
Release:Removed CPython test git submodules from the repository.
These submodules caused issues, such as being cloned duringpipinstalland sometimes failing, potentially breaking Nuitka installation.
Tests
Enabled passing extra options via theNUITKA_EXTRA_OPTIONSenvironment variable fordistutilstest cases involvingpyproject.toml.
Removed the standalone test for thegipackage, as it’s better covered by Nuitka-Watch and prone to failures in CI due to lack of an X11 display.
Ensured tests correctly ignore the current directory when necessary by using the new--python-flag=safe_path.
This forces the use of the original source code as intended, rather than potentially finding modules in the current directory.
Corrected the implementation of theretrymechanism forpipenvinstallation within thenuitka-watchtool.
Added support for passing extra options via environment variables to thenuitka-watchtool.
Cleanups
Distutils:Standardized usage to--module=packagewhere appropriate, instead of manually adding package contents, resulting in more conventional Nuitka command lines.
Refactored.pyifile creation into a dedicated function, simplifying the post-processing code.
Summary
This release was supposed to focus on scalability, but that didn’t happen due to a variety of important issues coming up as well as unplanned private difficulties.
The added DLL mode will be very interesting to many users, but needs more polish in future releases.
For compatibility, working with the popular (yet - not yes recommendedUV-Python),WindowsUI fixes for temporary onefile andmacOSimprovements, as well as improvedAndroidsupport are excellent.
The next release of Nuitka however will have to focus on scalability and maintenance only. But as usual, not sure if it can happen.
