Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• Python/LibraryStyleGuide

This document is intended as an approachable style guide to the packaging of new Python modules for Debian, presenting the guidelines published in the official Debian Python Policy using less formal language. If you have not read that document, stop now and please read it; this information is designed to augment, not supplant, it and builds on information defined there.

This page captures in example form the best practices for simple Python modules as can be found on the Python Package Index (PyPI). A few assumptions about the Python package you are working on are being made, namely that it:

• Is PEP 517-compliant, with a working declarative pyproject.toml or setup.cfg file (and formerly setup.py) and should be generally installable without modification into a virtualenv.

• Is a "Pure-Python" or simple extension module that can be successfully installed using only pip install (or the older python3 ./setup.py install incantation),
   (For instructions on building more complex modules or extensions, you will need to adapt these guidelines to their needs.)

• Supports at least Python 3,
• Has an upstream-provided test suite, and
• Has some Sphinx-based documentation.

The design of this document is meant to present widely-held best practices in a format that facilitates easy adoption into your own packages, and is by its very nature a permanent work-in-progress.

NOTE: The Python App Style Guide describes an alternative approach useful for crafting a single package debian/rules file.

Overview

There are many ways to build Python libraries for Debian, and each package presents its own corner cases and challenges. In modern Debian (e.g. unstable as of November 2013) and derivatives, the easiest way to build Python library packages is with the pybuild debhelper build system. This link has details on pybuild, including more examples for your cargo culting pleasure.

If you can't use pybuild, say because you want to be able to backport your package to distribution versions before pybuild was introduced, you may be interested in the historical version of this document, which contains all the gory details you had to implement manually, but which pybuild now takes care of for you.

The examples below are taken from the package for the popular urllib3 module, with modifications for generality.

Note: It is highly recommended you explicitly set the dh build system to pybuild. By default, if there is a setup.py, dh will set the build system to python_distutils which only supports Python 2 and you'll get errors during package build as it tries to invoke some nonexistent Python 2 tools.

Packaging files

debian/control

Build-Depends

One very important thing to get right is the Build-Depends line in the source package stanza. setuptools/distribute-based packages have the nasty habit of downloading dependencies from PyPI if they are needed at python3 setup.py build time. If the package is available from the system (as would be the case when Build-Depends is up-to-date), then distribute will not try to download the package, otherwise it will try to download it. This is a huge no-no, and pybuild internally sets the http_proxy and https_proxy environment variables (to 127.0.0.1:9) to prevent this from happening.

dh_python3 will correctly fill in the installation dependencies (via ${python3:Depends}), but it cannot fill in the build dependencies. Take extra care in getting this right, and double check your build logs for illegal access to pypi.org.

You'll want to have at least the following build dependencies:

• debhelper-compat (= 13)
   package, man page

• dh-python
   package, man page
    …or after version 3.20190307, its far more convenient alias: dh-sequence-python3

• python3-all
   package, man page

• And at least one—and sometimes more—of:

  • python3-build
     package, man page

  • python3-setuptools
     package, official documentation

  • python3-wheel
     package, official documentation

If the package has documentation in reStructuredText format, often processed by way of Sphinx, these dependencies are also needed:

• python3-docutils

• python3-sphinx

Python versions

You may want this line in the first stanza, which defines the source package…

X-Python3-Version: >= 3.X

…where X is the minimum version of Python interpreter the package requires. The Debian stable distribution (trixie, as of 2024-12-11) includes python3.11, so if the minimum version is less than 3.11, there is no need to define X-Python3-Version. This is to support proper dependency generation for backports.

Documentation package

If you're building upstream documentation, it's best to put those files into a separate python-foo-doc package, like:

Package: python-foo-doc
Architecture: all
Section: doc
Depends: ${sphinxdoc:Depends}, ${misc:Depends}
Description: Python frobnicator (common documentation)
 This package frobnicates a Python-based doohicky so that you no longer
 need to kludge a dingus to make the doodads work.
 .
 This is the common documentation package.

debian/rules

The contents of this file depends on whatever build system you choose, however we highly recommend the pybuild build system for its simplicity (at least for setup-based packages, i.e. the majority of those on PyPI).

Here is a debian/rules file for you to start with. First, I'll show you the whole thing, then I'll explain it line by line.

   1 #!/usr/bin/make -f
   2 
   3 #export DH_VERBOSE = 1
   4 export PYBUILD_NAME = foo
   5 
   6 %:
   7         dh $@ --with python3 --buildsystem=pybuild

The file starts with the standard #! "shebang" line. I like adding the (commented out before uploading) DH_VERBOSE variable seen on line 3 because it can make build problems easier to debug.

Line 4 then defines the important PYBUILD_NAME variable. pybuild supports a number of environment variables which control such things as the destination directory for build artifacts. The defaults generally do the right thing, but you do typically need to at least tell pybuild the name of your package. Here, that the name is foo. This should match the module name, so for example, in python-urllib3, examining its debian/rules file reveals the presence of this line (#3)…

export PYBUILD_NAME = urllib3

…even though the name of the binary package that it produces is python-urllib3.

The final two lines (nos. 6–7) are the standard debhelper-based Makefile match-anything pattern rule used to catalyze the entire package build process:

%:
        dh $@ --buildsystem=pybuild --with python3

What's important to note here is that the dh_python3 helper is being invoked, and also that the build system that dh will use is pybuild. There's a lot of magic packed into this line, and the pybuild manpage goes into more detail, but essentially what this does is:

• Builds the Python 3 package for all supported Python 3 versions.
• Automatically detects the command to run the test suite.
• Runs the Python 3 test suite.

Once all that's done, it properly installs all the files into binary packages.

And that's it! You usually won't need a debian/python3-foo.install file even if you have multiple binary packages, because again, pybuild does the magic for you. You might need these if there are some non-standard installation tricks you need to implement.

debian/links

You may or may not need this file. It can be useful for Sphinx-based documentation, by providing the original reST files used to build the HTML documentation. Here is an example debian/python-foo-doc.links file.

usr/share/doc/python-foo-doc/html/_sources usr/share/doc/python-foo-doc/rst

debian/watch

Make sure that you have a working debian/watch file, for referencing upstream distribution tarballs. Usually, uscan is sufficient to download tarballs from the declarations in debian/watch (or, if you're using svn-buildpackage, this can be done automatically with svn-buildpackage --svn-download-orig), but if the source needs to be repackaged, you should then provide a get-orig-source (policy) target in debian/rules which does the job.

Since probably for most of our packages, the original tarball comes from the Python Package Index (a/k/a PyPI, or the "Cheese Shop"¹), here is a sample debian/watch file that you might be able to adapt for your own package:

version=4
opts="uversionmangle=s/(rc|a|b|c)/~$1/" \
  https://pypi.debian.net/urllib3/urllib3-@ANY_VERSION@@ARCHIVE_EXT@

Notice the use of pypi.debian.net instead of pypi.org. Debian Services now operates a public redirector service for PyPI to react to changes in the site's listing format so there's no need to change all of the debian/watch files each time in response. In fact, it's trivial to get a valid debian/watch file for your package! Just invoke this shell one-liner in the debian directory of your source package, substituting the module's name on PyPI for <module_name> (be advised that this will overwrite or "clobber" any existing watch file there, so rename it first if you wish to keep it):

wget -HN "https://pypi.debian.net/<module_name>/watch"

Sphinx documentation

The Sphinx documentation section has been moved to its own page, to better accommodate new contributors who were confused by the Python specificity of its directions, and by how it was difficult to discover this information when learning how to build Sphinxdocs for a non-Python package. The new page is found at SphinxDocumentation.

Overrides

If you need to override the dh_python3 defaults, do it with an override method in your debian/rules file. Something like this will work:

override_dh_python3:
        dh_python3 --shebang=/usr/bin/python3

If you need to override the default testing regime (e.g. to use pytest), you can do it one of two ways, either:

export PYBUILD_TEST_PYTEST = 1

or you can override the dh_auto_test rule with:

override_dh_auto_test:
        dh_auto_test -- --test-pytest

Building debug packages

As of 2022, you should not manually build separate -dbg packages for Python libraries. No special configuration is needed, and AutomaticDebugPackages will handle everything when you use dh; see "Re: Python3 -dbg packages" (2021-09-03), sent by Matthias Klose <doko AT debian DOT org> on the debian-python mailing list for more information.

If you need details on how this was done before, please consult prior revisions of this document available in its history.

autopkgtest

It's good idea to supply CI (DEP-8) tests for package. These tests check to make sure your built module can be imported, and that they are working in case any runtime dependencies change. autodep8 have support for Python packaging. To enable it, just add to debian/control:

Testsuite: autopkgtest-pkg-python

This will check if a Python module, named the same as the package, can be imported from the system. You could write more complex tests yourself.

You should check these tests locally by installing autopkgtest and autodep8 package. Then run autopkgtest command (see the manpage for details).

Gotchas

Executables and library packages

Let's say you have a Python package which results in libraries and an executable. What is the best practices for naming and organizing your binary packages?

Clearly you want to at least have:

• python3-foo -- the Python 3 library

but where should the /usr/bin/foo script go? You could put it in python3-foo, but that might not be ideal: users may not expect to find executables in packages that look like library package names, and it would make it difficult to support things like ?PyPy3 packages in future.

Here are some recommendations. We do not have a standard (though maybe we should):

• foo - this is nice because it parallels the /usr/bin/foo executable. In all likelihood, if the /usr/bin script name doesn't collide, then the foo binary package won't either.

• python3-foo - if the script is Python 3.x compatible.

MANIFEST.in and missing files

If the upstream package requires some non-standard files to be installed, it must include a MANIFEST.in file. Sometimes these files are omitted from the original tarball because they're mostly useful for creating the sdist. Case in point: a package contained a bunch of doctests in .rst files but python3 setup.py sdist does not include most .rst files by default. Thus, when pybuild went to build the package locally, the .rst files were omitted and the test suite failed because it could not find the doctest.

The solution for this is to either ensure that upstream includes the MANIFEST.in file, or that you add one before pybuild runs.

TODO

• describe egg related information, what should go in the binary package and what shouldn't.

• when to build-depend against (lib|)python3(-all|)(-dev|)

  • build-depending against lib* should never ever be needed.

  • the -dev variants are for when you need to compile C extension: if your project is pure python you don't want this

  • Using the -all variants will compile/test against all current Python(3) versions, rather than only the default one (i.e. binary .so modules will be compiled multiple times).

  • A bare dependency on python3-all-dev breaks cross building. Do use python3-all-dev:native, libpython3-all-dev instead. In a similar vein, rather than using python3-dev, it should be python3-dev:native, libpython3-dev.

  • The non -all variants are to be used when your project can't and is not building against all supported versions.

Notes

See also

• Python/Pybuild

• Python/GitPackaging

CategoryPackaging