4. General Notes

One of the major features of Python packaging on Debian systems is that pure Python modules are byte compiled for all officially supported versions of Python that are available, and are actually supported by the module itself. There are several utilities available to aid in this task. Two of the most popular (and mutually exclusive -- select one or the other for use in your packages) (python-central and python-support, for example). However, these utilities need information about modules in the package (for example, whether a pure Python module supports all versions of Python or not).

Python packages (unrelated to Debian packages) are related parts of a software system, that consist of a file __init__.py, pure Python modules, extension modules, and other packages. These Python packages must be packaged into the same directory structure employed by the upstream Python package, since splitting the package components across directories, or otherwise changing the layout, changes the layout order and may cause breakages, subtle bugs, render the documentation inaccurate, and confuse IDEs and other tools.

4.1. Naming python module packages

Packages containing public modules should be named, or should provide, python-foo, where foo is the name of the Python module.

4.2. Python versions supported by the source

The XS-Python-Version field in debian/control specifies the versions of Python supported by the package [1]. While this is a requirement only if using the utility package python-central (python-support, for example, prefers debian/pyversions), setting this is "appreciated" in any case, according to the new policy wiki[2]. This is used to track packages during Python transitions, and test rebuilds.

This can be a single version, or one or more of a list of non-overlapping ranges. The lowest range may optionally omit a low end, and the highest range may optionally omit an upper end. In other words, the overall range may be open ended. The ranges are often matched to the set of all known Python version that have existed, and the supported set is the intersection of the known versions of python and the range specification.

  1. If the current version of Python is supported by the package, then:

    • For packages with private modules or private extensions compiled for the current python version and for applications using /usr/bin/python, this should be set to the string "all" (or "-", in the case of debian/pyversions). [3] If the module doesn't work with all Python version, the set of versions supported should be used [4]

    • For packages with public modules, this should be set to the string "all" (or "-", in the case of debian/pyversions), unless not all versions of Python are supported (in which case the setting should specify the versions or set of versions actually supported, like ">= 2.4" or ">= 2.2, << 2.y".

  2. If the current version is not supported, or if the application uses /usr/bin/pythonX.Y explicitly, then this should be set to the (list of) version(s) supported.

4.3. Byte compilation

Any pure Python modules shipped in a package should not ship with the corresponding byte compiled versions, since the byte compiled file is specific to the version of Python that did the compilation. In order to ease transitions, it is desirable that the byte compilation be done on the target system on installation, and it should be feasible to redo the byte compilation at will at any time (for example, when a new version of Python is installed, or hen the default version of Python changes).

So the package maintainer scripts arrange to have the pure Python modules compiled on installation, and removed when the package is removed, and also arrange to hook into the rtupdate mechanism to redo the byte-compilation whenever the status of the installed Python versions on the system changes. There are a variety of tools (python-central and python-support) to facilitate this task.

A package should only compile pure Python modules that belong to it, and should honor the setting in the configuration file /etc/python/debian_config which allows the local administrator to configure how modules are byte-compiled.

4.4. Linking extention modules

There should be no reason to link the extension against the python library. Usually many extensions which are developed upstream on Windows do link by default to libpython. Other extensions linking against libpython are those with build infrastructure maybe predating distutils.

4.5. Depends:

The rules for calculating the dependencies a package has are simple.

  1. If a script invokes /usr/bin/pythonX.Y, than the package must depend on pythonX.Y. This is because no amount of automatic byte compiling would ever get rid of the requirement that /usr/bin/pythonX.Y has to be present for the script to function.

  2. For package that contains extensions, the set of Python versions required has to be restricted to Python versions for which the extensions have been built and shipped in the package (this is in intersection with the set of versions restrictions already in place, for example if the module explicitly requires some version of python). For packages with private extension modules, this means that the set of python versions it depends on has to be set to whatever version of Python was used during the build process (since private extension module packages can only be compiled for one version of Python at any time).

      1. If any version of python can be supported, there are two [5] sub cases:

        1. For pure Python modules, the package just depends on python. This includes pure Python private modules (as long as they do not depend on any private extensions -- in which case the extensions shall restrict the dependency).

        2. Private extension modules can only be compiled for one version of Python at a time, so even if they have no restrictions on the versions of python supported, they must depend on just the current version (which is the version they are likely to be compiled for).

          To do this, one determines the stop version: the stop version is next minor version after the current version (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor.

          In this case, the package must depend on: python (>= $current_version), python (<< $stop_version)

    1. If there is a set of values of versions of Python supported, there are two sub cases:

      1. The current version is supported. In this case, select the larger interval in the set of Python versions supported that contains the current version, and determine the lower and upper bounds.

        If the lower bound to the set exists, then add a dependency on python (>= $min_version)

        If the upper bound to the set exists, then the stop version is the next minor version after the upper bound (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. If the upper bound to the set exists, then depend on python (<< $stop_version)

      2. The package does not support the current version of Python, again, there are again two sub cases:

        1. The set of Python versions supported is higher than the current version. In this case, the package should depend upon: python (>= $min_version) | python$min_version)

          If the upper bound to the set exists, then the stop version is the next minor version after the upper bound (found by incrementing the minor version), whether or not it exists, and is form of Major.Minor. If the upper bound to the set exists, then depend on python (<< $stop_version)

        2. The set of Python versions supported is lower than the current version. This implies that the package is obsolete, and should either be updated to support the current Python version, or be dropped from the archive.

Packaged modules that require other modules to work, must depend on the corresponding python-foo packages. They must not depend on any pythonX.Y-foo package directly.

Packaged modules available for only one particular version of Python that need other modules (say, "bar"), must depend on the corresponding pythonX.Y-bar packages, and must not depend on any python-bar. For consistency, if the package ("foo") provides several pythonX.Y-foo packages, and it needs the module "bar", it must also depend on pythonX.Y-bar corresponding to each version "X.Y" for the virtual packages pythonX.Y-foo that it provides.

4.6. Provides

Packages with public modules and extensions should be named, or should provide, python-foo. Pure Python public modules that support all Python versions need not have a Provides field.

For package that contains public extensions, the set of versions supported has to be restricted to Python versions for which the extensions have been built and shipped in the package.

Public pure Python modules that have a subset of all python versions supported, or for public extensions, the Provides field indicates which versions of Python are supported (for which one may import the module). For every version of python supported the package should provide pythonX.Y-foo packages. This assumes that the package correctly depends on all the appropriate versions of any version specific module that it itself requires.

4.7. Build-Depends:

If the package provides public extension modules, then build depending on "python-all-dev (>= 2.3.5-11)" shall ensure that all the >pythonX.Y-dev packages are available during building. Pure python modules do not require any development packages at build time, since they are byte compiled on the target machine at install time.

On the other hand, if the package provides private modules, then build depending on "python-dev", or one specific "pythonX.Y-dev" (if there are restrictions on the version of Python supported) is sufficient, since only one version of Python is ever supported in any package.

If you are using helper packages (CDBS, yada, debhelper, etc) you should build depend on the appropriate version. This also applies if your build process uses either python-central or python-support.

4.8. Deprecating "current" in versions supported

Currently, the string "current" in the field XS-Python-Version is used by python-central to indicate that the package contains private modules, and explicitly state that the package is only built for the current Python version, and not for any other version supported in Debian. This is flawed, for the following reasons:

Notes

[1]

python-central also uses XB-Python-Version field in the control file, which is substituted in the binary package using the values from the XS-Python-Version field, filtered according to various rules.

[2]

For compatibility, each of the tools (python-central and python-support) are able to use each others data.

[3]

Note that pycentral uses the special string "current" here.

[4]

For pycentral, one may specify something like "current, >= 2.4" or even "current, >= 2.2, << 2.5". Packages with private extention modules also can use the value "current_ext", which will will expand to "current", if the package does not have any extensions and can be used with another python default version without a new upload. It's replaced by the version number of the current default version in the Python:Versions substitution variable.

[5]

Public extensions are restricted to the list of Python versions for which the extensions have been compiled, and thus are covered in the section below.