Wanna-build states: an explanation

This page tries to explain what every wanna-build state means and what will happen to a package when it's in that state. It's target audience are Debian package maintainers that try to understand why their package has, or has not, been built for a specific architecture. Also, an explanation of the different log results is given.

Finally, a flowchart version of the wanna-build states is available, but do note that it doesn't talk about everything mentioned in this document.

The wanna-build states

For every Debian-supported architecture, there's a wanna-build database installed on auric, with all packages and their current compilation state. There are 7 states: needs-build, building, uploaded, dep-wait, failed, not-for-us, and installed.

Their meaning is as follows:

needs-build

A package marked needs-build has seen an upload of a new version by its maintainer, but for a different architecture than the one this wanna-build database is for; as such, it needs a rebuild. If the state is needs-build, it has not been picked up by an autobuilder yet, but it will be (once one is available at a time the specific package is near the top of the list). People commonly say "a package is queued for rebuild" when they are talking about a package in the "needs-build" state.
It may be interesting to note that the needs-build queue is not a FIFO queue; rather, the ordering used is based on the following criteria:

1. Packages' previous compilation states; packages that have been built previously are given priority over new packages.
2. priorities (packages with required priority are built before packages with extra priority)
3. The section a package is in. This ordering is based on what packages are deemed more important; e.g., section "games" is built after section "base", and section "libs" is built before "devel".
4. an asciibetical ordering on the package name.

Additionally, under certain conditions, it may happen that a buildd will not take packages at the head of the queue; for instance, when a buildd can't find the source of a given package, it will put it back in the queue (where it will then again be put at its previous position, i.e. the head of the queue), but it will ignore the package for a few hours. Another example where this might happen is when an architecture has multiple autobuilders; in that case, the architecture's porters may choose to build larger packages on their faster autobuilders, and leave the smaller ones for the slower machines in the pool. A buildd can theoretically also explicitely request a different section ordering, but that is not usually done.
There could be other situations where the queue order seems to be ignored; but note that they are all exceptions.

building

A package is marked building from the moment an autobuilder picks it from the top of the wanna-build queue until the moment the autobuilder admin replies to the log. As packages are not picked one by one, this means a package can be (and usually is) marked 'building' before the build has actually started; however, as buildd builds packages in its local queue on a FIFO basis, it should not take too long anymore. Also, note that the state of a package is not modified once the build is complete; only when the autobuilder admin comes around to replying to the logs.

uploaded

When a build attempt was successful, a build log is sent to the autobuilder admin and to buildd.debian.org. The autobuilder maintainer will then sign the .changes file which is embedded in the build log, and send it to the autobuilder. In reaction, the autobuilder will upload the package and set its state to uploaded. As such, a package in this state can be found in the incoming queue (somewhere).
An autobuilder will not touch a package anymore once it's state is uploaded, at least not until the next upload or until a porter manually modifies the state of a package.

dep-wait

When a package fails due to missing build-time dependencies, the autobuilder maintainer will send a mail to the autobuilder, instructing it to remove the package sources and to mark the package as dep-wait on the missing build-dependencies. A package in such a state will automatically, without human intervention, be marked needs-build once said dependencies are available.
As such, in two specific cases it may happen that a package is marked dep-wait forever; these are when a typing error happened by specifying the dep-wait dependencies (so that the package is marked dep-wait on a package that does not and will never exist) and when a build-time dependency is declared on a package which is marked not-for-us, or which is in the packages-arch-specific list.
As an example to the latter, consider three packages: a package foo, which exists for i386 only; a package bar, which exists for m68k only (and which roughly performs the same function); and a package baz, which can be built with one of foo or bar. Should the maintainer of the baz package forget to add bar to the Build-Depends, and should he or she add it when it is noticed that baz is dep-waiting on a non-existing foo for m68k, then the dep-wait state for m68k will have to be manually lifted by the m68k porters.

failed

If a build attempt failed, and the autobuilder maintainer decides it is really a failure that should not be retried, a package is marked as failed. A package will not leave this state until a porter decides it should do so, or until a new version is available. However, when a new version of a package is available which was marked as failed in the previous version, the autobuilder will ask its admin whether or not the package should be retried; this is so that packages which will obviously fail again will not waste buildd time. Although failing a package before trying a build is hardly ever the right thing to do, the option is available to the autobuilder admin.
Note that a package will never be marked failed without human intervention.

not-for-us

Certain specific packages are architecture-specific; for instance, "lilo", an i386 boot loader, should not be rebuilt on alpha, m68k, or s390. However, wanna-build does not look at the control file of a package when creating its database; only at the packages' name and section, its urgency, and its priority. As such, by the first upload of an architecture-specific package which should not be built on other architectures, a build attempt is tried none the less (but fails even before the build-time dependencies are downloaded and/or installed)
Since autobuilders should not waste time trying to build packages that aren't required for their architecture, there's need for a way to list packages for which even an attempt to build isn't required. The first solution to this problem was not-for-us; however, as that is difficult to maintain, not-for-us is deprecated nowadays; autobuilder maintainers should use packages-arch-specific instead, which is a list of packages specific to one or more architectures instead of a wanna-build state.
A package in not-for-us or packages-arch-specific will not leave this state automatically; if your package specifically excluded a given architecture in its control file previously, but now includes more architectures, it must be manually requeued.

installed

As the name suggests, a package marked installed is compiled for the architecture the wanna-build database is for. Before Woody was released, a package's state changed from uploaded to installed after the daily katie runs. With the implementation of Accepted-autobuild, however, this is no longer true; nowadays, a package goes from state uploaded to installed when it is accepted into the archive. This means that a package is usually marked installed after 15 minutes, on average.

In addition to these seven states, wanna-build also knows two -removed states, which are really corner cases. These two states are dep-wait-removed and failed-removed. They relate to their respective 'plain' state as follows: when a package in state failed or dep-wait doesn't appear in a new Packages file which is fed to wanna-build – when it appears it has been removed – the information about that package isn't thrown away, as it could be that the package not appearing in the Packages file is just a temporary glitch, or that the package is temporarily removed for some reason (but that it will reappear in the archive, given time). Instead, in such a case, a package is moved to a -removed state, so that the information on why it failed or what it's waiting for can be retained. Should the package reappear in a following Packages file which is fed to wanna-build, it will then be moved from failed-removed back to failed, or from dep-wait-removed back to dep-wait before further processing.

It is not possible to access the wanna-build database directly; this database is installed on ftp-master.debian.org, which is a restricted host, and only buildd machines have an SSH key which allows them to access the wanna-build database of their architecture. This has been the case even before ftp-master was restricted; As the wanna-build does a database-level lock when accessing, even reading, the data, you had to be in the right group (wb-<arch>) to be able to directly access a wanna-build database

That said, you can see what state a package is in by going to the buildd stats page, except if it is in state installed (well, not unless you don't mind spitting through the multi-megabytes "<arch>-all.txt" files...). Alternatively, for some architectures, Ingo Juergensmann has set up some web pages at buildd.net, which is based on parsing of the <arch>-all.txt files, and gives an easier to read (at least for humans) way to see what state a package is in.

The build log results

When a package is built by sbuild (the buildd component which does the actual building), a log with the build result is sent, by mail, to the autobuilder admin and to logs@buildd.debian.org (so that it can end up at http://buildd.debian.org). The build log result can be one successful, failed, given-back, or skipped. Note that, at the buildd log overview page, the prefix maybe- is added, because among other things, the fact that a build can be marked failed there for things that aren't really a failure has caused confusion in the past.

The meaning of the log results is as follows:

successful

The build was successful. When the autobuilder maintainer receives this log, he will extract the embedded .changes file, sign it, and send it back to the autobuilder, which will cause the package to be uploaded.

failed

The build failed. As there can be a number of reasons why a build fails, enumerating all them would be tedious, so no attempt is done here. If a package of yours is marked (maybe-)failed, you will want to read the above, and check its current wanna-build state.

given-back

The build failed due to a temporary problem with the autobuilder; examples include network problems, the unavailability of the packages' source with the current sources.list, low disk space, and others.
A package which is given-back is marked as needs-build again; as such, it will be automatically picked up by a different autobuilder once one is ready.

skipped

In the time between the package was picked by the/an autobuilder and marked building, a new version for this package was uploaded, or a porter manually modified the wanna-build state for another reason. When that is done, a mail is sent to the autobuilder, which will mark the package as not to be built; sbuild sees this, and will skip the build (although a build log with this result is sent, describing the fact that this happened).

The graphical version

Because, as we say in Dutch, "a picture says more than a thousand words", I've provided a flowchart-version of this procedure. It's been a while since I learned to use flowcharts, so I might've made a mistake in some of the symbols I've used; if I did, feedback is welcome.