This talk is aimed at people maintaining .deb packages, for Debian, distributions derived from Debian, or for themselves. It is intended as a 45 minute long talk.

In early summer of 2005, I wrote piuparts, a Debian package installation, upgrading, and removal testing tool. Since then, I've used it to test packages in Debian and reporting any bugs I find.

As usual, when a new kind of automatic testing is used for the first time in a project, many trivial and some not so trivial problems are found. That has also been the case with piuparts: most problems found with it have been small and relatively insignificant, but there have also been some important problems.

The talk will begin with a tutorial on piuparts with the goal to explain why it is a good idea to test packages with piuparts before uploading, and then how to do it. I will explain the three kinds of tests piuparts runs.

In the first kind of test, piuparts sets up a chroot with the desired version of Debian, installs the package and its dependencies, and then purges anything it installed. It then compares the state of the chroot before the package was installed, and after it was purged. Any files that have been added, removed, or modified are reported as errors (ignoring a number of files that always change in benign ways during dpkg runs).

In the second kind of test, piuparts first installs the previous version of the package into the package by downloading it from the Debian archive, and then upgrades it to the new .deb file a user gives it. This is so that package maintainers can test that a package upgrades cleanly before they upload it.

In the third kind of test, piuparts upgrades the entire chroot twice from one Debian release to another, once with and once without the package being tested (plus any dependencies). It then compares the state of the chroot at the end of each upgrade run; any differences are reported as errors. This is good for testing that a package does not prevent or complicate upgrading a system to the next Debian release.

For example, if the package modifies its conffiles in its postinst scripts, and the version of the package in the next Debian release has a modified default version of the conffile, this can result in dpkg stopping the upgrade to ask the system administrator to choose which version to use. What dpkg does is correct when the system administrator has modified the conffile, but wrong when it is the package itself, or another package altogether, that modified the conffile.

I will give examples of doing all these tests with piuparts, with live demonstrations if possible. I will explain ways of setting things up to create cached versions of chroots to speed things up, and ways of keeping those chroots up to date. I will discuss ways of integrating these tests into the package building, testing, and uploading work flow.

After everyone knows what piuparts is and how they can use it themselves I will explain what happens if they do not: I will file bugs on their packages. I will discuss my experiences running piuparts on as many Debian packages as possible. I have been doing that since the summer of 2005, and have filed a number of bugs found in the process.

I will give some statistics about the the bugs I've filed: number of bugs filed, and number of bugs fixed, both as a function of time (to see whether people start running piuparts themselves); the average time of getting them fixed, broken down by severity levels; and other numbers that may be interesting. More importantly, I will discuss common problems and ways of noticing, fixing, and avoiding them.

Please note that this is work in progress: I expect to continue developing piuparts and coming up with new ways to torture packages. This talk proposal is what I could write and talk about immediately, not a projection of what I hope to be able to do in the future. If my talk is accepted, I will cover any further developments between now and the time of giving the talk.

I will possibly also talk about this subject at FOSDEM, in February. Andreas assures me that this won't be a problem, and also, in the time in between, I expect to further develop piuparts, so the talk at Debconf6 in May would be an expanded and updated version, not a mere repeat. If I have to choose, I'd rather give the talk at Debconf6, since the developer audience is greater there.