What does it take for Bio++ to enter the official repositories of Debian GNU/Linux ? Julien Dutheil and Loïc Dachary worked together to polish the packages and contribute them to Debian. This page is a summary of the topics that were discussed, by mail and on IRC.
Request For Packaging
The first step is to ask for a Debian Developer to package the software. This is called a RFP as in Request For Packaging and it is done by filling a bug report against the wnpp pseudo package. Since Debian packages for Bio++ are already provided the task of the Debian Developer is facilitated.Even when Debian packages are already available, the odds of seducing a Debian Developer into taking responsibility for the package are small. The quality standards for a package entering Debian are higher than what is implemented by the amateur packager. However, meeting these standards is not a difficult task when using the proper tools.
Bio++ is made of height packages. In theory it would be necessary to create one RFP per package. When a package is uploaded and shows in the new packages queue, a human being will review its content and read the corresponding RFP. As long as all packages are uploaded at the same time, it is likely that the same person will attend all of them at once and won’t object to the single RFP.
The link between a package and the corresponding RFP is done by the following line in debian/changelog
* RFP: Bio++ -- The Bio++ bioinformatics libraries. (Closes: #616373).
Resolving packages dependencies
Using pbuilder ensures that the package can be built on a virgin Debian GNU/Linux installation.
When trying a build on a machine used for other purposes, it is common to forget that it can’t be built unless a specific library is already installed.
apt-get install pbuilder pbuilder --create
Will create a virgin GNU/Linux and archive it in a tar.gz (in /var/cache/pbuilder). It can then be used to build the package (for instance libbpp-core) as follows:
pbuilder --build libbpp-core_2.0.1.dsc
will copy the package in a chroot extracted from the archive created by pbuilder and try to build it. When finished the result can be found in /var/cache/pbuilder/result.
Packages interdependencies
What should be done for libbpp-seq to build ? It relies on libpp-core which is not standard in Debian and should retrieve it from a previous build. Assuming a web server is installed locally, a Debian repository can be built as follows:
ln -s /var/cache/pbuilder/result /var/www/result
and a reference to this repository can be added to the pbuilder chroot as follows:
pbuilder --login --save-after-login echo deb http://localhost/result/ ./ >> /etc/apt/sources.list Control-D
From now on, building a package will lookup this repository in addition to the standard ones. Whenever a new package is added to this repository, as a result of a successfull call to pbuilder, the pbuilder chroot must be updated as follows:
( cd /var/www/result ; dpkg-scanpackages . /dev/null | gzip > Packages.gz ; dpkg-scansources . /dev/null | gzip > Sources.gz ) pbuilder --update
Julien Dutheil used this technique to ensure that the Build-Depends of each package was accurate.
Packaging farm
In order to review the packages, Loic Dachary created a dedicated virtual machine and installed packaging-farm on it. It essentially does the same as the above with less manual tweaking but the learning curve is steeper. The source packages were downloaded into /var/cache/packaging-farm/sources and extracted as follows
for i in libbpp-core libbpp-seq libbpp-raa libbpp-phyl libbpp-popgen libbpp-qt bppsuite bppphyview ; do mkdir ; mv *.{dsc,gz} ; done
for i in libbpp-core libbpp-seq libbpp-raa libbpp-phyl libbpp-popgen libbpp-qt bppsuite bppphyview ; do ( cd ; dpkg-source -x *.dsc ) ; done
For each of them a Makefile was created with the form:
PACKAGE=libbpp-core DISTRIBUTIONS=unstable ARCHITECTURES=i386 x86_64 LIBDIR ?= /usr/lib/packaging-farm include /Makefile
Using the information from the debian/control files, the dependencies between each package was calculated with:
packaging-farm depends
and each package was built, both for i386 and amd64, with
packaging-farm libbpp-seq # which built libbpp-core first because it depends on it packaging-farm bppsuite etc.
The generated packages have been aggretated in a repository that can be used on a Debian GNU/Linux unstable by adding to /etc/apt/sources.list
deb http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main deb-src http://biopp.dachary.org/packaging-farm/bpp/gnulinux/debian unstable main
Which allows to install bppsuite with a single apt-get install bppsuite
apt-get install bppsuite Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libbpp-core libbpp-core-dev libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev The following NEW packages will be installed: bppsuite libbpp-core libbpp-core-dev libbpp-phyl libbpp-phyl-dev libbpp-seq libbpp-seq-dev 7 newly installed, 0 to remove Need to get 10.5MB of archives. After this operation, 43.6MB of additional disk space will be used. Do you want to continue [Y/n]? y Get:1 http://biopp.dachary.org unstable/main libbpp-core 2.0.1 [589kB] Get:2 http://biopp.dachary.org unstable/main libbpp-core-dev 2.0.1 [1,310kB] Get:3 http://biopp.dachary.org unstable/main libbpp-seq-dev 2.0.0-1 [1,227kB] Get:4 http://biopp.dachary.org unstable/main libbpp-seq 2.0.0-1 [540kB] Get:5 http://biopp.dachary.org unstable/main libbpp-phyl-dev 2.0.1 [4,660kB] Get:6 http://biopp.dachary.org unstable/main libbpp-phyl 2.0.1 [1,779kB] Get:7 http://biopp.dachary.org unstable/main bppsuite 0.6.0-2 [410kB] Fetched 10.5MB in 18s (575kB/s) Selecting previously deselected package libbpp-core. (Reading database ... 330640 files and directories currently installed.) Unpacking libbpp-core (from .../libbpp-core_2.0.1_amd64.deb) ... Selecting previously deselected package libbpp-core-dev. Unpacking libbpp-core-dev (from .../libbpp-core-dev_2.0.1_amd64.deb) ... Preparing to replace libbpp-seq-dev 1.7.1 (using .../libbpp-seq-dev_2.0.0-1_amd64.deb) ... Unpacking replacement libbpp-seq-dev ... Preparing to replace libbpp-seq 1.7.1 (using .../libbpp-seq_2.0.0-1_amd64.deb) ... Unpacking replacement libbpp-seq ... Preparing to replace libbpp-phyl-dev 1.9.1 (using .../libbpp-phyl-dev_2.0.1_amd64.deb) ... Unpacking replacement libbpp-phyl-dev ... Preparing to replace libbpp-phyl 1.9.1 (using .../libbpp-phyl_2.0.1_amd64.deb) ... Unpacking replacement libbpp-phyl ... Selecting previously deselected package bppsuite. Unpacking bppsuite (from .../bppsuite_0.6.0-2_amd64.deb) ... Processing triggers for install-info ... install-info: warning: no info dir entry in `/usr/share/info/ispell.info.gz' install-info: warning: no info dir entry in `/usr/share/info/bppsuite.info.gz' Setting up libbpp-core (2.0.1) ... Setting up libbpp-core-dev (2.0.1) ... Setting up libbpp-seq (2.0.0-1) ... Setting up libbpp-seq-dev (2.0.0-1) ... Setting up libbpp-phyl (2.0.1) ... Setting up libbpp-phyl-dev (2.0.1) ... Setting up bppsuite (0.6.0-2) ...
unstable and testing
When a new package enters Debian, it is uploaded in the unstable repository. The software it contains is expected to work but the package itself may be partially broken. It is supposed to be fixed as soon as possible. When and if a package builds successfully and no bug is filled against it during a few days, it is automatically migrated to testing. For instance openscenegraph shows that the Testing column is 2.8.3-6 meaning it migrated from the Unstable column. When a stable version of Debian is published, which takes more than a year, it is made from the content of Testing.
Versioning libraries
A library binary package must contain a version number that indicates the revision of the ABI. The general idea is that any package that depends on it must keep running as long as the ABI is stable. For instance, package simgear depends on libopenscenegraph65. When openscenegraph which depends on libopenscenegraph71 is installed, both simgear and openscenegraph can run although they depend on different versions of the ABI. If the libraries had the same name, they could not co-exist.
When a package is in a state of flux, with frequent minor versions being uploaded to unstable, this convention may be relaxed. Users of the unstable repository have a higher tolerance to this kind of policy violation. It is also useful to avoid the proliferation of libraries.
Since Bio++ use CMake, the library version will be created from stanzas such as
SET(BPPQT_VERSION_CURRENT "2") SET(BPPQT_VERSION_REVISION "0") SET(BPPQT_VERSION_AGE "0")
which will lead to the file name
/usr/lib/libbpp-qt.so.2.0.0
The Debian GNU/Linux policy imposes that the package name containing a shared library is related to the shared library version according to the following:
objdump -p /usr/lib/libbpp-core.so.2.0.0 | sed -n -e's/^[[:space:]]*SONAME[[:space:]]*//p' | sed -e's/([0-9]).so./-/; s/.so.//'
which suggests the following package name:
libbpp-core2
When ”” is added to the Depends: line of a library, the dpkg-shlibdepends will automatically figure out the libraries packages that need to be added to the Depends line. For instance in libbpp-seq it will create the following debian/libbpp-seq9.substvars file:
shlibs:Depends=libbpp-core2, libc6 (>= 2.1.3), libgcc1 (>= 1:4.1.1), libstdc++6 (>= 4.5)
Note that there is no version constraint on libbpp-core2. Such a constraint could be useful if a version of libbpp-core2 has a known bug and libbpp-seq requires a higher version. In this case the #MINVERS# tag can be used as explained in ImprovedDpkgShlibdeps
Legal
All Bio++ software is released under CeCILL version 2. Although it is not among the list of licenses supported by Debian GNU/Linux is has been discussed for a long time and over a dozen packages have already been accepted. Since CeCILL v2 can be replaced transparently by the GNU GPL according to the section 5.3.4 COMPATIBILITY WITH THE GNU GPL, there is little doubt that it will be refused.
The debian/copyright files for each Bio++ packages have been updated to list the licenses and of all third party authors.
Further work
- Close the RFP
- Watch the NEW queue
This was really a nice adventure! I’m sure this will be most useful for anybody like me with little knowledge in debian packaging.
for (int i = 0; i < 1000; ++i) cout << "thanks a lot!!!" << endl;