A naming problem was identified that would prevent upgrading and distributing the packages accross multiple distributions. A new naming convention is proposed for discussion, based on the Debian GNU/Linux backport conventions. The 1.2.40 release of the packaging-farm was published with a documentation for the packaging-farm.conf configuration file and support for Ubunutu natty.
package names and multiple distributions
As it is, packaging-farm will create packages with names that are not dependent on the distribution. It creates a problem that is immediately visible when trying to build a repository that includes all the packages: the pool can only accomodate for a single package with a given name.
When building packages for multiple distributions, packaging-farm should rename the packages built so that they carry the distribution name. The names must compare in the same order as the distribution chronology. For instance, foo-1.2-1 would be renamed into foo-1.2-1~3wheezy. Such a behavior is intrusive and may not be appropriate if building for a single distribution. The proposed names are as follow, for each of the supported distributions, following the Debian GNU/Linux backport naming scheme, with the distribution name added for clarity:
* Ubuntu ** natty = ~bpo1104natty+1 ** maverick = ~bpo1010maverick+1 ** lucid = ~bpo1004lucid+1 ** etc. * Debian GNU/Linux ** wheezy = ~bpo70wheezy+1 ** squeeze = ~bpo60squeeze+1 ** lenny = ~bpo50lenny+1 ** etch = ~bpo40etch+1 ** etc.
For the record, here is the chat log that led to the proposal above.
(10:29:46 AM) loicd: Hi, I'm facing a problem organizing repositories made with reprepro. A package (the name is xivo) is fit to build for Debian wheezy, unstable & Ubunty maverick and naty, for i386 & amd64, without requiring any change. Because reprepro uses a pool to store the packages, these four packages (for wheezy, unstable, maverick & naty) cannot coexist with the exact same name. I could either a) change the package name by adding the target distribution although these packages are strictly identical, b) make a separate repository for each distribution (which still is useful to aggregate the various architectures). Do you have any advice ? (10:30:43 AM) twb: FWIW I just use apt-ftparchive (10:31:44 AM) twb: http://paste.debian.net/115729/ (10:31:49 AM) twb: Not very nice, but it does the job (10:32:51 AM) loicd: twb: you mean that it is imune to this kind of trouble (the pool) ? (10:34:57 AM) twb: Basically each dir is an apt repo (10:35:26 AM) twb: So instead of dists/ and pool/x/xivo, you'd just have xivo/ as a repo, and you can point whatever distros you want at it (10:41:02 AM) loicd: twb: it looks like what I'm after. One drawback of having the same package name for two distro is upgrade. The following use case is broken : a) install wheezy + xivo, b) upgrade to unstable => xivo does not get upgraded. Although the source packages are the same, the resulting .deb are different because the context is different. The ideal situation would be to be able to create .deb with a name that contains the distro in such a way that it upgrades. (10:41:54 AM) ***OdyX does -1 upload to Debian, then -1~n0 to natty, -1~m0 to maverick -1~l0 to lucid. (10:44:39 AM) loicd: OdyX: smart :-) (10:45:24 AM) OdyX: loicd: more or less ~. It means you have to rebuild+upload each time (only source if you upload to PPAs) (10:46:55 AM) loicd: twb: I actually have the same problem with squeeze + wheezy. (10:47:09 AM) twb: loicd: squeeze is BPO's problem, not yours :P (10:47:20 AM) loicd: -1~s0 = squeeze, -1~w0 (10:47:24 AM) loicd: that works too (11:10:37 AM) loicd: twb: actually, it is. The vendor of the software needs to provide squeeze backports to its clients. This happens usually in a rush and before it becomes available to bpo. It actually is a concern shared by most software vendors : the delays between meeting the client request and meeting the debian requirements do not always match. (11:41:55 AM) loicd: I think I will go for names that use the version number to ensure it compares as expected foo-1.2-1 => foo-1.2-1~7.0wheezy ( wheezy = ~7.0wheezy, squeeze = ~6.0squeeze, lenny = ~5.0lenny etc.). What do you think ? (11:42:31 AM) daemonkeeper: why use the codename at all then? (11:43:45 AM) twb: If the backport doesn't involve a source package change, I don't really see why it's necessary to fuck about at all (11:45:14 AM) pabs: loicd: don't you want dch --bpo? (11:45:53 AM) loicd: daemonkeeper: for readability. I agree it's redundant. (11:46:06 AM) loicd: but it will help the customer support, I suppose (11:48:37 AM) loicd: twb: because the dependencies change and the binary packages being built are actually different. I agree that it would be completly pointless if they were python sources for instance. But in the case of compiled binaries, if the distribution is being upgraded the binary package must also be upgraded. If it had the same name it would not upgrade and be broken. (11:49:05 AM) twb: loicd: ah right (12:07:54 PM) loicd: pabs: thanks for pointing out bpo. I will follow the bpo naming scheme so that it's compatible with bpo. And it's a lot better to adopt an existing convention ;-) http://backports.debian.org/Contribute/#index5h3 :-)
natty support and documentation
Support for natty was added by creating a virtual machine using the server distribution and creating i386 and x86_64 chroots with:
pbuilder --create --architecture i386 --basetgz /var/cache/pbuilder/natty-i386.tgz pbuilder --create --architecture amd64 --basetgz /var/cache/pbuilder/natty-amd64.tgz
and untaring them into the packaging-farm rsync repository for i386 and x86_64.
The content of the packaging-farm.conf was documented as a manual page:
NAME
packaging-farm.conf - define the default environment to build a package
DESCRIPTION
The packaging-farm command uses makefiles to run. Each package defines
variables to influence the way it is built, as described in the packag-
ing-farm(1) manual page. The packaging-farm.conf file is included
before the package specific makefile and can be used to define vari-
ables common to all packages. The packaging-farm.conf is a makefile and
its syntax depends on GNU make
There are no default values. If a variable is commented out, it will be
set to a value that triggers an error. The default packaging-farm.conf
file distributed contains sensible defaults, there is no fallback. If
they are removed, it is presumably because of a typo or an error and it
needs to trigger an error instead of silently falling back on an unex-
pected default.
VARIABLES
For each variable listed below, the values that can be used are listed
in the comments of the distributed packaging-farm.conf file. It is
important to check that a variable does not contain a trailing space.
Otherwise it will be used and lead to unexpected results.
DISTRIBUTIONS
is the white space separated list of distributions for which a
package must be built.
ARCHITECTURES
is the white space separated list of architectures for which a
package must be built. For each DISTRIBUTION, the package will
be built for all ARCHITECTURES. For instance if DISTRIBUTIONS
contains squeeze and maverick and ARCHITECTURES contains i386
and x86_64, the package will be built for squeeze+i386,
squeeze+x86_64, maverick+i386 and maverick+x86_64. There is no
way to ask that only i386 is built for squeeze and only x86_64
is built for maverick.
PACKAGES
The list of the packages for which dependencies must be calcu-
lated with
packaging-farm depends
so that package1 is always built before package2 if package2
depends on package1. It is recommended to not manually set this
variable. Instead, the ${CACHEDIR}/build (which is usually
/var/cache/packaging-farm/build ) should only contain directo-
ries matching the names of the packages to work on. And the
variable can be set with:
PACKAGES=$(shell ls ${CACHEDIR}/build)
AUFS The AUFS File System (http://aufs.sourceforge.net/) is used by
the packaging farm to create a local copy of a chroot with a
minimum overhead. If not available (value set to no) a rsync
based copy will be used instead. This uses a lot more disk
space. On Debian GNU/Linux, aufs is usually available by
default and can be loaded with modprobe aufs
CHROOT_WANTED
The list of chroot that must be loaded in order to build the
packages. They are retrieved from rsync://packaging-
farm.dachary.org/packaging-farm via the command:
packaging-farm chroot-master-sync
For instance
CHROOT_WANTED=debian/i386/squeeze debian/x86_64/squeeze
will retrieve squeeze for the i386 and x86_64 architectures. The
chroot repository can be browsed via the web at http://packag-
ing-farm.dachary.org/chroot/. The available chroot must cover
all combinations of the DISTRIBUTIONS and ARCHITECTURES vari-
ables, both in the packaging-farm.conf file and in the invivid-
ual makefiles of each package. For instance, if a packaging-
farm.conf has DISTRIBUTIONS=squeeze ARCHITECTURES=i386 only but
a package overrides this with DISTRIBUTIONS=wheezy packaging-
farm will need debian/i386/wheezy to build it. If the distribu-
tion is omitted, all chroot for a given architecture will be
retrieved.
packaging-farm CHROOT_WANTED=debian/i386 chroot-master-sync
A cron job will synchronize on a daily basis. There is no need
to manually run the synchronization, unless a specific update is
needed immediately. After being copied, the chroot will get the
/etc/resolv.conf file from the host.
CHROOT_ROOT
Directory in which copies of the CHROOT_WANTED are kept. As of
May, 2011, if CHROOT_WANTED=debian it will use 5GB.
SUBMIT The suffix of the submit script that will be used when running:
packaging-farm submit
The action is carried out by as script named after this vari-
able.
/var/lib/packaging-farm/submit/submit-${SUBMIT}.sh
TROUBLESHOOTING
check for trailing spaces in variable values
DISTRIBUTIONS=A B with a trailing space is different from DIS-
TRIBUTIONS=A B withing a trailing space and will lead to unex-
pected behavior.
SEE ALSO
submit-xivo.sh(1), packaging-farm(1)
AUTHORS
Loic Dachary <loic@dachary.org>
local packaging-farm.conf(5)
The default packaging-farm.conf file was self documented and set with default
values so that the user only need to copy/paste from it without a need to actually
understand the details.
########################################################################
#
# White space separated list of the distributions for which the packages
# must be built.
# There must be NO SPACE at the end of the line
#
# DISTRIBUTIONS=lenny squeeze wheezy unstable maverick natty
#
DISTRIBUTIONS=unstable
#
########################################################################
#
# White space separated list of the distributions for which the packages
# must be built.
# There must be NO SPACE at the end of the line
#
# ARCHITECTURES=i386 x86_64
#
ARCHITECTURES=x86_64
#
########################################################################
#
# The list of the packages for which dependencies must be calculated
# with
#
# packaging-farm depends
#
# so that package1 is always built before package2 if package2 depends
# on package1. It is recommended to not manually set this variable.
# Instead, the ${CACHEDIR}/build (which is usually
# /var/cache/packaging-farm/build ) should only contain directories
# matching the names of the packages to work on.
# There must be NO SPACE at the end of the line
#
# PACKAGES=package1 package2
#
PACKAGES=$(shell ls ${CACHEDIR}/build)
#
########################################################################
#
# The AUFS File System (http://aufs.sourceforge.net/) is used by the
# packaging farm to create a local copy of a chroot with a minimum
# overhead. If not available (value set to no) a rsync based copy will
# be used instead. This uses a lot more disk space.
# On Debian GNU/Linux, aufs is usually available by default and
# can be loaded with modprobe aufs
#
#AUFS=yes
AUFS=no
#
########################################################################
#
# The list of chroot that must be loaded in order to build the
# packages. They are retrieved from rsync://packaging-farm.dachary.org/packaging-farm
# For instance:
# CHROOT_WANTED=debian/i386/squeeze debian/x86_64/squeeze
# will retrieve squeeze for the i386 and x86_64 architectures.
#
# The values are:
#
# debian/i386/lenny
# debian/i386/maverick
# debian/i386/natty
# debian/i386/squeeze
# debian/i386/unstable
# debian/i386/wheezy
# debian/x86_64/lenny
# debian/x86_64/maverick
# debian/x86_64/natty
# debian/x86_64/squeeze
# debian/x86_64/unstable
# debian/x86_64/wheezy
#
# There must be NO SPACE at the end of the line
#
# Retreive all of debian
# CHROOT_WANTED=debian
# Retreive all of i386 debian
# CHROOT_WANTED=debian/i386
# Retreive all of x86_64 debian
# CHROOT_WANTED=debian/x86_64
#
CHROOT_WANTED=debian/x86_64/unstable
#
########################################################################
#
# Directory in which copies of the CHROOT_WANTED
# are kept.
#
CHROOT_ROOT=/chroot
#
########################################################################
#
# The suffix of the submit script that will be used
# when running:
#
# packaging-farm submit
#
######
# Read man submit-xivo.sh(1) for more information
# SUBMIT=xivo
#####
# Not yet documented.
# SUBMIT=pokersource
#
SUBMIT=none
#
########################################################################
#
# If set, the debsign(1) command will be called to
# sign the package, if it is built successfully.
#
# debsign ${SIGN_debian} *.changes
#
# It should specify the key with which the package should be
# signed. The meta-packages will use SIGN_WITH to sign
# the packages.
#
#SIGN_debian=-kABCDEFGH
SIGN_debian=
#
########################################################################
#
# Sign the packages of the repository created for a meta-package
# See reprepro(1) for more information.
#
#SIGN_WITH=SignWith: yes
SIGN_WITH=
#