packaging infrastructure for XiVO

A new hardware dedicated to XiVO packaging was installed. The former virtual machine was migrated there. A first attempt to actually use the packaging-farm led to version 1.2.36 of packaging-farm with improved documentation and some bug fixes.

Trying out packaging-farm

Nicolas Hicher and Loic Dachary worked together on building the skaro repository available at:

deb skaro-squeeze main
deb-src skaro-squeeze main

The goal was to demonstrate the use of the packaging-farm and discover show stoppers. A few documentation shortcomings led to modifications of the manual page for packaging-farm. A problem with the handling of the dependencies was identified and fixed. The 1.2.36 release was published with the corresponding changes.
Nicolas Hicher feels confortable to give a try to the packaging-farm. That will help uncover real world usage issues.

VM host installation

A hardware (500GB disk, 8GB RAM, Intel(R) Xeon(R) CPU X3430 @ 2.40GHz) was installed at to be dedicated to XiVO packaging. It was configured to host kvm based virtual machines using ganeti.To better track the changes, a git repository was created in /etc and used to archive the configuration changes described below.


In squeeze all the software required to run ganeti2 are standard. A year ago it was significantly more difficult as it required the compilation of the DRBD from sources, among other things.
The second partition of the disk was set for LVM:

root@host01:/srv/ganeti# pvs
  PV         VG   Fmt  Attr PSize   PFree
  /dev/sda2  all  lvm2 a-   442.38g 332.38g

and that was bound to the all volume group:

root@host01:/srv/ganeti# vgs
  VG   #PV #LV #SN Attr   VSize   VFree
  all    1   3   0 wz--n- 442.38g 332.38g

The default DRBD configuration was removed so that ganeti can handle it on its own. It is not used at the moment but will come handy when another hardware is added.

mv /etc/drbd.conf /etc/drbd.conf.old
touch /etc/drbd.conf
/etc/init.d/drbd reload


A bridge and a fake lo:1 local interface were created in /etc/network/interfaces:

auto lo:1
iface lo:1 inet static
	up ip route add blackhole

auto br0
iface br0 inet static
        bridge_ports none
        bridge_stp off
	bridge_maxwait 5
        up ip route add dev br0
        up ip route add dev br0


Ganeti hosts and virtual machines names are DNS entries. It would be possible to use a public
DNS to create them. A local DNS configuration is created locally instead, to
reduce the propagation delays, ensure it can always be reached and isolate the maintainance
of the name space into a private network. The top level domain farm was created
and divided into host??.farm ( bound to ) for all the ganeti hosts
and ( bound to ) for all the virtual machines.


Do not listen or answer on public IPs.

allow-recursion {;; };
listen-on {;; };


zone "" {
    type master;
    file "/etc/bind/db.10.10";

zone "farm" {
        type master;
        file "/etc/bind/";


; -*- mode: zone; -*-
; BIND reverse data file for broadcast zone
$TTL    604800
@       IN      SOA     localhost. root.localhost. (
        2011031200      ; serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
@       IN      NS      localhost.

;; perl -n -e 'print if(s/(\S+)\s+IN\s+A\s+10\.10\.(\d+)\.(\d+)/\3.\2\tIN\tPTR\t\' <
1.60    IN      PTR
2.60    IN      PTR


; -*- mode: zone; -*-
$TTL 1h ; default Time-to-Live. defines the duration that the record may be cached (24h)

$ORIGIN farm.
@                       IN      SOA             ns hostmaster (
                        2011031200      ; serial
                                1h              ; refresh - time when the slave will try to refresh the zone from the master (8h)
                                30m             ; update retry - time between retries if the slave (secondary) (2h)
                                                ; fails to contact the master when refresh (above) has expired.
                                2w              ; expiry - time slave will continue to supply authoritative data for the zone (1w)
                                                ; after the master died
                                30m)            ; minimum - time a NAME ERROR = NXDOMAIN record is cached (24h)

                IN      NS      ns

ns              IN      A

gnt-cluster     IN      A
host01          IN      A

dev.dachary.vm  IN      A
skaro.xivo.vm   IN      A

A DHCP server is setup for each VM to use.




A MAC address is generated with the following oneliner:

# MACADDR="52:54:$(dd if=/dev/urandom count=1 2>/dev/null | md5sum | sed 's/^\(..\)\(..\)\(..\)\(..\).*$/\1:\2:\3:\4/')"; echo $MACADDR

and used to define the entries

group {
  option domain-name "farm";
  option routers;
  option domain-name-servers;
  subnet netmask {
    host {
        hardware ethernet 52:54:bf:3d:d4:ef;
        option subnet-mask;
    host {
        hardware ethernet 52:54:fb:77:1e:82;
        option subnet-mask;

Cluster and instance creation

The hostname of the machine is When creating a cluster, the hostname is used to create the first node of the cluster and make it the master. Its IP set in /etc/hosts to It is permanently set on eth0 in the /etc/network/interface as follows:

up ip addr add broadcast dev eth0

The prerequisite to run gnt-cluster init were met:

  • is not
  • IP is not bound to any interface

The cluster was created with:

gnt-cluster init --no-etc-hosts --enabled-hypervisors=kvm --nic-parameters link=br0 --master-netdev br0 --vg-name all

After downloading the installation CD of Debian GNU/Linux squeeze, the instance was created:

gnt-instance add -d -t plain -s 30G -B memory=512M,vcpus=1 -H kvm:boot_order=cdrom,cdrom_image_path=/srv/ganeti/debian-6.0.1a-amd64-CD-1.iso,vnc_bind_address= -n -o debootstrap+default --net 0:mac=52:54:fb:77:1e:82

Copying the development VM

Instead of proceeding with a regular installation, a copy of the VM at was made. It was stored in a 13GB bzip2 file:

root@host01:/srv/ganeti# ls -lh
-rw-r--r-- 1 root root  13G Mar 31 15:39 xivo.dachary.vm.gnt.bz2

The newly created instance was shutdown

gnt-instance shutdown --timeout=1

The location of its disk retrieved:

root@host01:/srv/ganeti# gnt-instance info | grep /dev/all
      on primary:  /dev/all/97500dbf-ca9e-49b8-a4c9-4d96e5152916.disk0 (254:2)

and the backup unpacked into it:

bunzip2 < xivo.dachary.vm.gnt.bz2 > /dev/all/97500dbf-ca9e-49b8-a4c9-4d96e5152916.disk0

It ended with an error:

bunzip2: I/O or other error, bailing out.  Possible reason follows.
bunzip2: No space left on device
        Input file = (stdin), output file = (stdout)

which was ignored. To make sure the restoration was sane, the partition was checked:

root@montreal:/srv/ganeti# kpartx -av /dev/all/97500dbf-ca9e-49b8-a4c9-4d96e5152916.disk0
add map all-97500dbf--ca9e--49b8--a4c9--4d96e5152916.disk0p1 (254:3): 0 61865984 linear /dev/all/97500dbf-ca9e-49b8-a4c9-4d96e5152916.disk0 2048
add map all-97500dbf--ca9e--49b8--a4c9--4d96e5152916.disk0p2 (254:4): 0 1046528 linear /dev/all/97500dbf-ca9e-49b8-a4c9-4d96e5152916.disk0 61868032
root@montreal:/srv/ganeti# fsck -f /dev/mapper/all-97500dbf--ca9e--49b8--a4c9--4d96e5152916.disk0p1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/mapper/all-97500dbf--ca9e--49b8--a4c9--4d96e5152916.disk0p1: 260424/1925760 files (3.8% non-contiguous), 4826992/7733248 blocks


A shorewall based firewall was configured based on the two interface example found at /usr/share/doc/shorewall/examples/two-interfaces/ and replacing the eth1 interface with br0. The idea is that the bridge contains virtual machines that are treated as if they were machines in the LAN for which this example was designed.

reverse proxy

A nginx based reverse proxy was installed on the to demultiplex the incoming http connections. The first configuration was added in /etc/nginx/sites-available/ to get to the virtual machine that will eventually replace

server {
 listen 80;
 access_log  /var/log/nginx/;

 location / {

and the HTTP port was accepted in the firewall in /etc/shorewall/rules

DNAT           net             loc:$VM_SKARO_XIVO:22 tcp             22002           -       $IP_FAILOVER