The osm.fsffrance.org was migrated to the tetaneutral datacenter. It was reconfigured to benefit from a 250GB SSD. The IPv4 assigned to it goes from OVH to tetaneutral (and back) using an IPv6 tunnel.
SSD and ganeti
The version of ganeti on which the FSF France cluster hosting osm.fsffrance.org runs does not allow to select a volume group when adding a disk to an instance. The SSD (available as /dev/sdb1) was added to the volume group used by ganeti but the volume group was forbidden to allocate any space on it:
pvchange --allocatable n /dev/sdb1
When the time came to add a disk to osm.fsffrance.org, the SSD was made allocatable, and allocation on the SATA disk (available as /dev/sda1) was forbidden:
pvchange --allocatable y /dev/sdb1 pvchange --allocatable n /dev/sda1
The disk is then added added using:
gnt-instance modify --disk add:size=250G osm.fsffrance.vm.gnt
And then the SATA is made to be the default disk on which to allocate space again:
pvchange --allocatable n /dev/sdb1 pvchange --allocatable y /dev/sda1
.
IPv6 tunnel
The IPv6 bandwidth for tetaneutral does not cost. Instead of maintaining IPv4 tunnels between the nodes of the ganeti cluster located at tetaneutral and OVH, they were converted into IPv6 tunnels by adding:
proto udp6
to each end of the openvpn tunnel. Assigning IPv6 to the ganeti host themselves required different configuration on OVH and tetaneutral.
Public IP on a provider served by a VM on another
The osm.fsffrance.org public IP is assigned to a ganeti node located at OVH. However, the virtual machine is running on a node located at tetaneutral. The incoming traffic is routed to the VM using the following shorewall lines on the OVH ganeti node in the file /etc/shorewall/nat:
91.121.33.240 eth0:240 10.10.1.66 Yes
Where 91.121.33.240 is the osm.fsffrance.org IP and 10.10.1.66 is the private IP assigned to the virtual machine located at tetaneutral. The virtual machine receives the bytes which are routed to it using the IPv6 tunnel mentionned above. However, when the virtual machine sends a reply, the tetaneutral host sends it using the default route of the tetaneutral node. As a consequence, the OVH node never sees the packet and is unable the NAT it properly. On the tetaneutral node, a route is defined by which all outgoing traffic of 10.10.1.66 goes to the tunnel connected to the OVH host.
ip route add default via 10.1.6.9 dev tun6 table 6 ip rule delete from 10.10.1.66 table 6 > /dev/null || true ip rule add from 10.10.1.66 table 6
The configuration is documented in the
route all VM network traffic to a designated host and IP on the cluster HOWTO.