Running a virtual machine within a virtual machine (aka nested virtual machines) is supported by OpenStack by adding a stanza such as
<cpu mode='custom' match='exact'>
<model>Nehalem</model>
<feature policy='require' name='vmx'/>
</cpu>
in the libvirt.xml.template file. The nested feature must be enabled on the OpenStack node for intel processors by adding the nested=1 argument when loading the kvm_intel module so that:
# cat /sys/module/kvm_intel/parameters/nested Y
This feature can be used to exercise an automated Openstack deployement on Debian GNU/Linux wheezy with razor within OpenStack itself.
Activate nesting on the node
If the processor is Intel, the kvm_intel must loaded at boot time with the nested option. The list of options from the kvm_intel module can be displayed with:
# modinfo kvm_intel filename: /lib/modules/3.2.0-3-amd64/kernel/arch/x86/kvm/kvm-intel.ko license: GPL author: Qumranet depends: kvm intree: Y vermagic: 3.2.0-3-amd64 SMP mod_unload modversions parm: vpid:bool parm: flexpriority:bool parm: ept:bool parm: unrestricted_guest:bool parm: emulate_invalid_guest_state:bool parm: vmm_exclusive:bool parm: yield_on_hlt:bool parm: fasteoi:bool parm: nested:bool parm: ple_gap:int parm: ple_window:int
And the status of the nested boolean flag can be displayed with:
# cat /sys/module/kvm_intel/parameters/nested N
If the kvm_intel module is not in use because its reference count on the last column is 0:
# lsmod | grep kvm_intel kvm_intel 121968 0 kvm 287662 1 kvm_intel
it can be reloaded with the nested option set to 1:
# rmmod kvm_intel # modprobe kvm_intel nested=1 # cat /sys/module/kvm_intel/parameters/nested Y
For this option to be set at boot time, a modprobe.d(5) file can be created with the following puppet snippet:
file { '/etc/modprobe.d/kvm_intel.conf':
ensure => present,
owner => "root",
group => "root",
mode => 0444,
content => "options kvm_intel nested=1\n",
}
Create cluster wide cpu description
By default OpenStack does not expose features of the host processor to the instance. To ensure that all processors in the OpenStack cluster can support nested virtual machines, the virsh cpu-baseline command can be used as follows:
# nova-manage host list 2>/dev/null | \ sed -e '1d' | cut -f1 -d' ' | while read host ; do \ ssh $host virsh capabilities ; done | \ virsh cpu-baseline /dev/stdin <cpu mode='custom' match='exact'> <model fallback='allow'>Nehalem</model> <vendor>Intel</vendor> <feature policy='require' name='rdtscp'/> <feature policy='require' name='dca'/> <feature policy='require' name='pdcm'/> <feature policy='require' name='xtpr'/> <feature policy='require' name='tm2'/> <feature policy='require' name='est'/> <feature policy='require' name='vmx'/> <feature policy='require' name='ds_cpl'/> <feature policy='require' name='monitor'/> <feature policy='require' name='dtes64'/> <feature policy='require' name='pbe'/> <feature policy='require' name='tm'/> <feature policy='require' name='ht'/> <feature policy='require' name='ss'/> <feature policy='require' name='acpi'/> <feature policy='require' name='ds'/> <feature policy='require' name='vme'/> </cpu>
The virsh capabilities command is run on all hosts in the OpenStack cluster, as returned by nova-manage host list to get the cpu element that describes the processor. The result is streamed to the virsh cpu-baseline command which returns the common denominator. For instance, if one host does not support vmx, it won’t show in the output, even if all the others do. This common subset between all hosts ensures that an instance created on any given host can be migrated to another. The output of virsh cpu-baseline is edited to remove all but the vmx feature which is the only feature of interest:
<cpu mode='custom' match='exact'>
<model fallback='allow'>Nehalem</model>
<feature policy='require' name='vmx'/>
</cpu>
Configure OpenStack to allow nested virtual machines
The libvirt.xml.template file on each OpenStack compute node is used as a template to launch the kvm process. The cpu element including <feature policy=’require’ name=’vmx’/> is added to it, manually. The addition is done in a copy of the file located in /etc/puppet/files/libvirt.xml.template on the puppetmaster. It is then broadcasted to all the node of the cluster using the following puppet manifest snippet:
file { '/etc/nova/libvirt.xml.template':
ensure => present,
owner => "root",
group => "root",
mode => 0444,
replace => true,
source => "puppet:///files/libvirt.xml.template",
}
nova_config { 'libvirt_xml_template': value => '/etc/nova/libvirt.xml.template' }
The first stanza copies the modified libvirt.xml.template to /etc/nova and the second instruct nova-compute to use it instead of the default template provided by the nova-compute package.