An OpenStack cluster is deployed using bare metal hardware provisionned from various hosting companies (eNovance, Hetzner etc.). The storage volume location is controlled by shutting down all nova-volume daemons, except the one running on the desired node. Each node (bare metal machine) is defined as an availability zone, which allows the user to target the desired node with the nova boot … –availability_zone=node … command line option.
OpenStack setup
The OpenStack cluster is setup using the Debian GNU/Linux puppet HOWTO. All hosts are named bm0001.the.re, bm0002.the.re, bm0003.the.re etc.
Availability zone
An availability zone is defined by adding the node_availability_zone configuration flag in the /etc/nova/nova.conf file for each node. For instance, the following puppet manifest snippet defines that each node is an availability zone.
$availability_zone = regsubst($::fqdn, '^(bm\d+).*', '\1')
nova_config { 'node_availability_zone': value => $availability_zone }
translates into the following line in /etc/nova/nova.conf for the node bm0001.the.re:
--node_availability_zone=bm0001
Locating the node for a volume
Create the volume
# nova volume-create --display_name volume01 1
Figure out the ID of the volume
# nova volume-list +----+-----------+--------------+------+-------------+-------------+ | ID | Status | Display Name | Size | Volume Type | Attached to | +----+-----------+--------------+------+-------------+-------------+ | 8 | available | volume01 | 1 | None | | +----+-----------+--------------+------+-------------+-------------+
Get the volume provider_location and guess the node from it
# mysql -e "select provider_location from volumes where id = 8" nova +------------------------------------------------------------------+ | provider_location | +------------------------------------------------------------------+ | 192.168.100.1:3260,2 iqn.2010-10.org.openstack:volume-00000008 0 | +------------------------------------------------------------------+
Each node in the cluster has an IP address matching the node name. 192.168.100.1 is bm0001.the.re, 192.168.100.2 is bm0002.the.re etc. Hence the node on which the volume was created in the example above is bm0001.the.re.
SameHostFilter does not work in Essex
The scheduler configuration can be displayed with:
grep scheduler_ /var/log/nova/nova-scheduler.log
immediately after a
/etc/init.d/nova-scheduler restart
otherwise the logs rotatation will move them to another file. For instance:
root@bm0001:~# grep scheduler_ /var/log/nova/nova-scheduler.log 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_json_config_location : from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] volume_scheduler_driver : nova.scheduler.chance.ChanceScheduler from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_default_filters : ['AvailabilityZoneFilter', 'RamFilter', 'ComputeFilter'] from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] compute_scheduler_driver : nova.scheduler.filter_scheduler.FilterScheduler from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_topic : scheduler from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_host_manager : nova.scheduler.host_manager.HostManager from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_manager : nova.scheduler.manager.SchedulerManager from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_driver : nova.scheduler.multi.MultiScheduler from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411 2012-09-13 11:08:14 DEBUG nova.service [-] scheduler_available_filters : ['nova.scheduler.filters.standard_filters'] from (pid=534) wait /usr/lib/python2.7/dist-packages/nova/service.py:411
In the example above, the filters deciding on which node a new resource is located are listed in:
scheduler_default_filters : ['AvailabilityZoneFilter', 'RamFilter', 'ComputeFilter'
The SameHostFilter can be used by adding it to the list, in /etc/nova/nova.conf:
scheduler_default_filters=AvailabilityZoneFilter,RamFilter, SameHostFilter,ComputeFilter]
However, using
nova boot ... --hint 'same_host=[f7f3b33c-e5e7-43b2-b1b6-687dc6d0d8a6]' ...
as instructed in the documentation won’t work because the –debug shows that it translates into
{"os:scheduler_hints": {"same_host": "[f7f3b33c-e5e7-43b2-b1b6-687dc6d0d8a6]"}...
instead of
{"os:scheduler_hints": {"same_host": ["f7f3b33c-e5e7-43b2-b1b6-687dc6d0d8a6"]}...
It is a string and not an array and the corresponding code fails to select the appropriate host. There are no bugs filled against this problem. The same parsing problem happens with the force_hosts hint, which requires admin privileges.
Create a volume and a virtual machine on a designated node
Shutdown the nova-volume daemon on each node except the one that is targeted ( for instance bm0001.the.re ) with :
/etc/init.d/nova-volume stop
and check that only one remains:
# nova-manage service list | grep nova-volume nova-volume bm0001.the.re bm0001 enabled :-) 2012-09-13 13:22:07 nova-volume bm0002.the.re bm0002 enabled XXX 2012-09-13 13:11:41
Create the volume:
nova volume-create --display_name volume01 1
Restart the nova-volume daemon on each node with
/etc/init.d/nova-volume start
and check that they are all back online:
# nova-manage service list | grep nova-volume nova-volume bm0001.the.re bm0001 enabled :-) 2012-09-13 13:24:48 nova-volume bm0002.the.re bm0002 enabled :-) 2012-09-13 13:24:44
Create the virtual machine on the same node using the availability zone defined to include the designated node:
nova boot --image fe44546e-66db-4983-8a49-6bed4afc1301 --flavor 1 --key_name test_keypair --availability_zone=bm0001 instance01 --poll
Check that it shows when listing the instances hosted on a designated node:
# nova list --host bm0001 +--------------------------------------+------------+--------+------------------------+ | ID | Name | Status | Networks | +--------------------------------------+------------+--------+------------------------+ | 2194fc07-9443-4ce8-8ae0-4b9360757d36 | instance01 | ACTIVE | novanetwork=10.145.0.8 | +--------------------------------------+------------+--------+------------------------+
Attach the volume to the server
# nova volume-attach 2194fc07-9443-4ce8-8ae0-4b9360757d36 8 /dev/vda
Check that it has been attached successfully:
# nova volume-list +----+--------+--------------+------+-------------+--------------------------------------+ | ID | Status | Display Name | Size | Volume Type | Attached to | +----+--------+--------------+------+-------------+--------------------------------------+ | 8 | in-use | volume01 | 1 | None | 2194fc07-9443-4ce8-8ae0-4b9360757d36 | +----+--------+--------------+------+-------------+--------------------------------------+
Check that it is accessible from the instance with:
root@bm0001:~# ssh -i test_keypair.pem cirros@10.145.0.8 $ sudo fdisk /dev/vdb Command (m for help): p Disk /dev/vdb: 1073 MB, 1073741824 bytes