Public OpenStack providers usable within the hour

The OpenStack marketplace provides a list of OpenStack public clouds, a few of which enable the user to launch an instance at most one hour after registration.

Enter Cloud Suite has a 2GB RAM, 2 CPU, 40GB Disk instance for 0.06 euros / hour (~40 euros per month) and there is no plan to provide a flavor with only 1 CPU instead of 2 CPU. The nova, cinder and neutron API are available.

HP Helion Public Cloud has a 2GB RAM, 2 CPU, 10GB Disk instance for 0.05 euros / hour (0.06 USD / hour) (~40 euros per month).

OVH has a 2GB RAM, 1 CPU, 10GB Disk instance for 0.008 euros / hour (~3 euros per month). The nova API is available, not cinder nor neutron.

Rackspace has a 2GB RAM, 1 CPU, 10GB DIsk instance for ~40 euros per month (plus ~50 euros / month service fee, regardless of the number of instances). The nova and cinder API are available, not neutron.

DataCentred has 2GB RAM, 1CPU, 40GB Disk instance for ~40 euros per month. The nova, cinder and neutron API are available (but the router quota are set to zero by default). There are 2GB RAM, 1CPU, 40GB Disk AARCH64 instances for ~80 euros per month.

Cloudwatt has no 2GB RAM instance but a 3.75GB RAM, 1CPU, 50GB Disk instance for ~35 euros per month which makes it less expensive than all but OVH. The nova, cinder and neutron API are available.

Setting a custom name server on an OpenStack instance

In an OpenStack tenant that is not allowed to create a network with neutron net-create, the name server can be set via cloudinit. The resolv-conf module although documented in the examples is not always available. It can be worked around with

#cloud-config
bootcmd:
 - echo nameserver 4.4.4.4 | tee /etc/resolvconf/resolv.conf.d/head
 - resolvconf -u

for Ubuntu or

#cloud-config
bootcmd:
 - echo nameserver 4.4.4.4 | tee /etc/resolv.conf
 - sed -ie 's/PEERDNS="yes"/PEERDNS="no"/' /etc/sysconfig/network-scripts/ifcfg-eth0

for CentOS.

OpenStack instance name based on its IP address

A DNS has a set of pre-defined names such as:

...
the-re018 10.0.3.18
the-re019 10.0.3.19
...

If nova fixed-ip-reserve is denied by the OpenStack policy and neutron net-create is not available to create a network with the 10.0.3.0/24 subnet that is exclusive to the OpenStack tenant, the naming of the instance must be done after openstack server create completes.
A cloudinit user-data file is created with:

 - url=http://169.254.169.254/2009-04-04/meta-data \
  ( curl --silent $url/hostname | sed -e 's/\..*//' ; \
    printf "%03d" $(curl --silent $url/local-ipv4 | \
       sed -e 's/.*\.\(.*\)/\1/') \
  ) | \
  tee /etc/hostname
- hostname $(cat /etc/hostname)
preserve_hostname: true

Where $url/hostname retrieves the prefix of the hostname (multiple instances can have the same name, two simultaneous instance creation won’t race), $url/local-ipv4 gets the IPv4 address, keeps the last digits (sed -e ‘s/.*\.\(.*\)/\1/’)) and pad them with zeros if necessary (printf “%03d”). The hostname is stored in /etc/hostname and displayed in the /var/log/cloud-init.log logs (tee /etc/hostname) for debugging. This is done early in the cloudinit sequence (bootcmd) and the default cloudinit setting of the hostname is disabled (preserve_hostname: true) so that it does not override the custom name set with hostname $(cat /etc/hostname).
The instance is created with

$ openstack server create \
  --image 'ubuntu-trusty-14.04'
  --key-name loic \
  --flavor m1.small \
  --user-data user-data.txt \
  -f json \
  --wait \
  the-re
... {"Field": "addresses", "Value": "fsf-lan=10.0.3.19"} ...
... {"Field": "id", "Value": "cd1a8a0f-83f9-4266-bd61-f3e2f583d59d"} ...

Whe user-data.txt contains the above cloudinit lines. The IPv4 address returned by openstack server create (10.0.3.19) can then be used to rename the instance with

$ openstack server set --name the-re019 cd1a8a0f-83f9-4266-bd61-f3e2f583d59d

where cd1a8a0f-83f9-4266-bd61-f3e2f583d59d is the unique id of the instance which is preferred to the the-re prefix that could race with another identical openstack server create command.
To verify that the instance name matches the IPv4 address that is pre-set in the DNS:

$ ssh ubuntu@the-re019 hostname
Warning: Permanently added '10.0.3.19' (ECDSA) to the list of known hosts.
the-re019

Thanks to Josh Durgin for suggesting this solution.

Delete the last port of an OpenStack router

When trying to delete an OpenStack subnet and the associated router, the command neutron router-delete complains because of the port allocated for the gateway and the port of the gateway cannot be removed with neutron port-delete because it is owned by the router. The solution is to clear the owner of the port with something like:

neutron port-update --device-owner clear 7f9685cb-794d-4847

and then delete the router. This is on Icehouse as provided by Enter Cloud Suite.

OpenStack script to pre-allocate fixed IPs

The create-ports.py script allocates ports and indirectly gets fixed IPs from the DHCP server. The ports are named openstack000, openstack001 etc. and they are displayed in a format suitable for dnsmasq:

$ python create-ports.py --count 2 --net fsf-lan |  \
   sudo tee /etc/dnsmasq.d/openstack
host-record=openstack000,10.0.3.32
host-record=openstack001,10.0.3.33

If fsf-lan is a network shared with other tenants, it makes sure the IP are reserved, although they are not yet bound to an instance.

$ neutron port-list
+--------------------------------------+--------------+..
| id                                   | name         | ...
+--------------------------------------+--------------+...
| 1d1a05b1-383d-49ef-ae75-5ddcb5c714db | openstack001 |....
...
+--------------------------------------+--------------+...

An new instance can then be given a known IP with:

$ openstack server create --image ubuntu-trusty-14.04 \
  --flavor 1cpu-1G \
  --key-name teuthology \
  --nic net-id=d936f445-5d68-485a-94f2-b852fd6b7d0c,v4-fixed-ip=10.0.3.33 \
  --wait openstack001

In the case of teuthology it is useful because the DNS can be configured once and for all while instances are dynamically created using IPs from the DNS instead of relying on allocation from the OpenStack DHCP server.
Continue reading “OpenStack script to pre-allocate fixed IPs”

create / delete an OpenStack instance with python-openstackclient

The python-openstackclient library has an example that provides the basic structure for a new command (the auth_url problem workaround may be needed). To create a virtual machine with 1GB RAM, 1CPU, ubuntu-14.04, using the teuthology keypair on the fsf-lan network, the matching flavor, image, keypair and network objects can be found with:

    for flavor in client_manager.compute.flavors.list():
        if flavor.ram == 1024 and flavor.vcpus == 1:
            break
    for network in client_manager.compute.networks.list():
        if network.label == 'fsf-lan':
            break
    for image in client_manager.compute.images.list():
        if 'ubuntu' in image.name and '14.04' in image.name:
            break
    for keypair in client_manager.compute.keypairs.list():
        if keypair.name == 'teuthology':
            break

The test instance can then be created

   server = client_manager.compute.servers.create('test',
                        image, flavor,
                        key_name=keypair.name,
                        nics=[{'net-id': network.id}])

but it won’t be immediately active and the wait_for_status can be used to block until it is:

from openstackclient.common import utils
...
    utils.wait_for_status(
        client_manager.compute.servers.get,
        server.id)

Deleting the instance is simpler:

    client_manager.compute.servers.delete(server.id)
    utils.wait_for_delete(client_manager.compute.servers.get, server.id)

See create-delete.py for a standalone script including the above lines that can be run as:

$ python create-server.py --help
usage: create-server.py [-h] [--os-compute-api-version ]
...
$ python create-server.py
FLAVOR: {'name': u'm1.small', ...
NETWORK: {'cidr_v6': None, 'dns2': None, 'dns1': None, 'netmask': None, 'label': u'fsf-lan',...
IMAGE: {'status': u'ACTIVE', 'updated': u'2014-05-19T11:43:00Z', 'name': u'ubuntu-trusty-14.04',...
KEYPAIR: {'public_key': u'ssh-rsa AAAAB3...

Continue reading “create / delete an OpenStack instance with python-openstackclient”

Teuthology docker targets hack (5/5)

The teuthology container hack is improved to run teuthology-suite. For instance:

./virtualenv/bin/teuthology-suite \
  --distro ubuntu \
  --suite-dir $HOME/software/ceph/ceph-qa-suite \
  --config-file docker-integration/teuthology.yaml \
  --machine-type container \
  --owner loic@dachary.org \
  --filter 'rados:basic/{clusters/fixed-2.yaml fs/btrfs.yaml \
     msgr-failures/few.yaml tasks/rados_cls_all.yaml}' \
  --suite rados/basic --ceph ANY \
  $(pwd)/docker-integration/ubuntu.yaml

schedules a single job out of the rados suite and the results can be collected in the teuthology-worker archive directory:

$ tail -5 /tmp/a/loic-2015-06-06_16:06:57-rados:\
    basic-ANY---basic-container/22/teuthology.log
06:57-rados:basic-ANY---basic-container/22/teuthology.log
    tasks/rados_cls_all.yaml}', duration: 1017.5819008350372, \
  flavor: basic, owner: loic@dachary.org,
  success: true}
2015-06-06T16:24:38.634 WARNING:teuthology.report:No result_server \
  in config; not reporting results
2015-06-06T16:24:38.634 INFO:teuthology.run:pass

Continue reading “Teuthology docker targets hack (5/5)”