Posted on

HOWTO OpenStack Grizzly and Ceph with Puppet on Ubuntu 12.04

For months I’ve asked people working with puppet modules on a daily basis for a HOWTO that I could follow to setup a new cluster with the Grizzly OpenStack release. Such a HOWTO is not needed for people who develop the modules or deploy OpenStack for a living. It is however very helpful for the casual system administrator willing to get it running in a few hours, all by herself/himself.
The packstack seems to be exactly that : a walkthru of a well tested procedure that anyone with a basic understanding of what OpenStack is can rely on. It requires an RPM based distribution and this may be a significant effort for someone used to DEB based operating systems.
For Ubuntu users, the kickstack project was started in summer 2013 and targets hands on sessions, with the declared goal to make it easy for people new to both OpenStack and puppet. Later on, it inspired Dan Bode to use a new approach based on dependency injection to implement openstack-installer for Cisco.
The proposed HOWTO uses openstack-installer to deploy OpenStack against an existing Ceph cluster and provides:

  • keystone
  • nova ( kvm )
  • quantum ( openvswitch + gre )
  • cinder ( Ceph backend )
  • horizon
  • glance ( Ceph backend )

Continue reading “HOWTO OpenStack Grizzly and Ceph with Puppet on Ubuntu 12.04”

Posted on

gerritexec: continuous integration one-liner

gerritexec is a command line tool listening to gerrit on a designated project. On each new patchset, it will:

  • git clone the project
  • git pull the patchset
  • cd in the git tree and run a script
  • positively review the patchset ( +1 ) if the program exit(0)
  • negatively review the patchset ( -1 ) otherwise

For instance It can be used to run the integration tests found in the git tree of the stackforge/puppet-ceph project:

GEM_HOME=~/.gems gerritexec --hostname review.openstack.org \
           --username puppetceph \
           --script ' ( bundle install ; bundle exec rake spec:system ) > /tmp/out 2>&1 ; r=$? ; pastebinit /tmp/out ; exit $r #' \
           --project stackforge/puppet-ceph

Larger projects should consider using zuul or a gerrit jenkins plugin.
Continue reading “gerritexec: continuous integration one-liner”

Posted on

setting up an openstack-installer test environment

openstack-installer is a data oriented replacement of puppet-openstack. The following HOWTO runs some basic tests on vagrant virtual machines that are preserved for introspection with:

# vagrant status
control_basevm           running
# vagrant ssh control_basevm
vagrant@control-server:~$ ps -ax | grep keystone
15020 ?        Ss     0:01 /usr/bin/python /usr/bin/keystone-all

The control_basevm runs the horizon dashboard:

Continue reading “setting up an openstack-installer test environment”

Posted on

puppet master hierarchy organization and conventions

April hosts dozens of services on less than ten hardware machines and less than fifty virtual machines. Their configuration is centralized in a puppet master repository. The order of magnitude is not expected to change in the next few years. The hierarchy is organized in manifests and modules. The manifests directory contains the inventory of all virtual machines and bare metal associated with the inclusion of classes and the instantiation of types.

node 'harmine.pavot.vm.april-int',
     'backuppc.novalocal' inherits vserver-pavot {
  include backuppc::server
  include april_nagios::nrpe_server
  include april_nagios::check_backuppc
}

The modules contain the configuration logic for a sub system such as backuppc, screen or ssh.

class screen {
  package { 'screen': ensure => present, }

  file { '/root/.screenrc':
    ensure  => present,
    owner   => 'root',
    group   => 'root',
    mode    => 0400,
    replace => true,
    source  => 'puppet:///screen/.screenrc',
  }

}

Continue reading “puppet master hierarchy organization and conventions”

Posted on

anatomy of an OpenStack based integration test for a backuppc puppet module

An integration test is run by jenkins within an OpenStack tenant. It checks that the backuppc puppet module is installed

ssh root@$instance test -f /etc/backuppc/hosts || return 3

A full backup is run

ssh root@$instance su -c '"/usr/share/backuppc/bin/BackupPC_serverMesg \
   backup nagios.novalocal nagios.novalocal backuppc 1"' \
                              backuppc || return 4
ssh root@$instance tail -f /var/lib/backuppc/pc/nagios.novalocal/LOG.* | \
    sed --unbuffered -e "s/^/$instance: /" -e '/full backup 0 complete/q'

and a nagios plugin asserts its status is monitored

    while ! ( echo "GET services"
        echo "Filter: host_alias = $instance.novalocal"
        echo "Filter: check_command = check_nrpe_1arg"'!'"check_backuppc" ) |
        ssh root@nagios unixcat /var/lib/nagios3/rw/live |
        grep "BACKUPPC OK - (0/" ; do
        sleep 1
    done

Continue reading “anatomy of an OpenStack based integration test for a backuppc puppet module”

Posted on

realistic puppet tests with jenkins and OpenStack (part 2/2)

The April infrastructure uses puppet manifests stored in a git repository. On each commit, a jenkins job is run and it performs realistic tests in a dedicated OpenStack tenant.

If the test is successfull, jenkins pushes the commit to the production branch. The production machines can then pull from it:

root@puppet:/srv/admins# git pull
Updating 5efbe80..cf59d69
Fast-forward
 .gitmodules                      |    6 +++
 jenkins/openstack-test.sh        |   53 +++++++++++++++++++++++++++
  jenkins/run-test-in-openstack.sh |  215 +++++++++++++++++++++++++++
 puppetmaster/manifests/site.pp   |   43 ++++++++++++++++++++--
 puppetmaster/modules/apt         |    1 +
 6 files changed, 315 insertions(+), 165 deletions(-)
 create mode 100755 jenkins/openstack-test.sh
 create mode 100644 jenkins/run-test-in-openstack.sh
 create mode 160000 puppetmaster/modules/apt
root@puppet:/srv/admins# git branch -v
  master     5efbe80 [behind 19] ajout du support nagios, configuration .... refs #1053
* production cf59d69 Set the nagios password for debugging ...

Continue reading “realistic puppet tests with jenkins and OpenStack (part 2/2)”

Posted on

realistic puppet tests with jenkins and OpenStack (part 1/2)

A git repository containing a puppet module is bound to a jenkins project. When the repository changes, jenkins boots a virgin puppetmaster OpenStack instance in a dedicated tenant. It runs the run-jenkins-test-in-openstack.sh script in the puppetmaster instance. In addition to the puppet unit tests, the script will launch realistic tests by launching OpenStack instances and checking their state. The checks are done with nagios which can also be used in a production environment to continuously monitor the deployment.
Continue reading “realistic puppet tests with jenkins and OpenStack (part 1/2)”

Posted on

creating a Debian GNU/Linux Wheezy puppet client for OpenStack

A Debian GNU/Linux wheezy image is booted and modified to set its hostname based on the content of the http://169.254.169.254/latest/meta-data/hostname metadata. The /etc/rc.local file is changed to run puppet agent –waitforcert 60 at boot time. The instance is then snapshoted and the corresponding file system reduced to a minimal size with resize2fs -M.
Continue reading “creating a Debian GNU/Linux Wheezy puppet client for OpenStack”