Free Software developer journey
The ceph-conf command line queries the /etc/ceph/ceph.conf file.
# ceph-conf --lookup fsid 571bb920-6d85-44d7-9eca-1bc114d1cd75
The –show-config option can be used to display the config of a running daemon:
ceph -n osd.123 --show-config
When no name is specified, it will show the default Ceph configuration
ceph --show-config --conf /dev/null
On ganeti, shutdown the instance and activate its disks:
z2-8:~# gnt-instance shutdown nerrant Waiting for job 1089813 for nerrant... z2-8:~# gnt-instance activate-disks nerrant z2-8.host.gnt:disk/0:/dev/drbd10
On an OpenStack Havana installation using a Ceph cinder backend, create a volume with the same size:
# cinder create --volume-type ovh --display-name nerrant 10
+---------------------+--------------------------------------+
| Property | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2013-11-12T13:00:39.614541 |
| display_description | None |
| display_name | nerrant |
| id | 3ec2035e-ff76-43a9-bbb3-6c003c1c0e16 |
| metadata | {} |
| size | 10 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| volume_type | ovh |
+---------------------+--------------------------------------+
# rbd --pool ovh info volume-3ec2035e-ff76-43a9-bbb3-6c003c1c0e16
rbd image 'volume-3ec2035e-ff76-43a9-bbb3-6c003c1c0e16':
size 10240 MB in 2560 objects
order 22 (4096 KB objects)
block_name_prefix: rbd_data.90f0417089fa
format: 2
features: layering
On a host connected to the Ceph cluster and running a linux-kernel > 3.8 ( because of the format: 2 above ), map to a bloc device with:
# rbd map --pool ovh volume-3ec2035e-ff76-43a9-bbb3-6c003c1c0e16 # rbd showmapped id pool image snap device 1 ovh volume-3ec2035e-ff76-43a9-bbb3-6c003c1c0e16 - /dev/rbd1
Copy the ganeti volume with:
z2-8:~# pv < /dev/drbd10 | ssh bm0014 dd of=/dev/rbd1 2,29GB 0:09:14 [4,23MB/s] [==========================> ] 22% ETA 0:31:09
and unmap the device when it completes.
rbd unmap /dev/rbd1
The volume is ready to boot.
OpenStack Havana is installed on machines rented from OVH and Hetzner. An aggregate is created for machines hosted at OVH and another for machines hosted at Hetzner. A Ceph cluster is created with a pool using disks from OVH and another pool using disks from Hetzner. A cinder backend is created for each Ceph pool. From the dashboard, an instance can be created in the OVH availability zone using a Ceph volume provided by the matching OVH pool.
Continue reading “Collocating Ceph volumes and instances in a multi-datacenter setup”
For months I’ve asked people working with puppet modules on a daily basis for a HOWTO that I could follow to setup a new cluster with the Grizzly OpenStack release. Such a HOWTO is not needed for people who develop the modules or deploy OpenStack for a living. It is however very helpful for the casual system administrator willing to get it running in a few hours, all by herself/himself.
The packstack seems to be exactly that : a walkthru of a well tested procedure that anyone with a basic understanding of what OpenStack is can rely on. It requires an RPM based distribution and this may be a significant effort for someone used to DEB based operating systems.
For Ubuntu users, the kickstack project was started in summer 2013 and targets hands on sessions, with the declared goal to make it easy for people new to both OpenStack and puppet. Later on, it inspired Dan Bode to use a new approach based on dependency injection to implement openstack-installer for Cisco.
The proposed HOWTO uses openstack-installer to deploy OpenStack against an existing Ceph cluster and provides:
Continue reading “HOWTO OpenStack Grizzly and Ceph with Puppet on Ubuntu 12.04”
gerritexec is a command line tool listening to gerrit on a designated project. On each new patchset, it will:
For instance It can be used to run the integration tests found in the git tree of the stackforge/puppet-ceph project:
GEM_HOME=~/.gems gerritexec --hostname review.openstack.org \
--username puppetceph \
--script ' ( bundle install ; bundle exec rake spec:system ) > /tmp/out 2>&1 ; r=$? ; pastebinit /tmp/out ; exit $r #' \
--project stackforge/puppet-ceph
Larger projects should consider using zuul or a gerrit jenkins plugin.
Continue reading “gerritexec: continuous integration one-liner”
OpenStack is installed with puppet and configured to use a Ceph cluster installed with ceph-deploy. The hardware is composed of about 25 heterogeneous HP machines that are over three years old. Five of them have been racked in a basement that not as dry as it should be. The 100Mb/s Internet connection uses a 30m category 5 cable going through two holes in the walls before reaching the rack.
The cluster will be connected to other similar clusters to reduce the risk of loosing data.
Continue reading “OpenStack + Ceph on junk hardware and hazardous hosting”
Adding, moving and removing disks in a Ceph cluster can easily be automated and require no manual intervention. New disks are formatted when the configuration tool ( Puppet, Chef etc. ) notices they are unknown according to ceph-disk. When disks are removed for whatever reason, Ceph recovers the information it contained, as expected. If disks are hot plugged from one machine to another, udev rules will automatically allow them to rejoin the cluster and update the crush map accordingly.
Continue reading “Fully automated disks life cycle in a Ceph cluster”
The micro-osd bash script is 91 lines long and creates a Ceph cluster with a single OSD and a single MON in less that five seconds
$ time bash micro-osd.sh single-osd starting osd.0 at :/0 osd_data single-osd/osd single-osd/osd.journal # id weight type name up/down reweight -1 1 root default 0 1 osd.0 up 1 export CEPH_ARGS='--conf single-osd/ceph.conf' ceph osd tree real 0m4.677s
It is meant to be used for integration tests, for example in the context of the openstack-installer puppet manifest to deploy OpenStack. Ceph is configured to run from a directory, on a single host, without cephx, in a non privileged environment and uses about 100MB of disk space.
$ du -sh single-osd/ 103M single-osd/
The UMR de Génétique Végétale is a state funded french research facility located in a historical monument: a large farm surrounded by fields harvested for experiments. Johann Joets was assigned a most unusual mission : setup a state of the art datacenter in a former pigsty. His colleague Olivier Langella does the system administration for PAPPSO and researched an extendable storage solution able to transparently sustain the loss of any hardware component. A simple Ceph setup was chosen and is in use since 2012.
Continue reading “Ceph use case : proteomic analysis”
This is a dialog where Samuel Just, Ceph core developer answers Loïc Dachary’s questions to explain the details of PGBackend (i.e. placement group backend). It is a key componenent of the implementation of erasure coded pools.
Continue reading “Ceph dialog : PGBackend”