Category: docker

The teuthology container hack is improved to run teuthology-suite. For instance:

./virtualenv/bin/teuthology-suite \
  --distro ubuntu \
  --suite-dir $HOME/software/ceph/ceph-qa-suite \
  --config-file docker-integration/teuthology.yaml \
  --machine-type container \
  --owner loic@dachary.org \
  --filter 'rados:basic/{clusters/fixed-2.yaml fs/btrfs.yaml \
     msgr-failures/few.yaml tasks/rados_cls_all.yaml}' \
  --suite rados/basic --ceph ANY \
  $(pwd)/docker-integration/ubuntu.yaml

schedules a single job out of the rados suite and the results can be collected in the teuthology-worker archive directory:

$ tail -5 /tmp/a/loic-2015-06-06_16:06:57-rados:\
    basic-ANY---basic-container/22/teuthology.log
06:57-rados:basic-ANY---basic-container/22/teuthology.log
    tasks/rados_cls_all.yaml}', duration: 1017.5819008350372, \
  flavor: basic, owner: loic@dachary.org,
  success: true}
2015-06-06T16:24:38.634 WARNING:teuthology.report:No result_server \
  in config; not reporting results
2015-06-06T16:24:38.634 INFO:teuthology.run:pass

Continue reading “Teuthology docker targets hack (5/5)”

The teuthology container hack is improved by adding a flag to retrieve packages from a user specified repository instead of gitbuilder.ceph.com. The user can build packages from sources and run a job, which will implicitly save a docker image with the package installed. The second time the same job is run, it will go faster because it reuses the image. For instance the following job:

machine_type: container
os_type: ubuntu
os_version: "14.04"
suite_path: /home/loic/software/ceph/ceph-qa-suite
roles:
- - mon.a
  - osd.0
  - osd.1
  - client.0
overrides:
  install:
    ceph:
      branch: master
  ceph:
    wait-for-scrub: false
tasks:
- install:
    repository_url: http://172.17.42.1/trusty
- ceph:

runs under one minute:

{duration: 47.98, flavor: basic, owner: loic@dachary.org, success: true}

The automated make check for Ceph bot runs on Ceph pull requests. It is still experimental and will not be triggered by all pull requests yet.

It does the following:

• Create a docker container (using ceph-test-helper.sh)
• Checkout the merge of the pull request against the destination branch (tested on master, next, giant, firefly)
• Execute run-make-check.sh
• Add a comment to the pull request with a link to the full output of run-make-check.sh.

A use case for developers is:

• write a patch and send a pull request
• switch to another branch and work on another patch while the bot is running
• if the bot reports failure, switch back to the original branch and repush a fix: the bot will notice the repush and run again

It also helps reviewers who can wait until the bot succeeds before looking at the patch closely.
Continue reading “A make check bot for Ceph contributors”

The teuthology container hack is improved so each Ceph command is run via docker exec -i which can read from stdin as of docker 1.4 released in December 2014.
It can run the following job

machine_type: container
os_type: ubuntu
os_version: "14.04"
suite_path: /home/loic/software/ceph/ceph-qa-suite
roles:
- - mon.a
  - osd.0
  - osd.1
  - client.0
overrides:
  install:
    ceph:
      branch: master
  ceph:
    wait-for-scrub: false
tasks:
- install:
- ceph:

under one minute, when repeated a second time and the bulk of the installation can be reused.

{duration: 50.01510691642761, flavor: basic,
  owner: loic@dachary.org, success: true}

Continue reading “Teuthology docker targets hack (3/5)”

The teuthology container hack is improved to snapshot the container after Ceph and its dependencies have been installed. It helps quickly testing ceph-qa-suite tasks. A job doing nothing but install the Firefly version of Ceph takes 14 seconds after the initial installation (which can take between 5 to 15 minutes depending on how fast is the machine and how much bandwidth is available).

...
2014-11-17 01:21:00,067.067 INFO:teuthology.worker:Reserved job 42
2014-11-17 01:21:00,067.067 INFO:teuthology.worker:Config is:
machine_type: container
name: foo
os_type: ubuntu
os_version: '14.04'
overrides:
  install:
    ceph: {branch: firefly}
owner: loic@dachary.org
priority: 1000
roles:
- [mon.a, osd.0, osd.1, client.0]
tasks:
- {install: null}
tube: container
verbose: false

Fetching from upstream into /home/loic/src/ceph-qa-suite_master
...
completed on container001: sudo lsb_release '-is':  Ubuntu
reusing existing image ceph-base-ubuntu-14.04-firefly
running 'docker' 'stop' 'container001'
completed ('docker', 'stop', u'container001') on container001:  container001
...
2014-11-17 01:21:31,677.677 INFO:teuthology.run:Summary data:
{duration: 14, flavor: basic, success: true}
2014-11-17 01:21:31,677.677 INFO:teuthology.run:pass

Continue reading “Teuthology docker targets hack (2/5)”

teuthology runs jobs testing the Ceph integration on targets that can either be virtual machines or bare metal. The container hack adds support for docker containers as a replacement.

...
Running task exec...
Executing custom commands...
Running commands on role mon.a host container002
running sudo 'TESTDIR=/home/ubuntu/cephtest' bash '-c' '/bin/true'
running docker exec container002 bash /tmp/tmp/tmptJ7hxa
Duration was 0.088931 seconds
...

Continue reading “Teuthology docker targets hack (1/5)”

The Ceph command to setup a block device ( ceph-disk) needs to call partprobe after zapping a disk. The patch adding the partprobe call needs a block device to test that it works as expected. The body of the test requires root privileges:

 dd if=/dev/zero of=vde.disk bs=1024k count=200
 losetup --find vde.disk
 local disk=$(losetup --associated vde.disk | cut -f1 -d:)
 ./ceph-disk zap $disk

which is potentially dangerous for the developer machine. The run of the test is delegated to a docker container so that accidentally removing /var/run has no consequence. Although the test recompiles Ceph the first time:

main_docker "$@" --compile
main_docker "$@" --user root --dev test/ceph-disk.sh test_activate_dev

it will reuse the compilation results if run a second time. Unless there is a new commit in which case it will recompile whatever make decides. The ceph-disk-root.sh script is added to the list of scripts that are run on make check but will only be considered if –enable-docker has been given to ./configure and docker is available. Otherwise it will be silently ignored

if ENABLE_DOCKER
check_SCRIPTS += \
 test/ceph-disk-root.sh
endif

Continue reading “Testing ceph-disk with block devices in docker”

After Ceph is built from sources, unit and functional tests can be run with make check. Delegating the execution to a container makes it possible to:

• keep working on the sources without disrupting the run
• run functional tests that require root privileges without modifying the development environment
• check various operating systems

The src/test/docker-test-helper.sh library can be used from the command line:

$ test/docker-test.sh --os-type ubuntu --os-version 14.04 make check &
$ test/docker-test.sh --os-type centos --os-version 7 make check &

Each run uses a clone of the current repository and pulls from origin before executing the command. For instance, if running from /srv/ceph, the centos run will run make check in /srv/ceph-centos-7 which is bind mounted in the container. A possible workflow is:

• work
• commit
• test/docker-test.sh make check which pulls the latest commits
• keep working
• check the make check output

In case an error happens, debugging starts by running a shell in the container

$ test/docker-test.sh --os-type ubuntu --os-version 14.04 --shell
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 10 (delta 8), reused 0 (delta 0)
Unpacking objects: 100% (10/10), done.
From /home/loic/software/ceph/ceph
 + 15046fe...8a39cad wip-9665 -> origin/wip-9665
HEAD is now at 8a39cad autotools: add --enable-docker
loic@203c085f3dc1:/srv/ceph-ubuntu-14.04$

The first time test/docker-test.sh runs, it creates a docker images populated with the packages necessary to compile and run Ceph. This lowers the overhead to run a test in the container:

$ time test/docker-test.sh --os-type ubuntu --os-version 14.04 unittest_str_map
HEAD is now at 8a39cad autotools: add --enable-docker
Running main() from gtest_main.cc
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from str_map
[ RUN      ] str_map.json
[       OK ] str_map.json (1 ms)
[ RUN      ] str_map.plaintext
[       OK ] str_map.plaintext (0 ms)
[----------] 2 tests from str_map (1 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (1 ms total)
[  PASSED  ] 2 tests.

real	0m3.340s
user	0m0.071s
sys	0m0.046s