Lowering Ceph scrub I/O priority

Note: the following does not currently work in Firefly because of http://tracker.ceph.com/issues/9677 . It has been backported to Firefly and will likely be in 0.80.8.

The disk I/O of a Ceph OSD thread scrubbing is the same as all other threads by default. It can be lowered with ioprio options for all OSDs with:

ceph tell osd.* injectargs '--osd_disk_thread_ioprio_priority 7'
ceph tell osd.* injectargs '--osd_disk_thread_ioprio_class idle'

All other threads in the OSD will be be (best effort) with priority 4 which is the default for daemons. The disk thread will show as idle:

$ sudo iotop --batch --iter 1 | grep 'ceph-osd -i 0' | grep -v be/4
 4156 idle loic        0.00 B/s    0.00 B/s  0.00 %  0.00 % ./ceph-osd -i 0 ..

Continue reading “Lowering Ceph scrub I/O priority”