rados bench 是 Ceph 自带的用来测试存储池性能的压测工具,其 main 函数在 src/tools/rados/rados.cc 中。rados.cc 集成了 rados 所有 bash 命令,可以通过 rados -h 查看帮助。开篇点题,让我们看看执行 rados bench 时,哪些线程被偷偷启动了。
#在一个终端开启 rados bench
[root@localhost build]# ./bin/rados -p rbd bench 60 write
#在另一个终端监控 rados 启动哪些线程
[root@localhost ~]# ps -ef | grep rados
root 17796 10743 0 16:08 pts/0 00:00:00 ./bin/rados bench -p rbd 60 write
root 17808 10765 0 16:09 pts/1 00:00:00 grep --color=auto rados
[root@localhost ~]# top -Hp 17796 -d 0.5
top - 16:19:51 up 6:36, 2 users, load average: 2.54, 0.73, 0.35
Threads: 16 total, 0 running, 16 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.3 sy, 0.0 ni, 87.4 id, 12.3 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 3861288 total, 201816 free, 1232584 used, 2426888 buff/cache
KiB Swap: 4063228 total, 4063228 free, 0 used. 2327116 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
20207 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.28 rados
20208 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 msgr-worker-0
20209 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.49 msgr-worker-1
20210 root 20 0 1328848 95972 20544 S 0.0 2.5 0:01.60 msgr-worker-2
20215 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 log
20216 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 service
20217 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 admin_socket
20222 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 rados
20223 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 ms_dispatch
20224 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 ms_local
20225 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 safe_timer
20226 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 fn_anonymous
20227 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 safe_timer
20228 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 safe_timer
20229 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.01 fn-radosclient
20230 root 20 0 1328848 95972 20544 S 0.0 2.5 0:00.00 write_stat
可以看到除了本身执行 rados bench 命令的线程外,还有十几个线程被创建出来。简单介绍下,线程号为20207的 rados 线程是 rados bench 命令的主线程,msgr-worker-0~3 是用来网络通信的三个工作线程,log 是用来记录日志的日志线程,service 是监控本地性能参数的线程,admin_socket 是对外提供查询接口的线程,ms_dispatch 和 ms_local 是用来处理消息分发的线程,safer_timer 是执行定时任务的线程,fn_anonymous 和 fn-radosclient 则是处理所有模块收尾工作的线程。