lsof也是Linux中比较常用的命令，是list open files的简写，在Linux系统中，一切皆为文件。
查看端口使用信息
(ENV) [root@ceph-2 log]# lsof -i:9090
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
openresty 1550 root 6u IPv4 31792 0t0 TCP ceph-2:websm (LISTEN)
prometheu 1591 root 5u IPv4 29207 0t0 TCP localhost:websm (LISTEN)
prometheu 1591 root 6u IPv4 31813 0t0 TCP localhost:57564->localhost:websm (ESTABLISHED)
prometheu 1591 root 7u IPv4 29208 0t0 TCP localhost:websm->localhost:57564 (ESTABLISHED)
openresty 1679 nfsnobody 6u IPv4 31792 0t0 TCP ceph-2:websm (LISTEN)
openresty 1680 nfsnobody 6u IPv4 31792 0t0 TCP ceph-2:websm (LISTEN)
openresty 1681 nfsnobody 6u IPv4 31792 0t0 TCP ceph-2:websm (LISTEN)
openresty 1682 nfsnobody 6u IPv4 31792 0t0 TCP ceph-2:websm (LISTEN)
查看数据库连接信息
lsof -Pni:5432 | fgrep ">" | fgrep -i established | wc -l
恢复打开但被删除的文件
首先，我们删掉/var/log/maillog 文件，然后调用lsof命令查看哪个进程占用了该文件，加上-w选项是抑制warning信息。
+|-w Enables (+) or disables (-) the suppression of warning messages.
如下所示，maillog文件已经被删除，但是还被进程619占用。
(ENV) [root@ceph-2 log]# lsof -w | grep deleted
rsyslogd 619 root 9w REG 8,1 1942 33576309 /var/log/maillog (deleted)
in:imjour 619 639 root 9w REG 8,1 1942 33576309 /var/log/maillog (deleted)
rs:main 619 640 root 9w REG 8,1 1942 33576309 /var/log/maillog (deleted)
java 1613 1000 248r FIFO 8,1 0t0 374779 /usr/share/elasticsearch/tmp/controller_log_1 (deleted)
查看/proc文件系统中进程619的fd信息。
(ENV) [root@ceph-2 log]# ll /proc/619/fd
total 0
lr-x------ 1 root root 64 Apr 11 23:30 0 -> /dev/null
l-wx------ 1 root root 64 Apr 11 23:30 1 -> /dev/null
l-wx------ 1 root root 64 Apr 11 23:30 2 -> /dev/null
lrwx------ 1 root root 64 Apr 11 23:30 3 -> socket:[22724]
l-wx------ 1 root root 64 Apr 11 23:30 4 -> /var/log/messages
lr-x------ 1 root root 64 Apr 11 23:30 5 -> /run/log/journal/93978c1c45e54d1e8843cc4727b1f26e/system.journal
lr-x------ 1 root root 64 Apr 11 23:30 6 -> anon_inode:inotify
l-wx------ 1 root root 64 Apr 11 23:30 7 -> /var/log/secure
l-wx------ 1 root root 64 Apr 11 23:30 8 -> /var/log/cron
l-wx------ 1 root root 64 Apr 11 23:30 9 -> /var/log/maillog (deleted)
使用重定向恢复。
(ENV) [root@ceph-2 log]# cat /proc/619/fd/9 > maillog
对于删除文件后空间不释放的情况
可以进行如下操作：
lsof +L1
to find out which process is using a deleted (unlinked) file.
root@t25:/proc/3591/fd# lsof +L1
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
systemd-j 1352 root 41u REG 253,0 8388608 0 54726495 /var/log/journal/e7f80dcce5d34395b4f4d546dccb0dc6/user-1000@9775dbb6fdbb4d04a9837fd07a7dd92b-0000000000000000-0000000000000000.journal (deleted)
unattende 2719 root 3w REG 253,0 112 0 54724830 /var/log/unattended-upgrades/unattended-upgrades-shutdown.log.1 (deleted)
glusterev 2839 root 3u REG 0,23 5 0 1490 /run/glustereventsd.pid (deleted)
kubelet 2983 root txt REG 253,0 200816272 0 224461090 /usr/local/bin/.~tmp~/kubelet (deleted)
glusterfs 3591 root 269u REG 253,0 0 0 208468691 /glusterfs/perception_labeled_data/.glusterfs/unlink/5a4ab33a-0729-4e20-96ac-4d1a44811433 (deleted)
root@t25:/proc/3591/fd# ls -al 269
lrwx------ 1 root root 64 Aug 10 09:02 269 -> '/glusterfs/perception_labeled_data/.glusterfs/unlink/5a4ab33a-0729-4e20-96ac-4d1a44811433 (deleted)'
type "> FD" to reclaim that space
eg:
> 269
You might need to repeat the operation if there are other processes holding the file.
查看运行某命令的进程打开的文件
使用-c选项查看运行某命令的进程打开的文件。如果-c选项跟着的进程名前面有^，代表忽略或排除此command的进程。
(ENV) [root@ceph-2 ~]# lsof -c prometheus
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
prometheu 1591 root cwd DIR 8,1 267 202737052 /prometheus
prometheu 1591 root rtd DIR 8,1 166 102497690 /
prometheu 1591 root txt REG 8,1 57884414 169372405 /bin/prometheus
prometheu 1591 root 9r REG 8,1 536845001 203327013 /prometheus/01E4TPBR3PZW9N9Y4WKA94CQMG/chunks/000001
prometheu 1591 root 10r REG 8,1 297192670 203327015 /prometheus/01E4TPBR3PZW9N9Y4WKA94CQMG/chunks/000002
prometheu 1591 root 11r REG 8,1 31754494 169745385 /prometheus/01E4TPBR3PZW9N9Y4WKA94CQMG/index
prometheu 1591 root 12r REG 8,1 171489821 135815646 /prometheus/01E5KJW61K2D04E2XND660JQPX/chunks/000001
prometheu 1591 root 13r REG 8,1 5377477 102777587 /prometheus/01E5KJW61K2D04E2XND660JQPX/index
prometheu 1591 root 15r REG 8,1 19142330 35714626 /prometheus/01E5MW2BCSQDCZTTRSXYZPVSCF/chunks/000001
prometheu 1591 root 16r REG 8,1 8522570 335544532 /prometheus/01E5BVP5DV4F05VMP1WC4KSKNN/index
查看某一进程打开的文件数量
[root@xsclaer-239 ~]# lsof -p 385279 | wc -l
20887
查看某端口的连接情况
[root@xsclaer-239 ~]# lsof -i4 -n -P | grep 8051 | wc -l
28098
# 从中也可以看出是哪个程序连接的
1 openresty 332229 nfsnobody 18u IPv4 84619305 0t0 TCP 127.0.0.1:35403->127.0.0.1:8051 (ESTABLISHED)
2 openresty 332229 nfsnobody 20u IPv4 76735473 0t0 TCP 127.0.0.1:56224->127.0.0.1:8051 (ESTABLISHED)
3 prometheu 332304 root 9u IPv4 29130144 0t0 TCP 10.0.11.239:48622->10.0.11.238:8051 (ESTABLISHED)
4 prometheu 332304 root 10u IPv4 29115246 0t0 TCP 10.0.11.239:48626->10.0.11.238:8051 (ESTABLISHED)
5 prometheu 332304 root 11u IPv4 29130145 0t0 TCP 10.0.11.239:48634->10.0.11.238:8051 (ESTABLISHED)
6 prometheu 332304 root 12u IPv4 29115251 0t0 TCP 10.0.11.239:48644->10.0.11.238:8051 (ESTABLISHED)
7 prometheu 332304 root 17u IPv4 29115252 0t0 TCP 10.0.11.239:48648->10.0.11.238:8051 (ESTABLISHED)
8 prometheu 332304 root 18u IPv4 29115253 0t0 TCP 10.0.11.239:48650->10.0.11.238:8051 (ESTABLISHED)
9 prometheu 332304 root 19u IPv4 29130147 0t0 TCP 10.0.11.239:48652->10.0.11.238:8051 (ESTABLISHED)
10 prometheu 332304 root 20u IPv4 29136124 0t0 TCP 10.0.11.239:48658->10.0.11.238:8051 (ESTABLISHED)
11 prometheu 332304 root 21u IPv4 29136125 0t0 TCP 10.0.11.239:48664->10.0.11.238:8051 (ESTABLISHED)
12 prometheu 332304 root 22u IPv4 29100979 0t0 TCP 10.0.11.239:48680->10.0.11.238:8051 (ESTABLISHED)
13 prometheu 332304 root 23u IPv4 29115255 0t0 TCP 10.0.11.239:48682->10.0.11.238:8051 (ESTABLISHED)
14 prometheu 332304 root 24u IPv4 29112130 0t0 TCP 10.0.11.239:38736->10.0.11.239:8051 (ESTABLISHED)
15 prometheu 332304 root 25u IPv4 29126119 0t0 TCP 10.0.11.239:38738->10.0.11.239:8051 (ESTABLISHED)
顺便提一下另一个命令fuser，如果希望查找挂载目录还有哪些进程使用或占用，可以使用-c选项，-c选项与-m选项类似，可以列出访问此文件系统的所有进程，-v选项为verbose。
(ENV) [root@ceph-2 mount]# fuser -cv /mount/
USER PID ACCESS COMMAND
/mount: root kernel mount /mount
root 17478 ..c.. bash
root 22141 F.c.. vim
扩展
查看系统的文件句柄数
(ENV) [root@ceph-2 ~]# cat /proc/sys/fs/file-max
1610634
(ENV) [root@ceph-2 ~]# cat /proc/sys/fs/file-nr
11168 0 1610634
(ENV) [root@ceph-2 ~]# sysctl -w fs.file-max=2000000
fs.file-max = 2000000
References
- df in linux not showing correct free space after file removal
- 如何查看linux中文件打开情况-lsof命令
- sysctl fs
- Number of file descriptors: different between /proc/sys/fs/file-nr and /proc/$pid/fd?
- Too many open files error
- 关于limits生效的四个案例
- Linux NFS Overview, FAQ and HOWTO Documents
- 为何root所使用的打开文件描述符的数量（或如何）超过ulimit -n？
