一、arthas基础使用
这里介绍arthas基于k8s使用方式。
1、首先Dockfile依赖arthas。
例如：
FROM openjdk:8
COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas
EXPOSE 8080
ARG JAR_FILE
ADD job.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]
2、构建镜像
docker build -t shujunjun.com/job:latest .
3、创建工作负载，略
4、查看运行的容器
kubectl get pods
4、进入容器内部
kubectl exec -it \${pod} --container \${containerId} -- /bin/bash
5、运行arthas
[root\@k8s-master \~]# kubectl exec -it job-8b59c98d6-hk4vt -- /bin/bash
运行时arthas会让你选择要监控的java程序，根据提示选择自己的程序id即可
root@sls600-control-service-8b59c98d6-hk4vt:/# java -jar /opt/arthas/arthas-boot.jar
[INFO] arthas-boot version: 3.5.5
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 1 app.jar
[2]: 388 arthas-boot.jar
1
[INFO] arthas home: /opt/arthas
[INFO] Try to attach process 1
[INFO] Attach process 1 success.
[INFO] arthas-client connect 127.0.0.1 3658
,---. ,------. ,--------.,--. ,--. ,---. ,---.
/ O \ | .--. ''--. .--'| '--' | / O \ ' .-'
| .-. || '--'.' | | | .--. || .-. |`. `-.
| | | || |\ \ | | | | | || | | |.-' |
`--' `--'`--' '--' `--' `--' `--'`--' `--'`-----'
wiki https://arthas.aliyun.com/doc
tutorials https://arthas.aliyun.com/doc/arthas-tutorials.html
version 3.5.5
main_class
pid 1
time 2024-04-13 13:27:50
出现以上信息代表arthas运行完成
二、基本命令使用
2.1、jvm
[arthas@1]$ jvm
RUNTIME
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MACHINE-NAME 1@sls600-control-service-8b59c98d6-hk4vt
JVM-START-TIME 2024-04-13 13:24:45
MANAGEMENT-SPEC-VERSION 1.2
SPEC-NAME Java Virtual Machine Specification
SPEC-VENDOR Oracle Corporation
SPEC-VERSION 1.8
VM-NAME OpenJDK 64-Bit Server VM
VM-VENDOR Oracle Corporation
VM-VERSION 25.312-b07
INPUT-ARGUMENTS -Djava.rmi.server.hostname=0.0.0.0
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.rmi.port=1199
-Dcom.sun.management.jmxremote.port=1199
-Dcom.sun.management.jmxremote.authenticate=false
-Duser.timezone=GMT+08
-Dcom.sun.management.jmxremote.ssl=false
CLASS-PATH app.jar
BOOT-CLASS-PATH /usr/local/openjdk-8/jre/lib/resources.jar:/usr/local/openjdk-8/jre/lib/rt.jar:/usr/local/openjdk-8/jre/lib/sunrsasign.jar:/usr/local/openjdk-8/jre/lib/jsse.jar:/us
r/local/openjdk-8/jre/lib/jce.jar:/usr/local/openjdk-8/jre/lib/charsets.jar:/usr/local/openjdk-8/jre/lib/jfr.jar:/usr/local/openjdk-8/jre/classes
LIBRARY-PATH /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CLASS-LOADING
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
LOADED-CLASS-COUNT 21960
TOTAL-LOADED-CLASS-COUNT 21960
UNLOADED-CLASS-COUNT 0
IS-VERBOSE false
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
COMPILATION
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
NAME HotSpot 64-Bit Tiered Compilers
TOTAL-COMPILE-TIME 26402
[time (ms)]
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
GARBAGE-COLLECTORS
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Copy name : Copy
[count/time (ms)] collectionCount : 202
collectionTime : 1378
MarkSweepCompact name : MarkSweepCompact
[count/time (ms)] collectionCount : 4
collectionTime : 228
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MEMORY
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
HEAP-MEMORY-USAGE init : 104857600(100.0 MiB)
[memory in bytes] used : 127068456(121.2 MiB)
committed : 149098496(142.2 MiB)
max : 1595473920(1.5 GiB)
NO-HEAP-MEMORY-USAGE init : 2555904(2.4 MiB)
[memory in bytes] used : 185784632(177.2 MiB)
committed : 195739648(186.7 MiB)
max : -1(-1 B)
PENDING-FINALIZE-COUNT 0
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATING-SYSTEM
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
OS Linux
ARCH amd64
PROCESSORS-COUNT 1
LOAD-AVERAGE 0.15
VERSION 3.10.0-1160.108.1.el7.x86_64
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
THREAD
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
COUNT 320
DAEMON-COUNT 54
PEAK-COUNT 334
STARTED-COUNT 1613
DEADLOCK-COUNT 0
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FILE-DESCRIPTOR
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MAX-FILE-DESCRIPTOR-COUNT 1048576
OPEN-FILE-DESCRIPTOR-COUNT 269
显示jvm的基本信息
其中的Copy表示eden区，MarkSweepCompact表示老年区。MEMORY，内容管理方式
2.2、thread命令（同jstack）
[arthas@1]$ thread
Threads Total: 325, NEW: 0, RUNNABLE: 47, BLOCKED: 0, WAITING: 251, TIMED_WAITING: 22, TERMINATED: 0, Internal threads: 5
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTED DAEMON
95 System Clock main 5 TIMED_WAITING 1.89 0.003 0:59.738 false true
93 arthas-command-execute system 5 RUNNABLE 1.03 0.002 0:0.062 false true
-1 C1 CompilerThread1 - -1 - 0.22 0.000 0:6.068 false true
99 RxComputationScheduler-1 main 5 TIMED_WAITING 0.07 0.000 0:1.433 false true
-1 VM Periodic Task Thread - -1 - 0.05 0.000 0:2.011 false true
25 redisson-timer-4-1 main 5 TIMED_WAITING 0.03 0.000 0:1.473 false false
-1 C2 CompilerThread0 - -1 - 0.0 0.000 0:18.020 false true
2 Reference Handler system 10 WAITING 0.0 0.000 0:0.027 false true
3 Finalizer system 8 WAITING 0.0 0.000 0:0.033 false true
4 Signal Dispatcher system 9 RUNNABLE 0.0 0.000 0:0.000 false true
9 RMI TCP Accept-1199 system 5 RUNNABLE 0.0 0.000 0:0.001 false true
10 RMI TCP Accept-0 system 5 RUNNABLE 0.0 0.000 0:0.000 false true
80 Attach Listener system 9 RUNNABLE 0.0 0.000 0:0.006 false true
82 arthas-timer system 9 WAITING 0.0 0.000 0:0.000 false true
85 arthas-NettyHttpTelnetBootstrap-3-1 system 5 RUNNABLE 0.0 0.000 0:0.026 false true
86 arthas-NettyWebsocketTtyBootstrap-4-1 system 5 RUNNABLE 0.0 0.000 0:0.000 false true
87 arthas-NettyWebsocketTtyBootstrap-4-2 system 5 RUNNABLE 0.0 0.000 0:0.000 false true
88 arthas-shell-server system 9 TIMED_WAITING 0.0 0.000 0:0.008 false true
89 arthas-session-manager system 9 TIMED_WAITING 0.0 0.000 0:0.003 false true
90 arthas-UserStat system 9 WAITING 0.0 0.000 0:0.000 false true
92 arthas-NettyHttpTelnetBootstrap-3-2 system 5 RUNNABLE 0.0 0.000 0:0.085 false true
217 RMI Scheduler(0) system 5 WAITING 0.0 0.000 0:0.000 false true
11 OkHttp ConnectionPool main 5 TIMED_WAITING 0.0 0.000 0:0.004 false true
12 Okio Watchdog main 5 TIMED_WAITING 0.0 0.000 0:0.021 false true
17 logback-3 main 5 WAITING 0.0 0.000 0:0.007 false true
20 webSocketReceiver main 5 RUNNABLE 0.0 0.000 0:0.552 false false
21 MQTT Rec: sls600-control-service-8b59c98d6-hk4vt main 5 TIMED_WAITING 0.0 0.000 0:0.233 false false
22 MQTT Snd: sls600-control-service-8b59c98d6-hk4vt main 5 WAITING 0.0 0.000 0:0.146 false false
23 MQTT Call: sls600-control-service-8b59c98d6-hk4vt main 5 WAITING 0.0 0.000 0:0.079 false false
24 MQTT Ping: sls600-control-service-8b59c98d6-hk4vt main 5 TIMED_WAITING 0.0 0.000 0:0.000 false false
26 redisson-netty-2-1 main 5 RUNNABLE 0.0 0.000 0:0.055 false false
27 redisson-netty-2-2 main 5 RUNNABLE 0.0 0.000 0:0.153 false false
28 redisson-netty-2-3 main 5 RUNNABLE 0.0 0.000 0:0.163 false false
29 redisson-netty-2-4 main 5 RUNNABLE 0.0 0.000 0:0.238 false false
30 redisson-netty-2-5 main 5 RUNNABLE 0.0 0.000 0:0.113 false false
31 redisson-netty-2-6 main 5 RUNNABLE 0.0 0.000 0:0.101 false false
32 redisson-netty-2-7 main 5 RUNNABLE 0.0 0.000 0:0.117 false false
33 redisson-netty-2-8 main 5 RUNNABLE 0.0 0.000 0:0.106 false false
34 redisson-netty-2-9 main 5 RUNNABLE 0.0 0.000 0:0.121 false false
35 redisson-netty-2-10 main 5 RUNNABLE 0.0 0.000 0:0.121 false false
36 redisson-netty-2-11 main 5 RUNNABLE 0.0 0.000 0:0.004 false false
37 redisson-netty-2-13 main 5 RUNNABLE 0.0 0.000 0:0.119 false false
38 redisson-netty-2-12 main 5 RUNNABLE 0.0 0.000 0:0.106 false false
39 redisson-netty-2-14 main 5 RUNNABLE 0.0 0.000 0:0.204 false false
40 redisson-netty-2-15 main 5 RUNNABLE 0.0 0.000 0:0.004 false false
41 redisson-netty-2-16 main 5 RUNNABLE 0.0 0.000 0:0.130 false false
42 redisson-netty-2-17 main 5 RUNNABLE 0.0 0.000 0:0.113 false false
43 redisson-netty-2-18 main 5 RUNNABLE 0.0 0.000 0:0.241 false false
2.2.1、如果想看某个线程堆栈调用情况。使用thread加线程号。
如：
[arthas@1]$ thread 407
"pool-3-thread-64" Id=407 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36d0c6c0
at sun.misc.Unsafe.park(Native Method)
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36d0c6c0
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2.3、dashboard(类似top命令)
[arthas@1]$ dashboard
ID NAME GROUP PRIORITY STATE %CPU DELTA_TIME TIME INTERRUPTED DAEMON
95 System Clock main 5 TIMED_WAITING 0.0 0.000 1:10.920 false true
-1 C2 CompilerThread0 - -1 - 0.0 0.000 0:20.293 false true
72 DestroyJavaVM main 5 RUNNABLE 0.0 0.000 0:8.187 false false
-1 C1 CompilerThread1 - -1 - 0.0 0.000 0:6.708 false true
-1 VM Thread - -1 - 0.0 0.000 0:3.924 false true
-1 VM Periodic Task Thread - -1 - 0.0 0.000 0:2.303 false true
25 redisson-timer-4-1 main 5 TIMED_WAITING 0.0 0.000 0:1.684 false false
99 RxComputationScheduler-1 main 5 TIMED_WAITING 0.0 0.000 0:1.632 false true
94 XNIO-1 task-1 main 5 WAITING 0.0 0.000 0:0.912 false false
97 pool-3-thread-2 main 5 WAITING 0.0 0.000 0:0.734 false false
20 webSocketReceiver main 5 RUNNABLE 0.0 0.000 0:0.640 false false
101 hystrix-goods-service-1 main 5 WAITING 0.0 0.000 0:0.328 false true
46 redisson-netty-2-21 main 5 RUNNABLE 0.0 0.000 0:0.320 false false
43 redisson-netty-2-18 main 5 RUNNABLE 0.0 0.000 0:0.313 false false
250 hystrix-goods-service-9 main 5 WAITING 0.0 0.000 0:0.309 false true
29 redisson-netty-2-4 main 5 RUNNABLE 0.0 0.000 0:0.307 false false
21 MQTT Rec: sls600-control-service-8b59c98d6-hk4vt main 5 TIMED_WAITING 0.0 0.000 0:0.288 false false
39 redisson-netty-2-14 main 5 RUNNABLE 0.0 0.000 0:0.280 false false
28 redisson-netty-2-3 main 5 RUNNABLE 0.0 0.000 0:0.203 false false
22 MQTT Snd: sls600-control-service-8b59c98d6-hk4vt main 5 WAITING 0.0 0.000 0:0.201 false false
92 arthas-NettyHttpTelnetBootstrap-3-2 system 5 RUNNABLE 0.0 0.000 0:0.196 false true
104 PollingServerListUpdater-0 main 5 TIMED_WAITING 0.0 0.000 0:0.192 false true
107 storage600-service-1 main 5 WAITING 0.0 0.000 0:0.192 false false
27 redisson-netty-2-2 main 5 RUNNABLE 0.0 0.000 0:0.190 false false
272 hystrix-goods-service-10 main 5 WAITING 0.0 0.000 0:0.177 false true
41 redisson-netty-2-16 main 5 RUNNABLE 0.0 0.000 0:0.170 false false
34 redisson-netty-2-9 main 5 RUNNABLE 0.0 0.000 0:0.161 false false
37 redisson-netty-2-13 main 5 RUNNABLE 0.0 0.000 0:0.159 false false
35 redisson-netty-2-10 main 5 RUNNABLE 0.0 0.000 0:0.156 false false
108 RxSchedulerPurge-1 main 5 TIMED_WAITING 0.0 0.000 0:0.156 false true
105 DataPublisher main 5 TIMED_WAITING 0.0 0.000 0:0.154 false true
32 redisson-netty-2-7 main 5 RUNNABLE 0.0 0.000 0:0.152 false false
42 redisson-netty-2-17 main 5 RUNNABLE 0.0 0.000 0:0.152 false false
55 redisson-netty-2-30 main 5 RUNNABLE 0.0 0.000 0:0.151 false false
49 redisson-netty-2-24 main 5 RUNNABLE 0.0 0.000 0:0.149 false false
51 redisson-netty-2-26 main 5 RUNNABLE 0.0 0.000 0:0.149 false false
Memory used total max usage GC
heap 108M 142M 1521M 7.14% gc.copy.count 212
eden_space 18M 39M 419M 4.42% gc.copy.time(ms) 1438
survivor_space 1M 4M 52M 3.71% gc.marksweepcompact.count 4
tenured_gen 88M 97M 1049M 8.40% gc.marksweepcompact.time(ms) 228
nonheap 183M 193M -1 94.78%
code_cache 52M 53M 240M 21.88%
metaspace 116M 124M -1 93.59%
compressed_class_space 14M 15M 1024M 1.40%
direct 183K 183K - 100.00%
mapped 0K 0K - 0.00%
Runtime
os.name Linux
os.version 3.10.0-1160.108.1.el7.x86_64
java.version 1.8.0_312
java.home /usr/local/openjdk-8/jre
systemload.average 0.03
processors 1
timestamp/uptime Sat Apr 13 14:38:13 GMT+08:00 2024/4408s
2.4、heapdump(类似jmap)
[arthas@1]$ heapdump
Dumping heap to /tmp/heapdump2024-04-13-14-407224920478052368735.hprof ...
Heap dump file created
2.4.1、指定下载路径
[arthas@1]$ heapdump /tmp/heapdump2024-04-13-14-407224920478052368735.hprof
2.4.2、下载日志(或者直接在k8s容器中分析)
[root@k8s-master ~]# kubectl cp job-8b59c98d6-hk4vt:/tmp /soft
tar: Removing leading `/' from member names
tar: /tmp/.java_pid1: socket ignored
2.4.2、分析日志
root@job-8b59c98d6-hk4vt:/tmp# jhat -J-mx512m /tmp/heapdump2024-04-13-14-407224920478052368735.hprof
Reading from /tmp/heapdump2024-04-13-14-407224920478052368735.hprof...
Dump file created Sat Apr 13 06:40:00 UTC 2024
Snapshot read, resolving...
Resolving 1996950 objects...
Chasing references, expect 399 dots...............................................................................................................................................................................................................................................................................................................................................................................................................
Eliminating duplicate references...............................................................................................................................................................................................................................................................................................................................................................................................................
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.
浏览器访问7000端口即可。
7000
也可以通过OQL语句查询类情况
OQL
查询
或者将该文件通过jvisualvm去分析。jvisualvm装入文件
jvisualvm分析.png
分析
