arthas基于k8s使用

一、arthas基础使用

这里介绍arthas基于k8s使用方式。

1、首先Dockfile依赖arthas。

例如:

FROM openjdk:8 
COPY --from=hengyunabc/arthas:latest /opt/arthas /opt/arthas 
EXPOSE 8080 
ARG JAR_FILE
ADD job.jar app.jar 
ENTRYPOINT ["java", "-jar", "app.jar"]

2、构建镜像

docker build -t shujunjun.com/job:latest .

3、创建工作负载,略

4、查看运行的容器

kubectl get pods

4、进入容器内部

kubectl exec -it \${pod} --container \${containerId} -- /bin/bash

5、运行arthas

[root\@k8s-master \~]# kubectl exec -it job-8b59c98d6-hk4vt -- /bin/bash

运行时arthas会让你选择要监控的java程序,根据提示选择自己的程序id即可

root@sls600-control-service-8b59c98d6-hk4vt:/# java -jar /opt/arthas/arthas-boot.jar
[INFO] arthas-boot version: 3.5.5
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.
* [1]: 1 app.jar
  [2]: 388 arthas-boot.jar
1
[INFO] arthas home: /opt/arthas
[INFO] Try to attach process 1
[INFO] Attach process 1 success.
[INFO] arthas-client connect 127.0.0.1 3658
  ,---.  ,------. ,--------.,--.  ,--.  ,---.   ,---.
 /  O  \ |  .--. ''--.  .--'|  '--'  | /  O  \ '   .-'
|  .-.  ||  '--'.'   |  |   |  .--.  ||  .-.  |`.  `-.
|  | |  ||  |\  \    |  |   |  |  |  ||  | |  |.-'    |
`--' `--'`--' '--'   `--'   `--'  `--'`--' `--'`-----'

wiki       https://arthas.aliyun.com/doc
tutorials  https://arthas.aliyun.com/doc/arthas-tutorials.html
version    3.5.5
main_class
pid        1
time       2024-04-13 13:27:50

出现以上信息代表arthas运行完成

二、基本命令使用

2.1、jvm

[arthas@1]$ jvm
 RUNTIME
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 MACHINE-NAME                                                       1@sls600-control-service-8b59c98d6-hk4vt
 JVM-START-TIME                                                     2024-04-13 13:24:45
 MANAGEMENT-SPEC-VERSION                                            1.2
 SPEC-NAME                                                          Java Virtual Machine Specification
 SPEC-VENDOR                                                        Oracle Corporation
 SPEC-VERSION                                                       1.8
 VM-NAME                                                            OpenJDK 64-Bit Server VM
 VM-VENDOR                                                          Oracle Corporation
 VM-VERSION                                                         25.312-b07
 INPUT-ARGUMENTS                                                    -Djava.rmi.server.hostname=0.0.0.0
                                                                    -Dcom.sun.management.jmxremote
                                                                    -Dcom.sun.management.jmxremote.rmi.port=1199
                                                                    -Dcom.sun.management.jmxremote.port=1199
                                                                    -Dcom.sun.management.jmxremote.authenticate=false
                                                                    -Duser.timezone=GMT+08
                                                                    -Dcom.sun.management.jmxremote.ssl=false
 CLASS-PATH                                                         app.jar
 BOOT-CLASS-PATH                                                    /usr/local/openjdk-8/jre/lib/resources.jar:/usr/local/openjdk-8/jre/lib/rt.jar:/usr/local/openjdk-8/jre/lib/sunrsasign.jar:/usr/local/openjdk-8/jre/lib/jsse.jar:/us
                                                                    r/local/openjdk-8/jre/lib/jce.jar:/usr/local/openjdk-8/jre/lib/charsets.jar:/usr/local/openjdk-8/jre/lib/jfr.jar:/usr/local/openjdk-8/jre/classes
 LIBRARY-PATH                                                       /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 CLASS-LOADING
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 LOADED-CLASS-COUNT                                                 21960
 TOTAL-LOADED-CLASS-COUNT                                           21960
 UNLOADED-CLASS-COUNT                                               0
 IS-VERBOSE                                                         false

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 COMPILATION
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 NAME                                                               HotSpot 64-Bit Tiered Compilers
 TOTAL-COMPILE-TIME                                                 26402
 [time (ms)]

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 GARBAGE-COLLECTORS
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Copy                                                               name : Copy
 [count/time (ms)]                                                  collectionCount : 202
                                                                    collectionTime : 1378
 MarkSweepCompact                                                   name : MarkSweepCompact
 [count/time (ms)]                                                  collectionCount : 4
                                                                    collectionTime : 228

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 MEMORY
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 HEAP-MEMORY-USAGE                                                  init : 104857600(100.0 MiB)
 [memory in bytes]                                                  used : 127068456(121.2 MiB)
                                                                    committed : 149098496(142.2 MiB)
                                                                    max : 1595473920(1.5 GiB)
 NO-HEAP-MEMORY-USAGE                                               init : 2555904(2.4 MiB)
 [memory in bytes]                                                  used : 185784632(177.2 MiB)
                                                                    committed : 195739648(186.7 MiB)
                                                                    max : -1(-1 B)
 PENDING-FINALIZE-COUNT                                             0

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 OPERATING-SYSTEM
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 OS                                                                 Linux
 ARCH                                                               amd64
 PROCESSORS-COUNT                                                   1
 LOAD-AVERAGE                                                       0.15
 VERSION                                                            3.10.0-1160.108.1.el7.x86_64

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 THREAD
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 COUNT                                                              320
 DAEMON-COUNT                                                       54
 PEAK-COUNT                                                         334
 STARTED-COUNT                                                      1613
 DEADLOCK-COUNT                                                     0

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 FILE-DESCRIPTOR
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 MAX-FILE-DESCRIPTOR-COUNT                                          1048576
 OPEN-FILE-DESCRIPTOR-COUNT                                         269

显示jvm的基本信息
其中的Copy表示eden区,MarkSweepCompact表示老年区。MEMORY,内容管理方式

2.2、thread命令(同jstack)

[arthas@1]$ thread
Threads Total: 325, NEW: 0, RUNNABLE: 47, BLOCKED: 0, WAITING: 251, TIMED_WAITING: 22, TERMINATED: 0, Internal threads: 5
ID        NAME                                                      GROUP                        PRIORITY            STATE              %CPU               DELTA_TIME          TIME               INTERRUPTED         DAEMON
95        System Clock                                              main                         5                   TIMED_WAITING      1.89               0.003               0:59.738           false               true
93        arthas-command-execute                                    system                       5                   RUNNABLE           1.03               0.002               0:0.062            false               true
-1        C1 CompilerThread1                                        -                            -1                  -                  0.22               0.000               0:6.068            false               true
99        RxComputationScheduler-1                                  main                         5                   TIMED_WAITING      0.07               0.000               0:1.433            false               true
-1        VM Periodic Task Thread                                   -                            -1                  -                  0.05               0.000               0:2.011            false               true
25        redisson-timer-4-1                                        main                         5                   TIMED_WAITING      0.03               0.000               0:1.473            false               false
-1        C2 CompilerThread0                                        -                            -1                  -                  0.0                0.000               0:18.020           false               true
2         Reference Handler                                         system                       10                  WAITING            0.0                0.000               0:0.027            false               true
3         Finalizer                                                 system                       8                   WAITING            0.0                0.000               0:0.033            false               true
4         Signal Dispatcher                                         system                       9                   RUNNABLE           0.0                0.000               0:0.000            false               true
9         RMI TCP Accept-1199                                       system                       5                   RUNNABLE           0.0                0.000               0:0.001            false               true
10        RMI TCP Accept-0                                          system                       5                   RUNNABLE           0.0                0.000               0:0.000            false               true
80        Attach Listener                                           system                       9                   RUNNABLE           0.0                0.000               0:0.006            false               true
82        arthas-timer                                              system                       9                   WAITING            0.0                0.000               0:0.000            false               true
85        arthas-NettyHttpTelnetBootstrap-3-1                       system                       5                   RUNNABLE           0.0                0.000               0:0.026            false               true
86        arthas-NettyWebsocketTtyBootstrap-4-1                     system                       5                   RUNNABLE           0.0                0.000               0:0.000            false               true
87        arthas-NettyWebsocketTtyBootstrap-4-2                     system                       5                   RUNNABLE           0.0                0.000               0:0.000            false               true
88        arthas-shell-server                                       system                       9                   TIMED_WAITING      0.0                0.000               0:0.008            false               true
89        arthas-session-manager                                    system                       9                   TIMED_WAITING      0.0                0.000               0:0.003            false               true
90        arthas-UserStat                                           system                       9                   WAITING            0.0                0.000               0:0.000            false               true
92        arthas-NettyHttpTelnetBootstrap-3-2                       system                       5                   RUNNABLE           0.0                0.000               0:0.085            false               true
217       RMI Scheduler(0)                                          system                       5                   WAITING            0.0                0.000               0:0.000            false               true
11        OkHttp ConnectionPool                                     main                         5                   TIMED_WAITING      0.0                0.000               0:0.004            false               true
12        Okio Watchdog                                             main                         5                   TIMED_WAITING      0.0                0.000               0:0.021            false               true
17        logback-3                                                 main                         5                   WAITING            0.0                0.000               0:0.007            false               true
20        webSocketReceiver                                         main                         5                   RUNNABLE           0.0                0.000               0:0.552            false               false
21        MQTT Rec: sls600-control-service-8b59c98d6-hk4vt          main                         5                   TIMED_WAITING      0.0                0.000               0:0.233            false               false
22        MQTT Snd: sls600-control-service-8b59c98d6-hk4vt          main                         5                   WAITING            0.0                0.000               0:0.146            false               false
23        MQTT Call: sls600-control-service-8b59c98d6-hk4vt         main                         5                   WAITING            0.0                0.000               0:0.079            false               false
24        MQTT Ping: sls600-control-service-8b59c98d6-hk4vt         main                         5                   TIMED_WAITING      0.0                0.000               0:0.000            false               false
26        redisson-netty-2-1                                        main                         5                   RUNNABLE           0.0                0.000               0:0.055            false               false
27        redisson-netty-2-2                                        main                         5                   RUNNABLE           0.0                0.000               0:0.153            false               false
28        redisson-netty-2-3                                        main                         5                   RUNNABLE           0.0                0.000               0:0.163            false               false
29        redisson-netty-2-4                                        main                         5                   RUNNABLE           0.0                0.000               0:0.238            false               false
30        redisson-netty-2-5                                        main                         5                   RUNNABLE           0.0                0.000               0:0.113            false               false
31        redisson-netty-2-6                                        main                         5                   RUNNABLE           0.0                0.000               0:0.101            false               false
32        redisson-netty-2-7                                        main                         5                   RUNNABLE           0.0                0.000               0:0.117            false               false
33        redisson-netty-2-8                                        main                         5                   RUNNABLE           0.0                0.000               0:0.106            false               false
34        redisson-netty-2-9                                        main                         5                   RUNNABLE           0.0                0.000               0:0.121            false               false
35        redisson-netty-2-10                                       main                         5                   RUNNABLE           0.0                0.000               0:0.121            false               false
36        redisson-netty-2-11                                       main                         5                   RUNNABLE           0.0                0.000               0:0.004            false               false
37        redisson-netty-2-13                                       main                         5                   RUNNABLE           0.0                0.000               0:0.119            false               false
38        redisson-netty-2-12                                       main                         5                   RUNNABLE           0.0                0.000               0:0.106            false               false
39        redisson-netty-2-14                                       main                         5                   RUNNABLE           0.0                0.000               0:0.204            false               false
40        redisson-netty-2-15                                       main                         5                   RUNNABLE           0.0                0.000               0:0.004            false               false
41        redisson-netty-2-16                                       main                         5                   RUNNABLE           0.0                0.000               0:0.130            false               false
42        redisson-netty-2-17                                       main                         5                   RUNNABLE           0.0                0.000               0:0.113            false               false
43        redisson-netty-2-18                                       main                         5                   RUNNABLE           0.0                0.000               0:0.241            false               false

2.2.1、如果想看某个线程堆栈调用情况。使用thread加线程号。
如:

[arthas@1]$ thread 407
"pool-3-thread-64" Id=407 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36d0c6c0
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36d0c6c0
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
    at java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

2.3、dashboard(类似top命令)

[arthas@1]$ dashboard
ID        NAME                                                      GROUP                        PRIORITY            STATE              %CPU               DELTA_TIME          TIME               INTERRUPTED         DAEMON
95        System Clock                                              main                         5                   TIMED_WAITING      0.0                0.000               1:10.920           false               true
-1        C2 CompilerThread0                                        -                            -1                  -                  0.0                0.000               0:20.293           false               true
72        DestroyJavaVM                                             main                         5                   RUNNABLE           0.0                0.000               0:8.187            false               false
-1        C1 CompilerThread1                                        -                            -1                  -                  0.0                0.000               0:6.708            false               true
-1        VM Thread                                                 -                            -1                  -                  0.0                0.000               0:3.924            false               true
-1        VM Periodic Task Thread                                   -                            -1                  -                  0.0                0.000               0:2.303            false               true
25        redisson-timer-4-1                                        main                         5                   TIMED_WAITING      0.0                0.000               0:1.684            false               false
99        RxComputationScheduler-1                                  main                         5                   TIMED_WAITING      0.0                0.000               0:1.632            false               true
94        XNIO-1 task-1                                             main                         5                   WAITING            0.0                0.000               0:0.912            false               false
97        pool-3-thread-2                                           main                         5                   WAITING            0.0                0.000               0:0.734            false               false
20        webSocketReceiver                                         main                         5                   RUNNABLE           0.0                0.000               0:0.640            false               false
101       hystrix-goods-service-1                                   main                         5                   WAITING            0.0                0.000               0:0.328            false               true
46        redisson-netty-2-21                                       main                         5                   RUNNABLE           0.0                0.000               0:0.320            false               false
43        redisson-netty-2-18                                       main                         5                   RUNNABLE           0.0                0.000               0:0.313            false               false
250       hystrix-goods-service-9                                   main                         5                   WAITING            0.0                0.000               0:0.309            false               true
29        redisson-netty-2-4                                        main                         5                   RUNNABLE           0.0                0.000               0:0.307            false               false
21        MQTT Rec: sls600-control-service-8b59c98d6-hk4vt          main                         5                   TIMED_WAITING      0.0                0.000               0:0.288            false               false
39        redisson-netty-2-14                                       main                         5                   RUNNABLE           0.0                0.000               0:0.280            false               false
28        redisson-netty-2-3                                        main                         5                   RUNNABLE           0.0                0.000               0:0.203            false               false
22        MQTT Snd: sls600-control-service-8b59c98d6-hk4vt          main                         5                   WAITING            0.0                0.000               0:0.201            false               false
92        arthas-NettyHttpTelnetBootstrap-3-2                       system                       5                   RUNNABLE           0.0                0.000               0:0.196            false               true
104       PollingServerListUpdater-0                                main                         5                   TIMED_WAITING      0.0                0.000               0:0.192            false               true
107       storage600-service-1                                      main                         5                   WAITING            0.0                0.000               0:0.192            false               false
27        redisson-netty-2-2                                        main                         5                   RUNNABLE           0.0                0.000               0:0.190            false               false
272       hystrix-goods-service-10                                  main                         5                   WAITING            0.0                0.000               0:0.177            false               true
41        redisson-netty-2-16                                       main                         5                   RUNNABLE           0.0                0.000               0:0.170            false               false
34        redisson-netty-2-9                                        main                         5                   RUNNABLE           0.0                0.000               0:0.161            false               false
37        redisson-netty-2-13                                       main                         5                   RUNNABLE           0.0                0.000               0:0.159            false               false
35        redisson-netty-2-10                                       main                         5                   RUNNABLE           0.0                0.000               0:0.156            false               false
108       RxSchedulerPurge-1                                        main                         5                   TIMED_WAITING      0.0                0.000               0:0.156            false               true
105       DataPublisher                                             main                         5                   TIMED_WAITING      0.0                0.000               0:0.154            false               true
32        redisson-netty-2-7                                        main                         5                   RUNNABLE           0.0                0.000               0:0.152            false               false
42        redisson-netty-2-17                                       main                         5                   RUNNABLE           0.0                0.000               0:0.152            false               false
55        redisson-netty-2-30                                       main                         5                   RUNNABLE           0.0                0.000               0:0.151            false               false
49        redisson-netty-2-24                                       main                         5                   RUNNABLE           0.0                0.000               0:0.149            false               false
51        redisson-netty-2-26                                       main                         5                   RUNNABLE           0.0                0.000               0:0.149            false               false
Memory                                            used             total            max             usage            GC
heap                                              108M             142M             1521M           7.14%            gc.copy.count                                             212
eden_space                                        18M              39M              419M            4.42%            gc.copy.time(ms)                                          1438
survivor_space                                    1M               4M               52M             3.71%            gc.marksweepcompact.count                                 4
tenured_gen                                       88M              97M              1049M           8.40%            gc.marksweepcompact.time(ms)                              228
nonheap                                           183M             193M             -1              94.78%
code_cache                                        52M              53M              240M            21.88%
metaspace                                         116M             124M             -1              93.59%
compressed_class_space                            14M              15M              1024M           1.40%
direct                                            183K             183K             -               100.00%
mapped                                            0K               0K               -               0.00%
Runtime
os.name                                                                                                              Linux
os.version                                                                                                           3.10.0-1160.108.1.el7.x86_64
java.version                                                                                                         1.8.0_312
java.home                                                                                                            /usr/local/openjdk-8/jre
systemload.average                                                                                                   0.03
processors                                                                                                           1
timestamp/uptime                                                                                                     Sat Apr 13 14:38:13 GMT+08:00 2024/4408s

2.4、heapdump(类似jmap)

[arthas@1]$ heapdump
Dumping heap to /tmp/heapdump2024-04-13-14-407224920478052368735.hprof ...
Heap dump file created

2.4.1、指定下载路径

[arthas@1]$ heapdump /tmp/heapdump2024-04-13-14-407224920478052368735.hprof

2.4.2、下载日志(或者直接在k8s容器中分析)

[root@k8s-master ~]# kubectl cp job-8b59c98d6-hk4vt:/tmp /soft
tar: Removing leading `/' from member names
tar: /tmp/.java_pid1: socket ignored

2.4.2、分析日志

root@job-8b59c98d6-hk4vt:/tmp# jhat -J-mx512m /tmp/heapdump2024-04-13-14-407224920478052368735.hprof
Reading from /tmp/heapdump2024-04-13-14-407224920478052368735.hprof...
Dump file created Sat Apr 13 06:40:00 UTC 2024
Snapshot read, resolving...
Resolving 1996950 objects...
Chasing references, expect 399 dots...............................................................................................................................................................................................................................................................................................................................................................................................................
Eliminating duplicate references...............................................................................................................................................................................................................................................................................................................................................................................................................
Snapshot resolved.
Started HTTP server on port 7000
Server is ready.

浏览器访问7000端口即可。


7000

也可以通过OQL语句查询类情况


OQL

查询

或者将该文件通过jvisualvm去分析。jvisualvm装入文件


jvisualvm分析.png

分析
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容