环境
CentOS 7.2
JMeter 3.2
Topologie
- (Client|Master): the system running Jmeter which controls the slave remote (and doesn't send any request)
- (Server|Slave|Remote) Node: the system running JMeter in server mod (jmeter-server), which takes commands from the client and send the test plan requests to the target system(s)
- Target: the server to stress test
Configure Client Node
JMeter uses Java RMI [Remote Method Invocation] to interact with objects in a distributed network.
JMeter master and slave communicate as shown in the below picture.
remote_hosts: in client JMeter machine,add value of JMeter servers IP address. Multiple servers can be added comma-delimited. A more convenient way to specify remote host is from command line. Use -R command line to specify which remote host to use
client.rmi.localport: Specify client.rmi.localport in jmeter.properties file (for example - client.rmi.localport=25000). This is the port on which the local JMeter instance will listen for the Remote Method Invocation (i.e. RMI connections) from the slave nodes. Results would be sent to client from slave nodes on this port
Configure Server Nodes
Server nodes should be running same version of JMeter and java as client
Test datafiles are not sent across by client, hence they should be available in appropriate directory in each server. You can keep test data file in /bin folder in each JMeter server. It gets you rid of specifying full qualified path of test data file on each server instance.
To use different values for properties use corresponding values for user.properties and system.properties on each server
- server.rmi.localport: By default, RMI uses a dynamic port for the JMeter server engine. This may cause issues with firewalls, so with versions of JMeter after 2.3.2 you can define the JMeter property server.rmi.localport to control this port number. To use a specific port for the JMeter server engine, define server.rmi.localport property before starting the server
JMeter/RMI requires:
a connection from the client to the server. Default port 1099.
and a reverse connection in order to return results from the server to the client. This will use a high-numbered port controlled by jmeter property “client.rmi.localport”
Prerequisites
All the nodes (client and servers):
- are running the same version of JMeter.
- are running the same version of Java. For JMeter 2.9, Java 6 runtime or higher
- have extra test data files available in the appropriate directory
are on the same subnet - can communicate (the firewalls must be turned off of set up to allow the connections)
- must be started with the same property (file and command line option)
JMeter安装
下载解压即可。
wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.2.tgz
tar -xzf apache-jmeter-3.2.tgz
JMeter命令
-h 帮助 -> 打印出有用的信息并退出
-n 非 GUI 模式 -> 在非 GUI 模式下运行 JMeter
-t 测试文件 -> 要运行的 JMeter 测试脚本文件
-l 日志文件 -> 记录结果的文件
-r 远程执行 -> 启动远程服务
-H 代理主机 -> 设置 JMeter 使用的代理主机
-P 代理端口 -> 设置 JMeter 使用的代理主机的端口号
$ jmeter -n -t test1.jmx -l logfile1.jtl -H 192.168.1.1 -P 8080
方案1
client,server均采用手动部署方式，方便调试查错。
启动server：
[root@VM_1_111_centos bin]# ./jmeter-server -Djava.rmi.server.hostname=10.0.1.111
Created remote object: UnicastServerRef [liveRef: [endpoint:[10.0.1.111:36522](local),objID:[-77441bd0:15bf138f424:-7fff, -7104802513269301355]]]
启动client：
[root@VM_1_246_centos bin]# ./jmeter -n -t baidu-test.jmx -R10.0.1.111:31099 -Djava.rmi.server.hostname=10.0.1.246
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.1.111:31099
Starting remote engines
Starting the test @ Wed May 10 15:19:58 CST 2017 (1494400798945)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 1 in 00:00:01 = 1.9/s Avg: 374 Min: 374 Max: 374 Err: 0 (0.00%) Active: 1 Started: 1 Finished: 0
summary = 1 in 00:00:01 = 1.9/s Avg: 374 Min: 374 Max: 374 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 15:20:00 CST 2017 (1494400800718)
... end of run
可以看到server上的调用信息：
Starting the test on host 10.0.1.111:31099 @ Wed May 10 15:19:59 CST 2017 (1494400799839)
Finished the test on host 10.0.1.111:31099 @ Wed May 10 15:20:00 CST 2017 (1494400800740)
方案2
压测需要部署大量server节点，采用docker部署。
生成JMeter base镜像需要的Dockerfile如下：
# Use Ubuntu
FROM ubuntu
MAINTAINER adeng <xxxxxx@qq.com>
# Install wger & JRE
RUN apt-get clean && \
apt-get update && \
apt-get -qy install \
wget \
default-jre-headless \
telnet \
iputils-ping \
unzip
# Install jmeter
RUN mkdir /jmeter \
&& cd /jmeter/ \
&& wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-3.2.tgz \
&& tar -xzf apache-jmeter-3.2.tgz \
&& rm apache-jmeter-3.2.tgz \
&& mkdir /jmeter-plugins \
&& cd /jmeter-plugins/ \
&& wget https://jmeter-plugins.org/downloads/file/JMeterPlugins-ExtrasLibs-1.4.0.zip \
&& unzip -o JMeterPlugins-ExtrasLibs-1.4.0.zip -d /jmeter/apache-jmeter-3.2/
# Set Jmeter Home
ENV JMETER_HOME /jmeter/apache-jmeter-3.2/
# Add Jmeter to the Path
ENV PATH $JMETER_HOME/bin:$PATH
生成JMeter slave镜像需要的Dockerfile如下：
FROM malfurionpd/jmeter-base
MAINTAINER adeng <xxxxxx@qq.com>
# Ports to be exposed from the container for JMeter Slaves/Server
EXPOSE 31099 32500
# Application to run on starting the container
ENTRYPOINT $JMETER_HOME/bin/jmeter-server \
-Dserver.rmi.localport=32500 \
-Dserver_port=31099
可以参考这里在Dockerhub上生成镜像，或者本地生成：
docker build -t malfurionpd/jmeter-slave .
部署三个容器，主机端口直接映射容器expose的端口，client直接远程调用：
[root@VM_1_246_centos bin]# ./jmeter -n -t baidu-test.jmx -R10.0.2.104:31099,10.0.3.248:31099,10.0.3.180:31099 -Djava.rmi.server.hostname=10.0.1.246
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.2.104:31099
Configuring remote engine: 10.0.3.248:31099
Configuring remote engine: 10.0.3.180:31099
Starting remote engines
Starting the test @ Wed May 10 17:09:50 CST 2017 (1494407390007)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary = 2 in 00:00:01 = 3.7/s Avg: 174 Min: 112 Max: 236 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 17:09:51 CST 2017 (1494407391402)
summary + 1 in 00:00:01 = 0.8/s Avg: 721 Min: 721 Max: 721 Err: 0 (0.00%) Active: 0 Started: 2 Finished: 3
summary = 3 in 00:00:02 = 1.7/s Avg: 356 Min: 112 Max: 721 Err: 0 (0.00%)
Tidying up remote @ Wed May 10 17:09:52 CST 2017 (1494407392649)
... end of run
... end of run
方案3
参考这里，使用Rancher部署测试集群。
坑1
直接执行jmeter-server报错：
[root@localhost bin]# ./jmeter-server
Created remote object: UnicastServerRef [liveRef: [endpoint:[127.0.0.1:39150](local),objID:[-3126fe29:14300b1102e:-7fff, -4078314045196249121]]]
Server failed to start: java.rmi.RemoteException: Cannot start. localhost is a loopback address.
An error occurred: Cannot start. localhost is a loopback address.
解决方法如下:
In latest version, you can run your script with:指定本地IP
./jmeter-server -Djava.rmi.server.hostname=xxx.xxx.xxx.xxx
坑2
使用全容器部署时，不指定Host IP情况下 RMI总是出问题:
root@VM_7_2_centos:/jmeter/apache-jmeter-3.2/bin# ./jmeter -n -t baidu-test.jmx -Djava.rmi.server.hostname=10.0.7.2 -Dclient.rmi.localport=60000 -R10.0.7.14,10.0.7.7
Creating summariser <summary>
Created the tree successfully using baidu-test.jmx
Configuring remote engine: 10.0.7.14
Configuring remote engine: 10.0.7.7
Starting remote engines
Starting the test @ Tue Jun 13 12:59:01 UTC 2017 (1497358741786)
Error in rconfigure() method java.rmi.ConnectIOException: Exception creating connection to: 10.42.37.96; nested exception is:
java.net.NoRouteToHostException: No route to host (Host unreachable)
Error in rconfigure() method java.rmi.ConnectIOException: Exception creating connection to: 10.42.255.116; nested exception is:
java.net.NoRouteToHostException: No route to host (Host unreachable)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
解决方法和上边一样，必须想办法指定 ./jmeter-server -Djava.rmi.server.hostname=xxx.xxx.xxx.xxx 。
参考文章1
参考文章2
参考文章3
参考文章4
参考文章5
参考Docker三部曲之1
参考Docker三部曲之2
参考Docker三部曲之3
