性能测试流程及相关概念
本文整理来自pro Apache jmter web application和Wikipedia
为什么做性能测试？
- 改善用户体验
- 提高用户留存率
- 品牌宣传
什么是性能测试
Performance testing is the testing performed on a system or application to measure some of its attributes such as response time, throughput etc
在一定的负载条件下，评估系统或应用的响应时间，吞吐量等相关指标
什么时候执行性能测试
功能测试通过之后
性能测试的目的
对系统进行性能评估，并通过测试发现系统瓶颈，分析定位瓶颈发生的原因，协助开发修复性能问题，提高系统的处理能力。对比不同的系统，确认更好的系统。
性能测试的指标（特性）
主要指标： 响应时间、吞吐量、资源利用率。
a. 响应时间
1. 定义
The time taken by the application to respond to the user’s request is called the response time.Every application should strive to minimize the response time
应用程序响应用户请求所需的时间称之为--响应时间。响应时间越小越好
对于常见的B/S结构的响应时间，如下图：
2. Absolute Response Time
Absolute Response Time: This is the total time taken from the instant a user clicks on a link or submits a form until the response from the server is rendered completely
从用户发起请求，到服务完全完成响应的时间。
3. Perceived Response Time
Perceived Response Time: The absolute response time for some complex web pages may be unacceptably high. To solve this issue,the request/response is structured into a set of request/responses that could be rendered progressively. This allows the users to continue reading or filling up a form, creating the perception that the response was fast even though the absolute response time is roughly the same.
Perceived response time is the response time as perceived by the users.
用户感知时间，有点预加载的意思。即本来一个页面全部信息同时返回，现在改成优先加载一部分内容
4. Server Processing Time
Server Processing Time: This is the time taken by the server to process the given input and generate the output. This can vary depending on the complexity of the request, server hardware, and the system load.
服务器处理时间。服务器处理请求并响应的这段时间。与请求的复杂性、系统的硬件及系统服负载有关。
5. Rendering Time
Rendering Time: This is the time taken by the browser to parse and render the response received from the server. This depends on the complexity of the web page, presence of multimedia, presence of JavaScript, and whether the browser needs to generate content. It also depends on the system load on the user’s device.
6. Network Latency
Network Latency: This is the round-trip time taken by a data packet to travel over the network the server and back. This does not include the server processing time or the rendering time. This can vary widely depending on the location of the user, time of the day, and the network load.
b. Throughput
A sequence of request/responses constitutes a transaction. The number of transactions per unit time is called the throughput. It is measured in transactions/second or bandwidth (bytes/second). It depends on server hardware, system load, and network latency. The application should strive to maximize the throughput.
吞吐量：单位时间内处理的事务个数。通常由transactions/second 或 bandwidth (bytes/second)来衡量。吞吐量越大越好。
c. Utilization
Utilization is the ratio of the throughput of the application relative to its maximum capacity. This is a measure of how well the application is being used. It is not desirable to operate above 80% utilization because the user requests do not arrive evenly, and the system should be able to handle a spike in the load.
服务器资源利用率，不应该超过80%的利用率。利用率越低越好。
d. Scalability
Scalability measures how well the system can expand its capacity when additional resources are added.Ideally the system capacity will increase linearly as additional resources are added. However, this is rarely achieved in practice. It’s a good measure to know the resources that would be needed so that the system can handle the projected future load.Vertical scalability is achieved by upgrading the hardware. For example, by adding more memory, disk,
a better CPU, or additional CPUs.Horizontal scalability is achieved by adding servers to the cluster. For example, by adding more web
servers and application servers to a webfarm/cluster.
性能测试类型
主要有压力测试，负载测试，配置测试等
1. Stress Tests
A stress test is a kind of performance test that tests the application beyond the normal limits. The application is subjected to excess load and after that its stability and performance are noted. This type of test is used to determine how the application responds to load spikes
压力测试是应用程序处于超出负荷状态下长时间运行，以测试服务器的稳定性和性能表现。
Note
performance tests evaluate and measure the application under a normal expected load. the stress test subjects the application to loads which are in far excess of normal.
2. Load Tests
A load test is a kind of performance test that’s performed at the specified load level. So ideally, we would like to perform load tests at varying load levels to note the behavior of the application.
在一定的负载条件下，测试应用程序的性能表现。通过不同的负载程度（load levels）,观察应用程序的性能表现来得到最大的用户数，响应时间以及资源利用率等指标。
3. Peak Load Tests
A peak load test is performed at the load that the application is expected to handle. For example, e-commerce web sites experience their peak traffic during Black Friday, Cyber Monday, and the Christmas holidays. So a peak load test in this case would test the application within the load specification but at the higher end.
Note stress tests test beyond the peak load.
4. Soak Tests or Endurance Tests
In a soak test (also called an endurance test), the application is subjected to a specified load that is within the specified limit but for a long duration. It is performed for many hours at a time. This test determines if the application is properly reusing its resources.
This test will surface problems like the following:
- Memory leaks in the application
- Database connections exhaustion
- Network connection exhaustion
- Log files becoming full and log rotation
- Other resource exhaustion
5. Scalability Tests
Successful web applications experience massive and sometimes exponential growth. So it is wise to measure how the application scales. Scalability is defined as how well the application handles the increase in load while still meeting the desired performance criteria.
A scalability test would increase the resources and test whether or not the application is providing
a corresponding increase in capacity. Ideally, we expect linear scalability (i.e., doubling the hardware resources should result in double the application capacity).
6. Capacity Tests
A capacity test is a load test that establishes the maximum load that the application can handle while
meeting the desired performance criteria. The resulting metric is called the maximum capacity. It is used in scaling the application and to estimate costs for future growth.
7. Spike Tests and Burst Capacity
A spike test is a load test where the application is subjected to brief periods of sudden increment in load, a small fraction beyond the maximum capacity. It is usually done to estimate the weakness/strength of an application. The application is expected to be robust and continue to meet the performance criteria during the spike. This metric is called the burst capacity.
8. Performance Smoke Tests
In a performance smoke test, a few common and essential use-cases along with use-cases pertaining to the
code subject to change are together tested for performance. It is only when the smoke test succeeds that
the full suite of performance tests are conducted. If the smoke test fails, no further performance tests areconducted until the performance defect has been rectified.
9. High Availability Test/Fail-Over Tests
Modern web application infrastructure is designed to be highly available and resilient to hardware and software failures. Ideally, the architecture should ensure that there is no single point of failure and that there are standby servers that can transparently take over without impacting the user experience.
In this test various equipment and software failures are simulated and relevant performance tests are
run to verify that the application is still meeting the performance criteria.
10. Configuration testing
Rather than testing for performance from a load perspective, tests are created to determine the effects of configuration changes to the system's components on the system's performance and behaviour. A common example would be experimenting with different methods of load-balancing
性能测试流程
