In internet system stress testing, we frequently encounter two tools with vastly different styles: one is extremely lightweight, pursuing extreme throughput—wrk; the other is feature-rich and simulates real business flows—JMeter.
Prompt: Outline the core ideas and write a科普 article (explanatory article): HTTP stress testing tools, wrk vs JMeter – what are the differences? What I know, wrk tends to use one thread with multiple connections for testing, while JMeter primarily employs a short connection mode, which can be adjusted via configuration to enable long polling.
Core Architecture: Multi-Threaded vs. Event-Driven
This is the fundamental reason for the performance gap between them.
1. JMeter: The “One Person, One Job” (Thread-per-Request) Model
JMeter is developed in Java and utilizes the classic multi-threaded model.
- Logic: Each concurrent user (Virtual User) corresponds to a physical thread within the JVM.
- Cost: Threads are an expensive resource. As concurrency increases to several thousand, context switching and memory consumption will significantly slow down the test machine itself, leading to the phenomenon of “the load generator collapsing first before it can actually crush the server.”
2. wrk: The Modern “Multi-Hand” System (Event-driven)
wrk is written in C and its core logic relies on Redis’s ae event loop framework (utilizing epoll/kqueue).
- Logic: wrk doesn’t create a thread for each connection. It only starts a small number of threads (typically equal to the number of your CPU cores), and each thread manages thousands upon thousands of connections simultaneously through non-blocking I/O.
- Advantages: This is what you referred to as “one thread, multiple connections.” It drastically reduces thread switching overhead, allowing single machines to achieve millions of RPS (Requests Per Second).
Connection Models: Skip Connections vs. Dense Connections
Regarding the connection patterns you mentioned, here’s a deeper level of detail:
1. JMeter’s “Heavy” and “Light”
JMeter defaults to simulating real user behavior.
- Short Connection Bias: In default configurations, some older versions or specific configurations of JMeter may not actively reuse connections, leading to numerous TCP handshakes.
- Tunability: You can enable long connection by checking the “KeepAlive” option in the
HTTP Requestor adjusting connection pool parameters in theuser.propertiesfile. However, even with long connections, limited by its thread model, it struggles to maintain tens of thousands of concurrent long connections.
2. wrk’s “Speed” and “Power”
wrk was designed with the intention of testing the performance of HTTP Keep-Alive.
- Long Connection Strategy: wrk establishes a specified number of connections at the start of the test (
-cparameter) and attempts to reuse these connections throughout the entire test. - Application Scenarios: It’s particularly well-suited for testing the throughput limits of Nginx, gateways (Gateways), or high-concurrency APIs under extreme long connection pressure.
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Development Language | C/Lua (Scripting) | Java (GUI) |
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Concurrency Model | Event Driven (epoll/kqueue) | Multi-threaded (Thread-per-User) |
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Resource Consumption | Extremely Low, Huge Throughput on a Single Machine | Higher, Requires Distributed Cluster for High Concurrency |
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Business Complexity | Low, primarily for single URLs | High, supports multi-step scripts, assertions, and extractors |
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Test Scenarios | Static API Load Testing, Capacity Assessment | Complex Business Link Simulations, Functional Regression Testing |
Deep Comparison Table
| Feature | wrk | Apache JMeter |
|---|---|---|
| Reporting Capabilities | Only text summaries | Extremely rich, supports various charts and HTML reports |
Summary: Which one should I choose?
These two tools are complementary rather than substitutional relationships:
Select Work
- Want to test the maximum throughput (RPS) of the server.
- The testing object is a single API or static resource.
- Aim to push the largest traffic using the fewest test servers.
- Familiar with Lua scripting to customize requests.
Choose JMeter
- Requires simulating complex business processes (such as: Login -> Search Products -> Place Order -> Payment).
- Needs a visual interface to observe response time distributions, error rates, and other detailed metrics.
- Testing requires handling dynamic parameters (such as extracting a Token from the previous interface and passing it to the next interface).
- The team is more accustomed to using graphical tools rather than command-line interfaces.