Do you know what is a capacity of your website? It’s a tricky question to ask because what does it really mean? Every website has multiple user journeys. Each of them come with a different cost. The cost might be different for a different dataset or a software version. A good answer to this question requires solid investigation and the right tools.
For a long time my favorite weapon of choice was “ab”. The Apache Benchmark is a great small tool but it has limitation. It’s designed to test only one URL at a time. There is a problem with that. If you stress only one entry point you might get an inaccurate reading. For example, lets imagine a script which is executing a very heavy query. The first request is going to be slow because a database hasn’t cached it yet. After results are cached every following request will take significantly less time. The “ab” will return some numbers but those will be only correct for this particular request. If the same query can be run with a different parameters and in fact this is how it’s used in the production you might experience completely different performance.
Apache JMeter is an open source software written in Java. It’s designed to load test functional behavior and measure performance. It might be use to test performance, simulate a heavy load on a server, network or object.
To demonstrate you how powerful and easy in use is the JMeter lets create a web crawler. This script will visit www.reddit.com and will go though all comments pages.
As always the first step is to download the software. JMeter is written in Java so make sure you have Java runtime installed. Visit http://jmeter.apache.org/download_jmeter.cgi and get the latest binaries.
Unpack downloaded archive, go to bin directory and execute the JMeter. You should see an empty test. If you want to see it in action open this example and click “play” button. To see what is the JMeter up to click on “View Results Tree” item in the left hand side panel. If you can’t see it you might need to expand the test items. Click on “Test Plan / Front Page Crawler / View Results Tree”.
Ok, allow me talk you though the used components and configuration.
1. Config Element / HTTP Request Defaults
This is a default configuration for HTTP requests. I set server name to “www.reddit.com”. It’s not required but very handy when you want to point your test to a different server.
2. Threads (Users) / Thread Group
A Thread Group defines a pool of users that will execute a particular test case against your server. Every test case should be defined in a separate Thread Group. I set number of threads and loops to 1 because I don’t want to DDoS the website but in real live benchmark you need some concurrency. For Apache Benchmark I usually go for 20-100 concurrent connections and 500-1000 loops. It might not be a good idea with JMeter. It’s resources hungry and your desktop might be not able to perform that many requests. There are some workarounds to this issue. I will mention one at point 9.
3. Sampler / HTTP Request
Request a home page. Path is set to “/”.
4. Post Processors / Regular Expression Extractor
This one is interesting. In order to craw though the latest links we need to know what are those links. I create a regular expression
<a class="comments.*?href="([^"]+)"
which looks for href value in anchors with “comments” class. Reference name is set to “links” which is where the URLs will be stored. The template set to “$1$” defines how the variable’s value is going to be build. Match number is “-1” which means we want an array of all values.
5. Sampler / Debug Sampler
This component is not required but you are going to use it a lot at the beginning. It will allow you to look inside variables and properties.
6. Logic Controller / ForEach controller
Regular expression extractor will return an array of links. ForEach is going to iterate though all them. Each of the links is going to be stored in “link” variable. This is defined in the “output variable name” field.
7. Sampler / HTTP Request
This sampler will visit each of the comments page. Path is set to “${link}” which is how you refer to a variable. I also put it in the name to have a better output in the “View Results Tree”.
8. Listener / View Results Tree
I use this listener for debugging. It gives you real time details about Thread Group actions. The screenshot shows “debug sampler” output. You can see variables created by the regular expression extractor.
9. Listener / Summary report
The summary report creates a table row for each differently named request in your test. This is where you look for total request per second or average load time.
Ironically JMeter is criticized for poor reporting components. It’s ironic because that’s what you really need from a benchmark. Don’t worry, It’s not that bad as it sounds. You can download JMeter plugins which are partially sponsored by BlazeMeter. The extension will bring some more listeners and other components. If that’s still not enough for you try the BlazeMeter. You will be charged for their service but they offer quality reports and lots of CPU and broadband for even the most demanding benchmarks.
I strongly recommend you to get familiar with JMeter. It’s a very useful tool. I would call it a “Swiss army knife” for testing websites. Even if you don’t measure performance you can use it for other things like continues integration. It can act as a functional testing tool and even QA officers can use it.
