OK, so here we are. The home stretch. If you manage to complete this post you can officially consider “JMeter” part of your skillset and brag about it with your coworkers. You have my permission at that point(sic). The goal of this post is to finally have our payoff by creating non-trivial test plans for a variety of scenarios to show how you can more fully leverage JMeter for productive ends.
Contents
How to Approach the Load Generation Problem
Probably the biggest obstacle with going from the abstract desire to start working with actual numbers to where you have a realistic test plan that works the way you want it is purely mental. Very rarely will you legitimately run into a JMeter issue. The usual obstacle is not understanding how the web application works or how to leverage the JMeter components to meet the application halfway.
REST API’s are the simplest because their interfaces are usually very well documented and they’re intended to be interacted with programmatically. That leaves the troublesome applications being the likes of HTTP or a database server.
For HTTP web applications, most of the time you can to go through the steps you would through a web browser and break down the relevant series <form> elements and re-constituting those HTML forms as HTTP samplers. For multi-stage forms this means examining all elements for hidden fields that you can extract using an XPath or CSS extractor.
Another obstacle is for load testing elements lower in the application stack such as databases or key stores. In order to construct productive test results, you must profile the application and identify database-related bottlenecks in critical workflows.
Note that it’s not enough to simply profile the entire application, you must profile the application’s use of various database resources during particular key workflows. For instance, you may determine that using in-memory table types speed the application in general but not in any meaningful way for the application workflows you’re concerned about. Since profiling is often language dependent, a discussion of application profiling is out of scope for this article but be aware of its necessity.
Drupal
Most of the basic Drupal operations are encapsulated in a single POST of a form where the operation is specified by the op POST field. For instance “Log in” or “Save” and is often submitted as part of the <button> tag. You must also include a form_idso that drupal core knows what form to associated incoming POST data with. Please keep these things in mind when examining the HTML forms for Drupal.
Logging in and creating content…
Scenario: You administer a website that takes progress reports on laboratory samples. As many as 100 different users continually submit new reports which create new nodes. This process needs to function as smoothly and as quickly as possible.
In response to these requirements I’ve drawn on up this test plan to monitor performance improvements:
The test plan has all the required elements:
- We log into drupal, saving the cookie in a cookie manager nested as a sibling of the HTTP request.
- We the fetch the page that hosts the form for submitting a new article and extract the unique form_token value for later use.
- We use the extracted value combine it with a dynamically generate title (the only value required by drupal when creating an “article” node type.
- We’ve placed a “Response Time” graph nested underneath the last HTTP request so that the last sampler is the only one contributing data to the graph.
When I run this, all works as expected. It generates nodes that I can see in Drupal and I can adjust the loop count as I run the test so that I collect however many samples I feel like I need to get useful information.
This works well enough but there’s a non-obvious drawback to this approach. Though we can stagger the requests out by introducing a “ramp up” period on the thread group, it doesn’t at all represent the randomness of actual user traffic. Sometimes several requests will come in at once, some times you go some time between requests. Staggering the requests out doesn’t simulate that. Adding a randomizing timer wouldn’t add enough randomness before the beginning of the test. As the iterations progress they would gradually get out of sync but we want our graph to start already in the state where they’re splayed out. Let’s see if we can fix that by mixing up the timing a bit.
Here is the revised (more complicated) test plan:
We’ve now introduced a randomizing loop at the beginning of the test plan. It goes through 100 iterations of a pointless request, each time pausing a randomized amount of time. The compound effect is so that each thread exits the loop at a random point and at any given point in time has made a truly random amount of progress.
The drawback to this approach is that the loop can now no longer be extended by modifying the thread group (which now includes our randomization loop). To work around this we introduce a new “Work Loop” with a counter and conditional that we can tweak to extend the sample period out as needed.
It should be noted that in most cases, the “simplified” test plan works fine and is in fact easier to for other people to understand. I mention this for cases where you need to create truly uniquely timed random background traffic. In that spirit you might also introduce some of the Random controllers so that each thread behaves wholly and completely different than any other in both timing and overall behavior.
WordPress
I’m going to assume that if you’re interested in WordPress, you probably skipped the Drupal section so I’ll retread some of the same ground. If you did read the previous section, just downloading the test plans and looking them over is probably enough to figure out how WordPress works differently than Drupal.
Logging in and creating content…
Scenario: You administer a website that takes progress reports on laboratory samples. As many as 100 different users continually submit new reports which create new posts. This process needs to function as smoothly and as quickly as possible.
In response to these requirements I’ve drawn on up this test plan to monitor performance improvements:
Let’s break it down. The elements at play here are:
- A
Cookie Managerso that when we log into WordPress it persists for the later requests. - An HTTP sampler that just issues
POST /wp-login.php - An HTTP sampler to fetch the
Add Postform and extract the automatically generated ID number for this post and the_wpnonceassociated with this form instance. - Finally we have another HTTP sampler for actually submitting
POST /wp-admin/post-new.phpto create the actual posting. I’ve also attached a Response Time graph to record the results and a random timer to spread the work threads out so they’re not all in sync.
This works well enough but there’s a non-obvious drawback to this approach. Though we can stagger the requests out by introducing a “ramp up” period on the thread group, it doesn’t at all represent the randomness of actual user traffic. Sometimes several requests will come in at once, some times you go some time between requests. Staggering the requests out doesn’t simulate that. Adding a randomizing timer wouldn’t add enough randomness before the beginning of the test. As the iterations progress they would gradually get out of sync but we want our graph to start already in the state where they’re splayed out. Let’s see if we can fix that by mixing up the timing a bit.
Here is the revised (more complicated) test plan:
We’ve now introduced a randomizing loop at the beginning of the test plan. It goes through 100 iterations of a pointless request, each time pausing a randomized amount of time. The compound effect is so that each thread exits the loop at a random point and at any given point in time has made a truly random amount of progress.
The drawback to this approach is that the loop can now no longer be extended by modifying the thread group (which now includes our randomization loop). To work around this we introduce a new “Work Loop” with a counter and conditional that we can tweak to extend the sample period out as needed.
It should be noted that in most cases, the “simplified” test plan works fine and is in fact easier to for other people to understand. I mention this for cases where you need to create truly uniquely timed random background traffic. In that spirit you might also introduce some of the Random controllers so that each thread behaves wholly and completely different than any other in both timing and overall behavior.
Rest API
OK, now we’re going to get a little more abstract. Let’s say you’re responsible for either developing or maintaining an application that exposes a RESTful API. There may be certain operations that are deemed “mission critical” where if they take too long or begin failing underneath a representative load then the business goals are impacted.
For these test plans I’m going to use jsonplaceholder.typicode.com since that’s a free REST API service you should also have access to. I would just ask that you don’t DoS their servers by running a thousand thread test plan indefinitely. These are only test values, please substitute with your own.
Adding a New Post…
For this first test, we’re going to try to add a post to the jsonplaceholder API. Here is the test I’ve constructed for this:
The elements in this plan are simple and straight forward:
- A single thread group
- A single HTTP samplers issuing a
POST /posts(as per their API documentation) with a JSON document in itsBody Datafield describing the values for our new post. - The
Header Manageris just there so that the MIME type sent by JMeter to the REST server is correctly set to JSON instead of the default form data. - A graph for recording the response time for adding new posts.
and that’s pretty much it. Please be aware that since this is a demo experience, our posts won’t actually show up. The only way you’ll know if what you did worked is if the Response Data in the Results Tree returns a valid JSON document describing your new POST back to you except including it’s “new ID” of 101. In an actual example, you would probably extract this using the JSON extractor and continue performing actions on this object (retrieval, modifying, deletion, etc) but since this service just validates we’re performing the right action, we’ll have to leave it just submitting a new post.
Retrieving a Random Post…
This example testplan should illustrate how at a basic level you can inter act with a REST API:
This test just consists of two samplers and a graph. The first sampler, retrieves all posts by issuing a GET /posts against the API. This first sampler also has a JSON extractor attached where it returns any random element that matches $.[*]['id'] in the JSON document that the sampler comes back with an stores in in a variable called ${postID}.
The second sampler is even simpler. It just issues a /posts/${postID} and disregards the results (since the goal is just to test arbitrary retrieval of documents.
If you’ve managed to follow all that, you’ve probably gotten the hang of it but lets try a few more scenarios.
Updating Random Objects Presented By The API…
OK so now we can retrieve random objects, let’s progress onto blindly updating these objects. For that I’ve created this test plan:
This still is just two samplers and a graph but the second sampler has been changed up a bit. We now have a Header Manager added again and there is now a JSON document in the Body Data field on the sampler itself. The JSON document includes all the attributes we’re trying to set and we’re using the PUT method (per their REST API documentation). The ${postID} variable used in both the JSON document and the Path field.
MariaDB
I can’t really construct a specific example here considering just how wide open the task of load testing a database is. My only suggestion would be to generate application-level load while profiling the application’s database usage and once you know what kinds of queries you’re running (lots of table joins, large number of rows returned, etc, etc) you can construct a contrived example workload to execute on the database directly. Once you know what kind of load to generate, you can perform your performance tweaks (or bug triage) and then have faith that your database-level changes will have the desired effects on the eventual performance of the application.
Installing The JDBC Driver…
Since JMeter is a java application, you have to provide it with the JDBC driver for communicating directly with your preferred database server. In this example, it’s MariaDB. There are MariaDB-specific drivers, but I’ve only ever used the MySQL driver so that’s why my test plan uses. Feel free to substitute with the MariaDB driver if you so choose.
If you’re unsure if that .jar file is in the path for the java you’re running JMeter within you can install the relevant package (libmysql-java on Ubuntu and mysql-connector-java on Fedora/RHEL) and create a symlink to it in the JMeter applications lib directory. For example on my system:
[root@joels-workstation Assets]# ln -sf /usr/share/java/mysql-connector-java.jar /home/joel/jmeter/lib [root@joels-workstation Assets]#
After which, you should relaunch the application so that it loads the newly available jar file.
Configuring JMeter to Connect To The Database Server…
OK so now that java can now connect to the database server, we need to tell JMeter the information (hostname, username, password, etc) for setting up the connection. To do that we add a JDBC Connection Configuration config element to the test plan and populate it with the necessary information:
Most of that is pretty straight forward but breaking the important bits down anyways:
- Variable Name Bound to Pool: This is the variable name that will be used to store this connection information. Your later JDBC samplers will refer back to this variable when it need to refer back to a particular connection.
- Connection Pool Configuration: Most of this can be kept at the defaults unless you start running into issues. These options are mainly for configuring resource restraints for things like evicting closed connections or the maximum total number of connections.
- Connection Validation by Pool: Can also be left default. Just specifies how JMeter/Java will determine whether a connection is still valid. If a
Validation Queryis provided, it’ll use that SQL query as a means of determine that a server is still responding on that connection. - Database Connection Configuration: This is the real meat and potatoes. In the screenshot I’ve made liberal use of variables set on the test plan itself to store the critical connection information. This keeps others from having to dig through your test plan for the various bits of information they may have cause to change. Going through each field individually
- Database URL: This is analogous to the URL you type into the web browser. It contains all the information needed to locate the specific database you’re interested in. Click here for more information on the JDBC url format.
- JDBC Driver Class: This is where we point JMeter at the particular JDBC driver we installed before. It provides a drop down so you don’t have to memorize which jar file to point it at but if you’re using the MySQL driver then the driver class is
com.mysql.jdbc.Driverbut if you’re using the mariadb driver it would beorg.mariadb.jdbc.Driver. - Username: The user you’re connecting as, duh.
- Password: The password. You’ll have to have faith that if you type a variable name into this box it’ll still expand when the test executes. I say that because the password field is masked (for obvious reasons) so you’ll have to just be very careful not to make any typos.
Creating a Table and Populating it With Data…
Alright, let’s create a basic and not entirely useful test plan. One that shows the general workflow you’d go through when subjecting a database to increased load. Here is one such test plan:
As you can see we have three basic thread groups:
- A setUp thread group ensuring that the temporary tables we’re going to manipulate are present but empty.
- The actual workload generation, including a response time graph specific to the JDBC sampler where I’m INSERTING data.
- A tearDown thread group for deleting the table so that that database server goes back to a clean slate once we’re done.
Please note that the tearDown thread group will only run when the main thread group finishes its task. If you hit “stop” or “shutdown” it gets skipped. For that reason, it’s probably preferrable that when you tweak you either leave a finite number of loops on the main thread group or remember to go back in and manually clean up the database after each test run.
That’s basically it. It’s a vague prescription sure, but that aren’t really any examples that I can think of that would give you any more insight into how to create a JMeter test plan for MariaDB. Once you know the above, it’s almost all about understanding the application you’re testing for and not so much the JMeter part.
Where To Go From Here
Phew. I know everything we’ve covered so far seems like a lot and you can probably already generate almost any load most people would need. There’s more to JMeter though. A lot more. It’s a professional grade tool for dealing with any situation you might find yourself in.
Some examples on where to go next if you’re interested:
- Browse the “Component Reference” in the official manual. I haven’t even cover 50% of the possible elements and when you would use them.
- One of the drawbacks to the current approach is that doing everything from a single node introduces issues where the bottleneck may be the bandwidth for the node performing the test. For this reason you may want to look into distributed testing in JMeter. This allows you to execute the same test plan on multiple nodes so that you can more accurately simulate inbound traffic from the internet.
- You may follow the
jmetertag on StackOverflow to help others with their questions or gain exposure to new JMeter topics. - and finally checkout Blazemeter. They has a metric ton of really well written JMeter tutorials for a large variety of tasks.