Load Testing a Web Service


The purpose of this tutorial is to illustrate the steps required to load-test a web service using Web Performance Load Tester. Although the demonstration service used in this tutorial is very simple, the concepts presented are directly applicable to more complicated services.


This tutorial is organized into 4 main steps:

  1. Create the testcase
  2. Configure the testcase for replay
  3. Configure the testcase for multiple users
  4. Run test and analyze results

Introduction to the Bank web service

The service tested in this tutorial provides an interface to bank accounts. A similar service might be used, for example, to allow ATM machines in bank branches to access accounts in the central bank system. The operations used in this tutorial are login and get-balance. These operations use a simple pseudo-XML format for ease of readability.

Login transaction

The login transaction sends the username and password in a login operation. If successful, the response contains a session identifier that must be used for future transactions. The XML sent and received in this transaction look like this:





Note that the above screenshot shows the content as viewed in Analyzer's Content view.


Get-balance transaction

The get-balance transaction sends the session identifier and, if successful, receives the account balance (in cents). The pseudo-XML sent and received in this transaction look like this:




In two example transactions show above, user "JAMES" logs into the system using the password "nkbmmfn". The result value "1" in the response indicates a successful login and provides a session identifier to be used in future transactions. He then checks his balance and finds that he has $7.49 in his account (749 cents).

Step 1 - Creating the testcase

The first step in the load-testing process is to create the testcase to be simulated. The testcase is a collection of HTTP transactions (requests and responses). When testing a browser-based application, this step is usually achieved by using the browser to interact with the website in the same manner any user would - with the browser configured to use a recording proxy. This proxy records the HTTP transactions between the browser (client) and server and creates the testcase using these transactions. If you have a client for your web service and it supports the use of a proxy, this is the fastest way to get a testcase created.


Since many web services do not have this ability, we will demonstrate how to create the transactions from scratch.

Creating the login request

The first step is to create the content of the request - that is the pseudo-XML shown above. Paste this into your favorite plain-text editor and then save the file (e.g. login-request.txt). Next, note the length of the file - this length will be needed when we add the HTTP headers.


Step 2 involves putting the HTTP start-line and headers into the request. Below on the left are the start-line and headers used for this example request - the required headers may be different depending on the requirements of your service.



Note that each part of the request headers will need modification as required by your service:

  1. In the start-line, the path of the service will be different (/demo/bank)
  2. The host should be changed to the hostname for the server your service resides on
  3. The content-type could be different (text/xml is common)
  4. The content-length must be changed to match the content you created for this request


The text file used to create the request shown above contains this:



POST /demo/bank HTTP/1.1
Host: www.webperformanceinc.com
Content-Type: text/xml
Content-Length: 111




The other request and responses are created in a similar manner.


Once you have the 4 files created (a request and response for each of the 2 transactions), we can create the testcase.


In the Navigator, select the Create Empty Testcase item from the pop-up menu on the Testcases node to create a blank testcase where the transactions can be placed:



We now have an empty testcase (renamed "Check balance"):



The next step is to create the two transactions required for the Check balance testcase.


Each transaction can be imported (request and response) using the Import->As New Web Page item from the pop-up menu in the testcase editor:





Note that Analyzer groups transactions together in pages within a testcase. For a web service, there are no pages, so Analyzer will simply treat then as one-transaction pages. In complex services that issue several groups of transactions that are logically grouped and related, it can be useful to group the transactions together within a single page. Analyzer will calculate separate metrics for each transaction and for each page, which can be useful when analyzing load test results.



The request and response files are selected in this dialog:



After importing both transactions, our testcase looks like this:



Step 1 is now complete.



Note that the duration of the imported transactions is 0. Since these transactions have not yet been analyzed with a live server, the duration metrics cannot be determined. After a successful replay has been completed, consider promoting the replay to the base recording: Select the Promote button on the Replay Properties dialog - accessible from the replay drop-down at the top of the testcase editor.



Step 2 - Configuring session-tracking

Before we can execute this testcase in the current form, we have to handle the session-tracking used by the service. We could replay the transactions exactly as we created them, but the service would fail on the second transaction, because the replay would send the session identifier that we put in the transaction. Since this session identifier has probably expired since we obtained it, the get-balance request needs to send the new session-identifier from the login response.



Note that some services use cookies for session-tracking, much like a browser. If this applies to your service and you can record it using the recording proxy or you have imported real transactions captured in some other way, the cookie headers might be in place and will be handled automatically by Analyzer. In this case, this step may not be necessary.



Two steps are required to handle session-tracking for this testcase:

  1. Extract the session identifier from the login response
  2. Modify the session identifier sent in the get-balance request


To do this, activate the Actors view and select the login transaction in the testcase editor. The Actors view will initially appear as below (empty).



Press the Add Extractor... button () to add a new extractor - the resulting dialog allows configuration of the extractor. In this case, we want to extract the session identifier which is located between the <sessionid> and </sessionid> tags (delimiters) in the response. As these values are entered in the Prefix anchor and Suffix anchor fields, the delimiters will be highlighted in the response content field in the lower third of the dialog. If either delimiter cannot be located, an error will be indicated at the top of the dialog. If both the prefix and suffix anchors are found, the target value for extraction will be displayed in the field at the bottom. Next we enter sessionid in the Extract value in user variable field. This will create a variable in the user state that will contain the session identifier when it is located by this extractor.



Note that if the delimiters appear several times and the first instance does not contain the desired value, the Repitition number... field may be changed to select the correct value.




After pressing the OK button, the Actors view will show the Extractor like this:




Once the session id has been extracted from the login response, it should be dynamically replaced in the get-balance request. This is accomplished by adding a modifier in the Fields view. Open the Fields view and then select the get-balance transaction in the testcase editor (2nd transaction). The fields in the request will be displayed as below.



When a POST (message which sends content to the server) is in a format that is not recognized by Load Testers parsers, it will list the field as a (Post content) field. Double-clicking the row will open a dialog for configuring a datasource on the field. Since we only need to replace a small part of the request content, select the Text with replaced regions datasource (1). The dialog will now appear as below. Highlight the region to be replaced (2) and then press the Add () button (3) to configure a datasource on that region.



When all parts of the POST have been configured, press the OK button to save the changes.



With session-tracking configured for the testcase, the testcase may be replayed. Pressing the Replay button () in the toolbar will invoke the testcase configuration wizard. This wizard is very useful for complicated web applications but for simple web services it is frequently unnecessary. Select the Do nothing... option and press the Finish button.



After dismissing the Testcase Configuration wizard, the replay will begin. It will likely end very quickly, since it is a very simple and very short testcase. When it is complete, open the Errors view and then select the testcase to verify that no errors occurred during the replay. Then open the content view and select the get-balance transaction in the testcase editor to view the result:



Checking the value of the session identifier we can see that it is different from the original transactions we created - indicating that Analyzer was able to successfully extract the session identifier from the previous response and modify this request to send the correct session identifier. Also note that the balance returned is correct (we'll come back to this later in the tutorial). If you replay this testcase again, you should see a different session identifier with the same balance returned each time.


At this point we can declare success step 2 of this tutorial - we have configured the web service testcase so that it can be successfully replayed.



Note that since the Bank demo service does not use true XML, the XML viewers may not always format the content properly. In the screenshot above, the plain-text viewer has been selected for both the request and response content. Additionally, the tabs have been locked to prevent the view from switching back to the XML viewer each time a transaction is selected.



Step 3 - Configure the testcase for multiple users

The key difference between the simple replay completed in the last step and a load test is volume. A load test is simply a lot of replays, all happening at the same time. We could perform a load test with the testcase right now, if we wanted. But it would not be a very accurate simulation of the expected use-case of the system since the same person is not likely to be checking their balance simultaneously from multiple locations over and over again. Instead we would like the load tests to simulate many different users checking their balance.

Creating test data

The first step in simulating multiple users is to create the list of users. For this tutorial, we will assume that the system is already populated with accounts and we have the list of user names, passwords and current account balances available to us.



Example users file:


username, password, balance

JAMES, nkbmmfn, 749

JOHN, ohgixm, 2913

ROBERT, qelslrjnja, 83801

MICHAEL, qfpqjxc, 2074

WILLIAM, nfbwhfh, 614

DAVID, hhjvksgue, 8365

RICHARD, rkipbo, 153



The next step is to import this data into Analyzer™. This is done from the Dataset node in the Navigator. Select the Import... item from the pop-up menu and complete the import dialog:



After importing the dataset, it will appear in the Navigator under the Datasets tree node and the dataset will open to show the imported data:



Note the 3 settings at the top of the dataset editor: Lifespan, Resuable & Sharable. The Testcase lifespan indicates that when a Virtual User (VU, or simulated user) selects a row of data from this data, it will use that row until the end of the testcase. When a dataset resuable, the row may be used more than once during a load test. When a dataset is not sharable, it can only be used by single VU at a time. The default settings, shown above, are usually correct for a dataset containing the user identity.

Customizing the testcase

Now that the user identity information is available, the testcase may be customized to use it. Two modifiers should be added to the first transaction to replace the username and password in the login request with values from the dataset. This configuration is similar to the configuration of the session-identifier described earlier in this tutorial:

  1. Open the Fields view
  2. Select the login transaction in the testcase editor
  3. Double-click the modifier column (M?) and select the partial content radio button
  4. Select the username in the Content text area, press the Add button () and select the accounts dataset and username field.
  5. Repeat previous step for the password



The testcase is now configured to use a different user identity each time the testcase runs. Replaying the testcase, as described earlier, will allow you to easily verify this behavior. Each time the testcase is replayed, a different username and password are sent in the requests and a different account balance is returned. The following shows the get-balance transaction from the 3rd replay. You can see that the balance, 83801 matches the balance for that user in the dataset.


Configure validation

Validating the results in this way is easy for a few replays, but is not practical for a load test. It would be more efficient to have Analyzer automatically validate the balance returned in each testcase against the value in the dataset.


To configure a validator, first return to the original testcase in the editor:



Next, open the Actors view and then select the get-balance transaction in the testcase editor. Then select the Validators tab at the bottom of the Actors view. You will see the automatically-applied status-code validator already present in the list of validators. Press the add button () to add a new validator.


Configure the validator to look for the account balance from the dataset in the response content by selecting the Dataset field and radio button and then select the balance field in the accounts dataset, as shown below:



After applying the validator, replay the testcase again and check the Errors tab to verify that no errors were encountered. There should be no errors because our dataset accurately reflects the system being tested.


To see what the error looks like from the validator, the dataset will have to be changed to purposely have wrong data in it. Open the accounts dataset and change the value of the first entry in the balance column (double-click a cell to edit it). Before replaying, we will need the force the replay mechanism to reload the dataset - to do this, open the Replay view and select the Reset Dataset State item from the drop-down menu on the right side of the view:



Replay the testcase again and then open the Errors view. The validation error displayed indicates that the account balance that I entered in the dataset (749X) could not be found in the response content for the get-balance transaction:



Don't forget to change the dataset value back to the correct value before moving on!


Step 3 is now complete - we can move on to running a load test.

Step 4 - Run load test and analyze results

Creating the load configuration

After recording, configuring and verifying each testcase, the next step towards a load test is to create a load configuration. Select the Check Balance testcase in the Navigator and then select the Create Load Configuration item from the pop-up menu:



The default settings, shown below, should be fine for demonstration purposes.



Running the load test

At this point, the hard work is done!  Push the start button to start the load test.


The test will add 5 VUs each minute - if you are using a demo license, it will stop at 10 VUs for the last minute of the test. When the test is complete, the results screen will look something like this:


Analyzing the Results

The test results screen picture above, provides some basic metrics for monitoring the progress and state of the test - most notably the number of Users, Hits/Sec and Errors. For a more detailed analysis open the test report: press the Report... button on the results screen. In the first section of the report, you will see overall metrics presented on charts such as the one pictured below. This chart shows that the total Hits/sec scaled linearly as the number of users increased. This is a key measure of server performance and this test did very well!



A large amount of information is available in the test report at varying levels of detail. For a good example of a load test report, see the website: http://www.webperformanceinc.com/load_testing/reports/


In this tutorial we have demonstrated the basic process for testing a web service:

  1. create the testcase
  2. configure the testcase to handle session-tracking mechanism and test using the replayer
  3. configure the testcase to simulate multiple user identities and test
  4. configure the testcase to validate the result of the testcase
  5. create a load configuration
  6. run the test
  7. analyze the results


Good luck!