Testing web services with soapUI

In this article, Mike Kelly details the finer points of web services testing using soapUI.

Five or six years ago I tested my first web service. The service was one piece of a larger system rewrite. In this

phase, we were integrating a legacy mainframe application and a new web platform using web services. We had a number of tools for testing web services floating around the office: SoapScope, a homegrown browser-based test tool, and several of us were writing straight Java or Ruby code to test. I remember at the time thinking there had to be an easier way.

It was about two weeks into the project, while I was struggling to get a couple of Ruby libraries working to test our service, that someone on the project team introduced me to soapUI. At the time, the soapUI project was still young - only basic functional and performance testing functionality and no professional version. Since the first time I used soapUI it's become my default choice for service testing.

Today, soapUI has both a commercial Pro version along with their open source version. They offer support for WS, REST, and HTTP based services, along with a recent announcement of support for JMS, AMF, and JDBC. In this article, we'll look at an example of functional and performance testing a WSDL service. In future articles we'll look at load testing, web service mocking, and integration with JUnit. For our example application in each of these articles, we'll be looking at the Atlassian JIRA soap web service. It's a good non-trivial interface that also has a publically available example.

Creating your first project

When you first open soapUI, no projects will be loaded. To create a new project, right-click on the Projects icon and select New soapUI Project as shown in figure 1 below:

Figure 1: Creating a new project in soapUI.

Figure 1: Creating a new project in soapUI.

Step 1: Creating a new project in soapUI.

This loads the New soapUI Project dialog shown in figure 2. Enter a name for the project and the initial WSDL for the project to be built off of. If your WSDL changes, you can import updates at a later date - don't think you need the final version to get started. For this example, I'll be using my own JIRA web service implementation so I can run tests. If you're following along, you can point your project to Atlassian's example version which can be found here.

Figure 2: Naming the project and importing the initial WSDL.

Figure 2: Naming the project and importing the initial WSDL.

Step 2: Naming the project and importing the initial WSDL.

You'll see a number of checkboxes for tasks that you can have soapUI perform automatically during project creation. You're welcome to try them all out, but typically I just check the "Create sample requests for all operations?" checkbox. When you're done entering the new project information, click OK.

Figure 3: Various JIRA requests shown under the project with a request window open in the main soapUI workspace.

Figure 3: Various JIRA requests shown under the project with a request window open in the main soapUI workspace.

When the project finishes loading, you should see all the different requests detailed in the WSDL definition displayed under the project as shown in figure 3. To view the details of a request, you can expand the request and double-click on the "Request 1" icon shown below. That should open the request in a window the main soapUI workspace.

Step 3: Various JIRA requests shown under the project with a request window open in the main soapUI workspace.

Figure 4: Use the green arrow to submit a request to the service.

Figure 4: Use the green arrow to submit a request to the service.

To submit this request to the web service manually, simply click the green arrow in the request window -- shown in figure 4.

Step 4: Use the green arrow to submit a request to the service.

If you do that for this particular request - the addVersion request - without making any changes, you'll get a reqponse containing the exception shown in listing 1 below.

<faultcode>soapenv:Server.userException</faultcode>

         <faultstring>java.lang.NumberFormatException: Invalid boolean</faultstring>

Listing 1: Exceptions from not passing in valid parameter data.
 

This exception gets returned for a couple of reasons. First, we haven't logged in yet. If you scroll down through the available requests in our project, you'll see that there's a login request. Second, all of the data values in our request had question marks in them, as shown in listing 2 below. Before submitting any requests, you'll want to enter test values into those fields where you see question marks. 

<in0 xsi:type="xsd:string">?</in0>

         <in1 xsi:type="xsd:string">?</in1>

         <in2 xsi:type="bean:RemoteVersion" xmlns:bean="http://beans.soap.rpc.jira.atlassian.com">

            <id xsi:type="xsd:string">?</id>

            <name xsi:type="xsd:string">?</name>

            <archived xsi:type="xsd:boolean">?</archived>

            <releaseDate xsi:type="xsd:dateTime">?</releaseDate>

            <released xsi:type="xsd:boolean">?</released>

            <sequence xsi:type="xsd:long">?</sequence>

         </in2>

Listing 2: By default, values are not populated for requests.

At this point, you can browse any of the requests; view the different request arguments; change the requests and run them manually to see the responses. In short, you should be able to browse and interact with your web service.

Writing and executing test cases

Browsing and interacting is all well and good, but I'm guessing you want to run some tests. To do that, you'll need to create a TestSuite. To do that, right-click on the project and select New TestSuite as shown in figure 5.

Figure 5: Creating a new TestSuite in soapUI.

Figure 5: Creating a new TestSuite in soapUI.

Step 5: Creating a new TestSuite in soapUI.

This should open the New TestSuite dialog as shown in figure 6. Enter a name for your test suite. Keep in mind that for most projects you'll have multiple suites in your project, so descriptive names can help. When finished, click OK.

Step 6: Naming your TestSuite in soapUI.

This will add your TestSuite to the project tree structure in the left-hand navigation. It will also open your TestSuite in the main soapUI workspace.

Figure 6: Naming your TestSuite in soapUI.

Figure 6: Naming your TestSuite in soapUI.

In soapUI TestSuites are made up of TestCases. For our example, we'll build a simple test suite that logs us into JIRA and then logs us out. This is a good example for a couple of reasons. First, it will show you how to pass values between TestCases, this is important because for most web services I've tested, I've had to do this. Second, we'll get a chance to look at some basic features - like assertions - without you having to know much about JIRA. Login and logout are fairly self-explanatory features.

To add our first TestCase, click on the "Create a new TestCase in this test suite" button. You'll find it in the TestSuite window, as shown in figure 7.

Figure 7: Create a new TestCase button in the TestSuite window.

Figure 7: Create a new TestCase button in the TestSuite window.

Step 7: Create a new TestCase button in the TestSuite window.

This opens the New TestCase dialog where you name the test case you're creating. It looks just like the New TestSuite dialog. Enter a name and click OK. Since this test case will test login, I'm naming my TestCase "Login."

When you click OK, two things happen. First, your TestCase is shown in the TestSuite window with an empty result bar as shown in figure 8 below. That result bar is white because you haven't run the test yet. When you run it, the bar will turn red for a failure or green for a pass.

Figure 8: Your TestCase is added to your TestSuite.

Figure 8: Your TestCase is added to your TestSuite.

Step 8: Your TestCase is added to your TestSuite.

Second, you'll also see a window open for the TestCase you just created. In figure 9, you'll see the window for the Login TestCase I created.

Figure 9: TestCase window for Login test case.

Figure 9: TestCase window for Login test case.

Step 9: TestCase window for Login test case.

Third, as shown in figure 10, in the left-hand navigation project tree you'll see your new TestCase listed under your TestSuite. You'll also see counters for how many test steps or load tests that TestCase has associated with it.

Step 10: TestCase added to the project navigation tree.

Much like a TestSuite is made up of TestCases, a TestCase is made up of TestSteps. For our Login test to work, we'll need to add the TestSteps to make it happen.

Figure 11: Create a new Test Request TestStep button.

Figure 11: Create a new Test Request TestStep button.

Figure 10: TestCase added to the project navigation tree.

Figure 10: TestCase added to the project navigation tree.

First, we'll add a login request to the JIRA web service. To do this, click the "Create a new Test Request TestStep" button in the TestCase window. This button is shown in figure 11.

Step 11: Create a new Test Request TestStep button.

In the Add Step dialog, enter a name for the TestStep and click OK. This will open the New TestRequest dialog shown in figure 12. Scroll down in the list and select the login request. Then click OK.

Figure 12: New TestRequest dialog.

Figure 12: New TestRequest dialog.

Step 12: New TestRequest dialog.

This opens the Add Request to TestCase dialog shown in figure 13. You can change the name for the request if you like, and you can select a series of stock assertions. By default, the SOAP Response Assertion is selected. For now, that's fine. Simply click OK.

Figure 13: Add Request to TestCase dialog.

Figure 13: Add Request to TestCase dialog.

Step 13: Add Request to TestCase dialog.

When you click OK, the request gets added to the TestCase and the request window opens for the request you just added. Just like earlier in the article, you can edit the values and run this request manually. However this time, whatever values you enter will be the values saved for this test case. For this example I'll use my username and password, shown in listing 3 below.

<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://soap.rpc.jira.atlassian.com">

   <soapenv:Header/>

   <soapenv:Body>

      <soap:login soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">

         <in0 xsi:type="xsd:string">mike.kelly</in0>

         <in1 xsi:type="xsd:string">password</in1>

      </soap:login>

   </soapenv:Body>

</soapenv:Envelope>

Listing 3: Login request with username and password populated.

If I run this request manually, I get the response shown in listing 4.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

   <soapenv:Body>

      <ns1:loginResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:ns1="http://soap.rpc.jira.atlassian.com">

         <loginReturn xsi:type="xsd:string">26lRGT7uX5</loginReturn>

      </ns1:loginResponse>

   </soapenv:Body>

</soapenv:Envelope>

Listing 4: Login response.

You'll see a 10-character string gets returned with my session. You may also notice that anywhere the TestStep icon appears, it's changed colors from white to green. That's a nice little visual tip to tell you your assertions with that TestStep have passed.

Currently, the only assertion we have on our TestStep is the one that was setup when we created it. All that assertion does is confirm our response is a valid SOAP response. While that's important, it's not enough for us. So we're going to add another assertion that confirms we get a 10-character string back in our response.

In the request window, you'll see an Assertions button at the bottom of the window. If you click it, it shows what current assertions you have associated with your TestStep. If you want to add another assertion -- which we do -- you'll want to click the Add Assertion button, shown in figure 14.

Figure 14: Button for adding assertions to a TestStep.

Figure 14: Button for adding assertions to a TestStep.

Step 14: Button for adding assertions to a TestStep.

When you add an assertion, the first dialog that appears is the Select Assertion dialog. There are a number of different assertions available, and in this article we're only going to look at one of them. For more details on each assertion type checkout the soapUI user guide. For our example, we're going to do an XPath Query. Select that option and click OK.

Figure 15: Selecting an XPath Query assertion.

Figure 15: Selecting an XPath Query assertion.

Step 15: Selecting an XPath Query assertion.

This will open the XPath Match configuration dialog. In this dialog, you can specify an XPath Epression to pull out the value(s) you want to test. In this dialog, you also specify the expected result for your expression. While you're writing your expression and result, you can test your declaration using the "Select from current" and "Test" buttons under the Expected Result section of the dialog. This provides you some real-time feedback on your expression and result definitions.

Figure 16: Regular expression for XPath verification of loginReturn.

Figure 16: Regular expression for XPath verification of loginReturn.

Step 16: Regular expression for XPath verification of loginReturn.

In the example shown in figure 16 above, you'll see that I look for the "loginReturn" element in the response XML and I then compare that to a regular expression for 10 characters includes upper or lower case characters and all ten digits. That declaration should return true if the element value matches those parameters. That's why my expected result is true.

When you click Save, you the assertion gets added to the test step. You should be able to see the added assertion at the bottom of the request window as shown in figure 17.

Figure 17: XPath Match added to the request.

Figure 17: XPath Match added to the request.

Step 17: XPath Match added to the request.

At this point, you can close the Login request window. We're ready to move on to the next TestStep. Next we want to logout. You can add a logout step following the same steps we used for login. When you get to the logout request, as shown in listing 5 below, you'll see that you'll need to pass in the session from login.

<soapenv:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://soap.rpc.jira.atlassian.com">

   <soapenv:Header/>

   <soapenv:Body>

      <soap:logout soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">

         <in0 xsi:type="xsd:string">?</in0>

      </soap:logout>

   </soapenv:Body>

</soapenv:Envelope>

Listing 5: Logout request XML.

If you run this request without passing in the session, you'll get a logoutReturn of "false." This is because JIRA didn't know who to logout.

To fix this, we'll need to pass our session id from login to our logout request. We'll do that using what's called a Property Transfer. To add a Property Transfer, you click on the Property Transfer button in the TestCase window.

Figure 18: Property Transfer button in TestCase window.

Figure 18: Property Transfer button in TestCase window.

Step 18: Property Transfer button in TestCase window.

In the InsertStep dialog, you can name your property. I normally name it after the value I'm passing.

Figure 19: Naming the Property Transfer.

Figure 19: Naming the Property Transfer.

Step 19: Naming the Property Transfer.

When you click OK, the Property Transfer window appears for the new element. To create a transfer, click on the add new property transfer button in the upper-left-hand corner of the window.

Step 20: Button for adding a new property transfer.

Figure 20: Button for adding a new property transfer.

Figure 20: Button for adding a new property transfer.

Again you'll be prompted to name the transfer. This is because you might want to transfer multiple properties. In this one window, you can map multiple properties to transfer. It feels redundant because we're only doing one property, but it's quite nice if you have five or six properties you need to move between requests.

Once you've added the transfer, you can configure it. You'll need to define both the source and target for the transfer. We'll select the value from the loginReturn element in the Login response and transfer it to the in0 element in the Logout request. You can see how I configured that in figure 21.

Figure 21: Setting the property transfer parameters in for session.

Figure 21: Setting the property transfer parameters in for session.

Step 21: Setting the property transfer parameters in for session.

If you clock the green arrow at the top of the Property Transfer window, it will run the transfer and show you the results in the Transfer Log at the bottom of the window.

You can see in the example above that the value "TD9CJdR3F1" was found and transferred. Now if you went back and reran the Logout request, it would return a true. At runtime, this value will be moved dynamically between the two requests.

If you look at your TestCase now, you'll see that all three elements are represented. If your elements aren't in the right order, you can simply drag and drop them to get them in the order you want.

Figure 22: Finished TestCase for login and logout.

Figure 22: Finished TestCase for login and logout.

Step 22: Finished TestCase for login and logout.

Go ahead and run your test case using the green button in the top toolbar for the window. You'll notice that both the status bar updates and a summary test log is displayed at the bottom of the TestCase window, as shown in figure 23.

Figure 23: Summary TestCase test results in soapUI.

Figure 23: Summary TestCase test results in soapUI.

Step 23: Summary TestCase test results in soapUI.

And with that, we've successfully setup and run our first test for our JIRA service.

Next Steps

In the next article we'll build on what we've done here and look at running some performance tests using soapUI. For more on soapUI, its features and what the soapUI team is working on, check out the soapUI website. For other articles and blog posts on soapUI, the soapUI team also maintains an "in the news" listing on their website. And for more on JIRA, it's interface and what it does, check out the Atlassian product website.

 

 

This was first published in October 2011

Dig deeper on Internet Application Security

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

2 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

SearchSOA

TheServerSide

SearchCloudApplications

SearchAWS

SearchBusinessAnalytics

SearchFinancialApplications

SearchHealthIT

Close