 | |||
|
|
||
| |||
| |||
| |||
|
||||||||||||||||||||
| |
|
Home > Software Quality News > The consequences of overlooking SOA testing blind spots |
| |
|
|
|
|
| Software Quality News: |
|
||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Continued from "SOA applications bring testing challenges".
Beware the unintended consequences, advised John Michelsen, when asked about the potential blind spots of SOA testing. Michelsen, chief scientist and co-founder of SOA testing software provider iTKO in Dallas, weighed in with other industry experts on what SOA testers should watch out for.
Unintended consequences, he said, result from failing to test the dependencies in a SOA application. "I can test my piece and you can test yours, but when it comes together to form a solution, the bug is in the line between the boxes, in the integration." The solution, Michelsen said, is not to retest all the services that have already been tested, but to test the business process in context. "I don't have to test all the service requirements, but I have to create a test [to prove] that it behaves as I expect it to in my context," he said. "The problem is that a service may be subject to change without my notice, so if I don't continuously validate my expected behavior, it's possible it will break my application." Continuous validation is key, not only in development and test, but in production as well, he added. Another potential blind spot, according to Rizwan Mallal, managing director of Crosscheck Networks, a SOA testing tools provider in Newton, Mass., is the reliance on HTTP return codes to determine the success of an operation. "For example," he said, "a return code of HTTP 200 will be used by a tester as a sign of a successful SOA operation, even though a return message might not be structurally sound. This requires deeper content inspection besides relying on HTTP error codes." Mallal said another SOA testing blind spot is "during performance testing of security-enabled messages, where testers ramp up load by simply sending duplicate WS-Security-enabled messages. Since all these security messages contain duplicate timestamps or nonces, the messages will be treated as a replay attack and thus rejected by an end service. To avoid this challenge, testers need to inject unique timestamps or nonces." Visibility into all layers important "Visibility itself has to reach into all the layers of the platforms, which requires native testing capability," she said. "There's a natural blind spot if you cannot reach into those layers. Now you have multiple black boxes that you have to keep opening." Test coverage is another way to prevent blind spots, Bayliss added. "How much you test is a way to gain insight and do risk management on quality," she said. Scott Barber, chief technologist of testing services PerfTestPlus in Palm Bay, Fla., and executive director of the Association for Software Testing, said not all test teams are used to testing all the inputs, outputs, and formats that SOA testing entails. Testers need to be especially diligent to test all kinds of potential malformed data input. "I think a lot of testers are not thinking of that, or they're being instructed that it's not in their scope because it's in the middle," he said. The assumption may be that the in-house developers took care of that, or that the third-party service developers did so, "but people make mistakes. If you're not testing that, I think that's an oversight," Barber said. Additionally, Barber said SOA testers need to look more closely at output when a third-party service is involved, "which is tricky, unless you're given a test server with test data, which also makes [testers] nervous, but that's where [service] contracts come in. It doesn't mean you can ignore all the output, but it comes down to trust of the provider." SOA quality: Who is responsible? "At a high level, the producer of a SOA has more responsibility for quality than the consumer," Mallal said. "Within the producing group, developers, QA, security, and support staff are all responsible for quality." Michelsen said quality still belongs in QA, but that the development team, as well as the governance group, also has responsibility. Barber, in contrast, said that whoever is buying or paying for the SOA application is ultimately responsible for quality. "We [testers] give them the best information we can to help them make good business decisions, but we don't decide to go live," he said. In addition to good due diligence around procurement of a third-party service and management of that contract, quality also depends on cooperation within and across SOA project teams, Barber said. With SOA, testers' jobs don't change, "but we might need extra tools or help from developers." That said, Barber's final caveat on a universal blind spot: "Don't over-trust your tools!"
|
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
About Us | Contact Us | For Advertisers | For Business Partners | Site Index | RSS |
|
|
||
|
|||||||
