Home > Software Quality News > Software consortium seeks standard quality metrics
Software Quality News:
EMAIL THIS

Software consortium seeks standard quality metrics

By Colleen Frye
14 Oct 2009 | SearchSoftwareQuality

Software quality news and advice
Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google

The software industry has struggled with metrics, and a key reason is lack of automation, said Dr. Bill Curtis, director of the recently formed Consortium for IT Software Quality (CISQ). It has taken the software industry "a long time to do what other areas of engineering do—measure." CISQ is holding its kick-off meeting in Europe next month, and its primary goal is developing standard software quality metrics that are computable.

Today, there are a lot of measurements that can go into how maintainable, reliable, robust, secure and comprehensive software is. The problem is that there's no agreement on which tests are best and no standardization there, so comparisons are not apple to apple, said Curtis, co-author of the Capability Maturity Model (CMM) framework and senior vice president at CAST, a New York, N.Y.-based software development company.

CISQ's intention is to drive toward automation to the lower costs of measurement, and to make quality metrics ubiquitous, with the potential of "building a marketplace of competitive products that people can choose from to measure software," primarily business applications, he said.

A partnership of the Software Engineering Institute (SEI) and the Object Management Group (OMG), the CISQ is a response to requests both organizations had been receiving about taking on the task of defining quality attributes that are computable, which many outsourcing contracts are starting to demand, Curtis said. "Existing standards like the ISO give a general description of some attributes, but not a computable metric," he said.

Curtis said the CISQ is not a competitive effort to the ISO, but rather will try to work closely with the ISO and perhaps speed the process. "Industy efforts move faster; the ISO tends to be more academic and slower." For example, he said, the ISO's 25000 standard for

In addition, he said, "the biggest frustration in standards efforts is the lack of executive follow-through to drive the effort. If we get executive support up front, the higher the probability it will get into commercial use."

Thus far, Curtis said, IBM, Morgan Stanley and Tata Consultancy Services have joined as members, and discussions are under way with other Fortune 200 companies. The first part of the effort will be to hold executive forums once a year in both Europe and the U.S. to focus on industry issues related to quality, which will leverage the strength of the SEI. "The SEI is good at getting different parts of the industry together to discuss common issues," Curtis said.

The second part of the effort will be to get the technology folks from the member companies together to work on the standards. "This is what the OMG is very good at. They have a well-oiled, proven standards development process that works quickly and is industry focused."

The kick-off meetings in Frankfurt next month and in the U.S. in December (which was postponed from October) will set objectives and prioritize what the CISQ will work on, he said. "The OMG already has standards in place for how to represent a software measure; what they don't have is specific measures [of quality]. In the outsourcing world in particular, Curtis said, there is increasing pressure to be able to answer the question: "Are you giving me software that's reliable and that I can afford to maintain, or junk?"

The answer to that question lies in standardization of metrics related to software quality, he said. It raises questions of what to count, how to count it, if the measures would differ by language, how adverse conditions may impact the metrics, etc.

Curtis said CISQ will also leverage the security world's body of work on good coding practices, the Common Weakness Enumeration (CWE), a community-developed dictionary of software weakness types maintained by Mitre Corp. Curtis said CISQ will "look at bad coding for all kinds of coding issues, and we need a common, defined way of how to represent a violation, which is what the OMG will work on."

Ideally, he said, companies will be able to leverage the standards to build technology that looks for and identifies patterns of quality violations in the code. And the CISQ will develop a certification for those providing services to assess the quality of IT application software, based on the standards. The certification would be similar to the SEI's work in the process area, "authorizing people who do a CMMI assessment," he said. "In this [CISQ] world, people would be authorized to use the standard to provide quality diagnostics on the code."

The hope, Curtis said, is to spawn a new market with new tools for this function. "CAST is in that market, security vendors are in this market, all providing different ways to analyze the code base. What the OMG has found, is when you have a standard it generates a lot of entrepreneurial opportunities and spurs growth of the technology market. When you have a standard you give the customer base confidence that smart people have thought this through, and if you pick a vendor you're not locking yourself into a dead end. It does spur the marketplace and competition for providing technology."

Curtis said CISQ's goal is to have an initial draft of a quality standard by the end of next year. What form that will take is still under discussion.



Tags: Automated software testingSoftware testing and quality assurance (QA) fundamentalsVIEW ALL TAGS

Digg This!    StumbleUpon Toolbar StumbleUpon    Bookmark with Delicious Del.icio.us    Add to Google



RELATED CONTENT
Automated software testing
ThoughtWorks Studios' Mingle captures "murmurs" and "waves" around project
Accelerating Agile testing with computer assistance
Improving software testing productivity using record-playback
Using automation to speed up software testing in Agile
Software testers facing six big challenges today, StarWest keynoter says
Affordable automated testing tools for securing websites
Classic inspiration for modern software test problems in QA
Expert advises on implementation of Selenium IDE for effective software testing
When should regression testing occur in an automated test plan?
Calculating mean time to failure in performance testing

Software testing and quality assurance (QA) fundamentals
How to deal with iteration issues in Agile
Five steps to fostering better software tester and QA results
Software Testing: New software testing technologies bring new challenges
Testing strategies for complex environments
Astronaut's STPCon advice: Teamwork delivers "The Right Stuff"
How to make your software tamperproof
Demo: Using WebGoat, a free software testing tool
Seven steps for a quality change and configuration management program
Winning responses to "Why is QA always the bottleneck?"
Where to find good methodology guides for software testing

RELATED GLOSSARY TERMS
Terms from Whatis.com − the technology online dictionary
automated test equipment  (SearchSoftwareQuality.com)

RELATED RESOURCES
2020software.com, trial software downloads for accounting software, ERP software, CRM software and business software systems
Search Bitpipe.com for the latest white papers and business webcasts
Whatis.com, the online computer dictionary



Software Development Methods - Extreme Programming, Agile Programming, Scrum
About Us  |  Contact Us  |  For Advertisers  |  For Business Partners  |  Site Index  |  RSS
SEARCH 
TechTarget provides technology professionals with the information they need to perform their jobs - from developing strategy, to making cost-effective purchase decisions and managing their organizations' technology projects - with its network of technology-specific websites, events and online magazines.

TechTarget Corporate Web Site  |  Media Kits  |  Site Map




All Rights Reserved, Copyright 2006 - 2009, TechTarget | Read our Privacy Policy
  TechTarget - The IT Media ROI Experts