Table of Contents
The Capability Maturity Model (CMM) and the ISO 9000 series of standards share a common concern with quality and process management. The two were created under the notion that to create a quality product, the underlying organization must be a quality driven. The purpose of this report will be to introduce the Capability Maturity Model and ISO 9000 as each relates to software development (ISO 9001), indicating their differences and similarities.
The Capability Maturity Model (CMM) is made up of five hierarchical levels that are used to grade an organization's ability to consistently and predictably create high quality software.[1] CMM does this by accessing the extent to which an IT organization uses predictable, manageable processes for building information systems. "By using record keeping and assessment tools based on this model, an IT organization can determine how its processes compare to a theoretical ideal and can see how quickly it is moving toward that ideal."[2] The underlining premise of CMM is that an organization that does not have a defined and standardized software development process is unable to provide a consistent and reliable product. The largest proponent of the Capability Maturity Model is the U.S. Department of Defense (DoD), which uses the model as a baseline to determine whether any specific organizations is qualified to bid on a US Military contract. Incidentally, the Software Engineering Institute (SEI), the author of CMM, was established by, receives its funding and mission from the DOD [3]
The purpose of CMM is to help organizations that produce software, improve the maturity of their software processes. The improvement is conceptualized as an evolutionary path from an AD-HOC, chaotic stage, to mature, disciplined software process. Each level within the CMM framework is referred to as a 'maturity level.' Each maturity level consists of several key processes areas (KPA) that identify requirements/best practices that an organization would need to demonstrate, to be graded at any specific maturity level. Please review Table 1 for detailed information regarding the maturity level framework of CMM.
Table 1 [4] | |||
Maturity Level |
Rating | Description | KPAs... |
5 | Optimizing | Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies. | cover the issues that both the organization and the process must address to implement continual,
measurable software process improvement. The KPAs are:
|
4 | Managed | Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled. | focus on establishing a quantitative understanding of both the software process and the software
work products being built. The KPAs are:
|
3 | Defined | The software process for both management and engineering activities is documented, standardized and integrated into standard software processes for the organization. All project use an approved, tailored version of the organization's standard software process for developing and maintaining software. | address both project and organizational issues, as the organization establishes an infrastructure
that institutionalizes effective software engineering and management processes across all projects.
The KPAs are:
|
2 | Repeatable | Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on project with similar applications. | Focus on the software project's concerns related to establishing basic project management controls.
The KPAs are:
|
1 | Initial | The software process is characterized as ad-hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics. | None |
KPAs are organized into a set of five common features "that help indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting." [5] The five common features of all KPAs are: Commitment to Perform; Ability to Perform; Activities Performed; Measurement and Analysis; and Verify Implementation. See Table 2 for detailed information about the five common features of KPAs.
Table 2 [5] | |
Common Feature | Description |
Commitment to Perform | Describes the actions the organization must take to ensure that the process is established and will endure. Includes practices on policy and leadership. |
Ability to Perform | Describes the preconditions that must exist in the project or organization to implement the software process competently. Includes practices on plans, procedures, work performed, tracking, and corrective action. |
Activities Performed | Describes the roles and procedures necessary to implement a key process area. Includes practices on plans, procedures, work performed, tracking, and corrective action. |
Measurement and Analysis | Describes the need to measure the process and analyze the measurements. Includes examples of measurements. |
Verifying Implementation | Describes the steps to ensure that the activities are performed in compliance with the process that has been established. Includes practices on management reviews and audits. |
An organization achieves a specific maturity level by means of an assessment, initiated by the organization wishing to be assessed. The assessment can be conducted by the organization being assessed or by an independent agency (SEI or SEI-licensed assessment vendor). The assessment provides feedback to the organization regarding its current software development capabilities and trains the organization on ways to improve its capabilities.[6]
Table 3 [6] | |
Phase | Description |
Selection | The Organization is identified as an assessment candidate, and the qualified assessing organization conducts an executive-level briefing. |
Commitment | The organization commits to the full assessment process whereby CEO signs an assessment agreement. |
Preparation | The organization's assessment team receives training, and the on-site assessment process is fully planned. All assessment participants are identified and briefed. The maturity questionnaire is completed by the organization. |
Assessment | The On-site assessment is typically conducted in a week. The assessment team then meets to formulate preliminary recommendations. |
Report | The entire assessment team helps prepare the final report and present it to the organizations assessment participants and senior management. The report includes team findings and recommendations for actions. |
Assessment Follow-up | The assessed organization's team, with guidance from the independence assessment organization, formulates an action plan. After approximately 18, months, it is recommended that the organization have a reassessment in order to assess progress and sustain the software process improvement cycle. |
An assessment conducted by a SEI-certified organization logically would be viewed to be more credible and objective that of a self-assessment. However, in a paper by Goldenson and Herbsleb with SEI, found that there is "evidence that people in fact try to answer survey questions honestly." The study, "After the Appraisal: A systematic Survey of Process Improvement, its Benefits, and factors that Influence Success, found 'little difference between the appraised and reported maturity levels, " of submitted information provided to SEI by organizations that had been previously assessed by SEI from 1992 to 1993.[7] However, an additional explanation could be that organizations that show significant statistical difference between an appraised review and a self reported review might be singled out for further study.
Table 4 [8] | ||
Category | Range | Median |
Years involved in CMM | 1 - 9 | 3.5 |
Yearly Cost per Engineer | $490 - $2004 | $1375 |
Productivity Gain/Year | 9% - 67% | 35% |
Early Defect detection gain per year | 6% - 25% | 22% |
Yearly reduction in time to market | 15% - 23% | 19% |
Yearly Reduction in post-release defect reports | 10% - 94% | 39% |
Business Value | 4.0 - 8.8 | 5.0 |
ISO 9000 is a family of standards, published by the International Organization for Standardization, is a set of five individual, but related international standards on quality management and quality assurance. They are generic and not specific to any particular product or service. ISO provides a certification process to organizations whereby stating to other organization that the certified organization has a Quality Management System in place and that the organization adheres to this system in conducting business. Please refer to Table 5 to a brief review of the ISO 9000 family of standards.
Table 5 [9] | ||
ISO Standard | Title | Description |
ISO 9000 | Quality Management and Quality Assurance Standards--Guidlines for Selection and Use | Guidelines for the selection and use of ISO 9001, 9002 and 9003. |
ISO 9001* | Quality Systems--Model for quality assurance in design/development, production, installation and servicing. | Standard covers design, development, production, installation, and servicing, this applies to the software industry. |
ISO 9002 | Quality systems--Model for quality assurance in production and installation. | Assesses the production and installation processes. |
ISO 9003 | Quality systems--Model for quality assurance in final inspection and test. | Evaluation the final inspection and test phase. |
ISO 9004 | Quality management and quality system elements--Guidlines | Defines the 20 fundamental quality system concepts included in the three models. |
*Of the ISO 9000 series, ISO 9001 is the standard most pertinent to software development and maintenance.[10]
ISO-9000 Standards generally are only used by organizations when: [10]
An organization can choose to be certified against one of the three quality systems in ISO 9000: ISO 9001, ISO 9002 and ISO 9003. The organization undergoing certification chooses which standard they wish to pursue based on the organizations business processes. None of the standards are considered more important than any one of the others and the organization that would undergo certification would need to identify which standard before pursuing certification. Once an organization chooses to undergo certification, a certified register audits the company to determine compliance with the applicable standard.
Please refer to Table 5, for a typical timetable for an organization preparing for certification.
Table 6 [11] | |
Time Period | Action(s) |
Months 1-3 | Organization commits at a board meeting to be certified within 14 months. Employees are instructed to familiarize themselves with the ISO 9000 concepts. |
Month 4 | Organization hires a certified quality auditor as a consultant to help prepare the company. |
Month 5 | Consultant spends one week talking to groups of 17 to 40 people about how they do their jobs and about existing company quality programs. Consultant submits 26-page document of findings and recommendations. |
Month 6-8 | Organization records descriptions of quality procedures and adds other procedures as recommended by the auditor. |
Month 9 | Organization completes all written materials and mails them to the auditing firm as a manual. |
Month 10 | Auditor reviews the manual, makes suggestions, and schedule an on-site visit in two to four months. |
Month 11 | Organization makes more changes as recommended by the auditor. |
Month 12 | Auditor makes an on-site visit for up to three days, concluding with a conference to discuss concerns and to listen to the organizations explanations. Auditor either grants or denies ISO 900 certification. |
Only a certified register can award ISO certification. To be a certified register, an organization must be accredited and its auditors must be certified. In the United States, the sole accrediting organization is the Register Accreditation Board.
The Similarities of the Capability Maturity Model to ISO-9000 Standards [12]
The Differences Between the Capability Maturity Model to ISO-9000 Standards [12]
Table 7 [13] | |
ISO | CMM |
Minimum requirements with implied continuous improvement | Explicit Continuous Quality Improvement |
Not specific to any one industry or service | Software specific |
Outwardly focused from the firm | Inwardly focused to the firm |
Registration Document | No Documentation |
Continual Audits | No follow up audits |
In many ways it is very difficult for the casual reader to get beyond the notion of thinking of CMM or ISO-900 Standards as nothing more than a set of prescription's you endure to get a participate with so to get a rating or certification. However, to accept this would be to completely misunderstand the CMM or ISO 9000 models. In a paper sponsored by the State of Washington regarding the Capability Maturity Model, the state said, "CMM is the foundation for systematically building a set of tools, including a maturity questionnaire, which is useful in software process improvement. The essential point to remember is that the model, not the questionnaire, is the basis for improving the software process." (http://www.wa.gov/) In Mark C. Paulk with SEI's review of the differences and similarities of ISO and CMM, he noted first off, both "have a common concern of quality and process management." So, in essence both models are concerned with that to produce quality in software, whether implicitly stated or not, the underlying organizations processes and systems are what dictates whether an organization can deliver a quality software product. That, is much must be considered and reviewed first before an organization can ever hope to produce repeatable, quality driven results.
[1] |
Saiedian, Hossein and Richard Kuzara, "SEI Capability Maturity Model's Impact on Contractors," IEEE Computer, January 1995, p. 16-25.
|
[2] |
Alter, Steven. Information Systems: A Management Perspective. 3rd ed. New York: Addison-Wesley, 1999. p. 428.
|
[3] |
Carnegie Mellon University-Software Engineering Institute: About the SEI-Welcome: (http://www.sei.cmu.edu/about/about.html)
|
[4] |
Carnegie Mellon University-Software Engineering Institute: Capability Maturity Model for Software (SW-CMM): (http://www.sei.cmu.edu/cmm/cmm.sum.html)
|
[5] |
Paulk, Mark C., "A Comparison of ISO 9001 and the Capability Maturity Model for Software." Software Engineering Institute, July 1994.
|
[6] |
Saiedian, Hossein and Richard Kuzara, "SEI Capability Maturity Model's Impact on Contractors," IEEE Computer, January 1995, p. 16-25.
|
[7] |
Goldenson, Dennis R., and James D. Herbsleb., "After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success." Software Engineering Institute, August 1995.
|
[8] |
Herbsleb, James., Anita Carleton, James Rozum, Jane Siegel and David Zubrow., "Benefits of CMM-Based Software Process Improvement: Executive Summary of Initial Results." Software Engineering Institute. September 1994.
|
[9] |
Dawood, Mark., "It's Time for ISO 9000." CrossTalk. (http://www.stsc.hill.af.mil/CrossTalk/1994/mar/xt94d03i.asp). October 22, 2001.
|
[10] |
Paulk, Mark C., "How ISO 9001 Compares with The CMM." Software Engineering Institute, January 1995.
|
[11] |
Brokaw, Leslie,"ISO 9000: Making the Grade," INC, June 1993.
|
[12] |
Craft, Dave. "ISO-CMM: Similarites, Differences." Slides 16 and 17. (http://www.umsl.edu/~sauter/analysis/cmm_iso/iso-cmm 11-22/index.htm)
|