Hello Everyone, I am new to the forums and pretty new to Quality.... I would like to say I am pleased to have found you!

Anyways....
What do you consider the difference of CMM internal Audits compared to ISO internal Audits for software. At this point I can only see the difference is prehaps some of the Quality Manual. I am interested in what others may think/know is the difference.


Thanks
Kimberly

I can't really say. I'm not an expert and the forums have never attracted software specialists. But CMM is a development plan in essence. You would take the CMM (Capability Maturity Model) requirements and audit them just as in an ISO 9001 audit you audit to the standard (and processes). In auditing CMM you would be auditing the program status and what point you are compliant to.

Maybe there's someone else which has more knowledge of CMM who can help out further and/or clarify my comments.

Sorry I can't help more.

Thanks Marc,

I guess I should have been a little more specific. I know CMM and I have performed CMM internal audits. However, for software production in Germany, they tend to lead towards ISO instead of CMM. Therefore, because I have only experience in the area of CMM, I only wanted to verify that I understood (in theory) some of the differences.

Kimberly

Are they leaning towards ISO 9001 or ISO 9000-3?

Personally, I have successfully used both models to effect significant improvements in Quality in software organizations.

Although there was a significant difference between the CMM and ISO9001:1994, the addition of Continuous Process Improvement in 9001:2000 has helped narrow the gap.

The main difference that remains is that the CMM is specifically oriented to Software Development organizations, whereas ISO9001 is by its nature generic to all industries.

The CMM is also a "staged" model (as they say), meaning that it contains a "roadmap" for improvement. That is, there is some guidance as to which specific quality management practices are generally more useful first. The downfall of the CMM as it often gets used is to believe that it is a checklist rather than a guide. (However, the same error can be made with ISO9001 as well.)

In my experience, the most important question to ask is whether the improvement program (ISO or CMM, it doesn't matter much) is being implemented for marketing reasons or for management reasons.

By the way, a full description of the difference between the CMM and ISO9001:1994 can be found on the SEI website. (http://www.sei.cmu.edu/publications/documents/94.reports/94.tr.012.html)

Mfg,
Bruce

Thanks for your answers......

Bruce... thanks for your information however, I do not want to know the difference between ISO and CMM only the Audits. I have already read the SEI information and many other documents that compare ISO to CMM; But no one address Audits that I have found so far.

Marc.... I would say I am more interested in the ISO 9001. I do not actually work for anyone, I am only studying ISO and was curious if there was much of a difference.


Kimberly

Kimberly,

It is difficult to answer your question properly, because strictly speaking, there is nothing called a "CMM Audit".

The purpose of the CMM is not certification, as is the case for ISO.

OK, now I will come down off my soapbox.

There are two types of operations which can be performed using the CMM.

The first is called "SCAMPI" (Standard CMMI SM Appraisal Method for Process Improvement), and generally occurs as follows (simplified explanation): A assessment team consisting of 1 or 2 outside experts plus 2 - 4 internal people reviews a representative cross-section of running projects, looking at the written procedures and the documented evidence, and then interviews with key people in the organization, including groups of development engineers. From this gathered information, the assessment team judges which key practices of the CMM are satisfied and which areas need additional work. These findings are presented back to the organization and then to Management as the basis for Continuous Improvement.

The second method is called SCE (Software Capability Evaluation) and consists of a customer's experts visiting a supplier or potential supplier and evaluating the maturity of the supplier's processes against the CMM. This has more the flavor of an audit.

This, of course, is the "official" description. In reality, many factors can influence the real performance of these assessments, so that they can become more like audits.

I hope this provides the information you were looking for.

Bruce (SEI-trained as a Lead Assessor in 1996)

Bruce,
Simple but excellent explaination. Thanx.
I have one question. When a company says 'We are CMM-Level x company' (I've yet to see any company below x=4 !), who certifies that?
rgds,
-Atul.

Atul Khandekar said:

Bruce,
Simple but excellent explaination. Thanx.
I have one question. When a company says 'We are CMM-Level x company' (I've yet to see any company below x=4 !), who certifies that?
rgds,
-Atul.

See another of my posts about "grade inflation". When the first companies started reaching level 3 in the early 90s, they would proclaim victory with all appropriate fanfare. Today everyone is 4 or 5. (Personally, I have trouble believing it.)

Anyway, to answer your question, OFFICIALLY, there is no such certification. OFFICIALLY, the purpose of an assessment is to identify process strengths and weaknesses, to better be able to manage the organization.

HOWEVER, the final assessment report and ratings MAY be made public if desired, and they MAY indicate that Company X, as judged by SEI-Approved Lead Assessor Y, satisfies the characteristics associated with level Z of the CMM. The veracity of this statement is backed up only by the integrity of the Lead Assessor, but as with many marketing claims, it's the first impression that counts.

Not that I would EVER call into question the moral fiber of the Assessor community; after all, they are just as irreproachable as Big 5 accounting firms (oops, did I say 5? I mean 4, or is that 3 now?):vfunny: :vfunny: :smokin:

Bruce

One last word, which helps sum up the differences between ISO certification and CMM assessment:

An official ISO certificate (quoting here from my company's) states:

"The QUALITY SYSTEM of Company X has been found to conform to the Quality System Standard ISO9001:1994."

The final report of a CMM assessment states:

"The performed processes of Company X have been found to satsify the goals of Level Y of the CMM".

A subtle but important distinction, often overlooked.

Bruce

Bruce,

Thank you for the reply. I have always wondered about the CMM level declarations - since majority of level 5 companies are here, in India. And as you say, it is indeed amazing what can be used as marketing material.

Is a certain level of CMM a requirement for getting business from customers in the US or Europe? Are customers asking for certain certification (CMM/ISO/other), esp. for big software projects?
rgds,
-Atul.

Bruce .... You are missing the point of my question. In order for a company to be declared ISO certified or to proclaim a CMM level, they should have gone through a event that should have analysed if they meet certain requirements that those standards have set. I myself call this an Audit. If you want to consider this is not really an Audit for CMM that is fine by me, but the definition of audit is to exam documents.

I have preformed what I will call an CMM Evaluation! (If this pleases you more.) But I have no experience with ISO. I guess I should have just ask how to preform an ISO audit to get my answer but hindsight is 20/20! I was really wanting to more about ISO Audits and the only thing I know is CMM evaluations so I though it might help me make the connection better knowing the differences.

And before I say something I regret I will leave it at that!

Kimberly

Plainly I prefer ISO compared to CMM. In ISO there is a surveillance audit every 6 months to have check on the QMS. In CMM they dont use the word "audit". They call it "assessment"

It is a one time affair. After assessment they declare the level and the curtains are down. The cost factor is enormous.

If in an organisation ISO is implemented in toto then I dont think the need for CMM. The mindset of the customers should change. They all feel that something - gold bar or kimberley diamond is stored within CMM. It is nothing like that.

ISO started and around the globe they started this activity with a model, etc., etc., Take for instance this CMM developed by Carnegie Mellon University. Some day some other University in USA /UK or Europe can come out with another model. Deos this mean that organisations should get certified complying to all standards.

The world recognises ISO and ISO is the best, irrespective the company i hardware, software, chemical, pharmaceutical, bank,textiles, automobile etc.,

Kimberly,

Forgive me if my answer seemed "off topic" compared to your question. Whereas I was trained to lead CMM assessments, I have only been the "target" of ISO and QS audits. I had hoped that one of the certified ISO auditors in this forum would have complemented my answer with another from the ISO perspective.

Since that didn't seem to happen, here is another attempt to compare the two events.

ISO Audit: External person arrives, meets with Senior Management, meets with the Quality representative, may meet with some individuals. Looks at the Quality System, looks at evidence of adherence to the Quality System. Produces a report of non-conformities.

CMM Assessment: Team of mixed external and internal people meets with individuals in all roles and all levels of the organization. Looks at the Quality System, looks for evidence of systematic adherence to the Quality System. Produces a report including overall rating, and recommendations for improvement.

On the surface, the only big difference is the Team approach versus an Individual approach. My experience, however, is that an ISO audit tends to be more adversarial, us-versus-them, how can we best hide our non-conformities from the auditor, whereas a CMM assessment is more open and sincere.

Your mileage may vary.

Bruce

Venkat,

Theoretically, organizations who wish to maintain CMM ratings are "encouraged" to undergo re-assessments every 12 - 18 months. If a company is advertising a CMM rating based on an years-old assessment, then I guess the watchword is "caveat emptor".

I certainly didn't mean to imply that there was any kind of competition between CMM and ISO. And, just to avoid the spread of misinformation, yes, the CMM was developed at CMU, but under contract from the US Government. It was not something that one university just happened to develop.

Bruce

In ISO they the overall compliance to the ISO 9001:2000 standard and if any deviations are noticed they suggest you to take corrective action and subsequently recommend for certification. It is in rarest of rare cases that the auditors refuse to certify because of high nonconformities.

But in CMM the events are different. If you are assessed at level 5 and if one of the key process areas of level three is not satisfied you will be assess at level 2. Inspite of spending huge money for certification and if the final outcome is this the morale of the QA team will plunge.

In this context the ISO audits are conducted well. We have been recommended for ISO certification and the auditors interviewed the team and not the individuals.

The objective evidence is crucial for the audit and the lack of objective evidence cannot bring down the level of assessment

Audit, Audit, Audit ..... We all know audit from the ISO perspective. But CMM they call it as assessment. Technically in an audit you cannot fail but in an assessment you can. This is how the business goes. There are companies who have gone upto the assessement level and made to eat humble pie.