SBS - The Best Value in QMS software

Tick-IT vs. ISO 9001 vs. CMM - Software Quality Assurance

Marc

Fully vaccinated are you?
Staff member
Admin
#1
Tick-IT & ISO 9001

From: ISO Standards Discussion
Date: Wed, 27 Jan 1999 08:54:28 -0600
Subject: Re: Tick-IT & ISO 9001 /Chen/Kirk/Peter/Dey

> Pat,
>
> Since you've made some general statements about the CMM and ISO, I'd
> thought I'd play a little devil's advocate with you. My point throughout
> this response is to offer up a theme. The theme is that models really
> don't amount to a hill of beans as much as a desire to change. I make
> this statement in support of the fellow who said that all models are
> wrong....some are useful. I would say that all models have warts...and
> that the key to software process improvement (or any improvement) is
> wanting to improve....to deliver products better and faster...

I agree.

> > The strength of the CMM is its model for continuous improvement. The SEI's
> > method includes, not only the process framework (the CMM itself) but the
> > methods for managing improvement by involving everyone, eg, through a
> > Software Engineering Process Group.
>
> But the reality is that the CMM is a staged model that doesn't really
> focus on defect prevention until level 5.

Hmm ... well as one devil's advocate to another:

1. Reviews are at level 3 in the CMM and, I hope, can be used for defect detection & removal (yes, not proactive defect prevention, but better than nothing).

2. I'm inclined to pull reviews down to level 2 because of the civilising behaviour they can be used to introduce to organisations. (They're at level 3, I believe, because they need the time management disciplines that level 2 introduces.)

3. More advanced processes like defect prevention are introduced at higher levels in the CMM after making the system stable, especially with level 2 (ie, Deming's idea of bringing the system into statistical control, then improving it - and ignoring what exactly "statistical control" means in software development). My experience of trying to introduce such stuff earlier (in ISO9k efforts) has been that the statistics merely measure noise & instability in the system, telling you little you don't know already. In other words, to follow an old software engineering adage, first design (the process) so that it works, then optimise it.

To put it another way: the staged model offers a strategy for what to do now, what to do later. Its philosophy is stabilise, then improve continuously - which logically puts proactive defect prevention last. What other strategies might there be for ordering the processes to put in place now, or later? (All together is too hard.)

> In addition, the SEPG can have
> the same problems you've referenced below with ISO auditors, in that they
> can end up driving an incredible bureaucracy that doesn't serve the
> developers or the rest of the product team.
>

A truth I had forgotten.

My experience is limited to organisations using the CMM willingly, mostly in Europe; not of organisations using it in the States unwillingly where I can imagine this happens

> In addition, the SEPG can't address a lack of management commitment.
> This is no different than a distributed model for ISO implementation that
> lacks true top-level support....you can have the buy-in from the troops
> and middle management and still fizzle.
>
Nothing and nobody can address lack of management commitment except the CEO, agreed.

> > A TickIT certificate is somewhere around level 2/3 of the CMM - it skews
> > across. the CMM has more software detail, ISO has more general business
> > stuff, both useful and overlapping.
>
> Actually, there is no true correlation between maturity levels and ISO
> implementation. However, it is true that there is strong support for ISO
> at all CMM levels, including Defect Prevention at Level 5. If one were to
> take a true organizational approach to an ISO implementation, then it
> would very much represent a level 3 organization. Both models pretty much
> say the same thing, whats not hows.....but one takes about 479 pages to do
> it.

I think ultimately they say more or less the same thing. But I maintain that the CMM, with its concept of levels, gives a strategy for getting there.

Specifically, I think for a software organisation to concentrate as the CMM suggests on level 2 processes such as planning, tracking, configuration management etc first, is a high leverage focus which pays immediate dividends in customer satisfaction. I think ISO9k lacks such a strategic sense of how to build the QMS.

Further, I think some form of the SEI's self assessment methodology is vital. It directly implements Deming's "involve everyone". Where is that, specifically, in ISO9k or TickIT? ISO 9k should insist upon involvement of engineers in continuous improvement, as it insists upon other necessary practices. Why doesn't it?

That's not to say the CMM can't lose it, when driven by unreasoning management hunger for a "CMM level Certificate"; but at least it's there in the SEI method, and in Watts Humphrey's book "Managing the Software Process" (seminal CMM reference for those who want the reference).

> > If an organisation is immature, the CMM offers a better strategy for
> > building a QMS because it offers a sense of priority . TickIT and ISO
> > require everything and can be overwhelming.
>
> If you remove the models and look at software engineering fundamentals,
> you have the same problem of trying to bite off more than you can chew. A
> phased approach in any implementation is necessary. And I think alot has
> to do whether you agree with the construction of the CMM which pretty much
> focuses strictly on the management side at level 2 and doesn't have an
> engineering focus until level 3.

That's because engineers know what they're doing and managers know neither what they nor their engineers are doing ;o)

> The simple framework that ISO offers can be phased-in on a project by
> project basis, with the areas that offer the biggest bang for the buck
> being addressed first.
>
> The biggest problem I see in SPI is that folks don't have good
> implementation planning skills. This is the same for the CMM and
> ISO...and when you look at how large the CMM model is and how little has
> been written about how to successfully implement it.....well...the job can
> be daunting.
>
> Although more commercial organizations are looking to the CMM for process
> improvement, it pales in comparison to the organizations that must
> implement or else....DoD contractors in bidding wars.

In telecommunications too process improvement is vital - customers typically ask for ISO9k but increasingly they understand and respect the CMM ideas - without insisting on achievement of a magic level (yet).

> > Further, the continuous references to clauses and how auditors might
> > interpret them takes ownership away from the people and gives it to
> > auditors. The SEI's approach leaves ownership with those who operate the
> > process, so it's better balanced, less inclined to be bureaucratic. Compare
> > the discussion traffic in this List with, eg, comp.software-engineering.
>
> The same darn problem exists in the CMM world. Don't kid yourself.
> Organizations face SCEs (software capability evaluations) and CBA
> IPIs....CMM based assessments for Internal Process Improvement. Many
> times the "Level Rating" is all that matters....even with the CBA IPI
> approach which is supposed to be a collaborative exercise for improvement.
> You would be surprised how many organizations coach their employees to get
> ready for a CBA IPI, when that is not the intent.....it's not supposed to
> be about the "score"....it's supposed to be about improvement.
>
> Again....this is a management issue similar to the ISO implementation that
> says....let's get the certificate.....and we're done.....
>
Yes. If management are strong then the certificate is real, else it's just paper and grief.

> > One way to approach this is to build the QMS using the SEI's CMM guidance,
> > document it soundly, include a reconciliation with TickIT clauses - and add
> > in the bits that the CMM does not explicitly require (eg, contract review,
> > security & backups, etc).
>
> Yes...there is a lot of information in the CMM that can assist one with
> an ISO implementation. I would also say that folks can also turn to the
> IEEE standards or ISO 12207 or other sources of information.
>
Quite so. Also, for software management, Software Program Managers Network has some good stuff.

> It's all fundamental stuff.
>
Indeed. Fred Brooks said a whole bunch of it 25 years ago, and yet we still have to teach managers that months are mythical. Why so?

> > Under the CMM, you can be a level 2, 3, 4 or 5 organisation (or, sadly,
> > level 1).
>
> Yes...but you could be a level 3 organization and that would mean little
> in many circumstances. For example, when was the organization's last
> assessment....3 years ago? I've been in shops that tout their level 3
> profile but were behaving as level 1 (chaotic). Remember CMM ratings are
> not a certification scheme of any sort. There is no requirement other
> than individual customer or market requirements that would require you to
> reassess your organization.
>
> > Under TickIT, you can be TickIT Certified. There's no measurement scale.
> >
> > Regards,
> > Pat
>
> Yes and TickIT is just an ISO 9001 registration, pure and simple. But the
> true measurement in implementing either model is whether you have a return
> on investment and whether it translates into better product and staying
> abreast of your competitors.
>
> The companies that succeed with the CMM and ISO succeed because they
> aren't driven by what is in the model and they overlook the shortcomings
> of the models. They embrace what is good for their business and question
> what is unnecessary. They go beyond the models to crush their
> competition.

I agree with that whole heartedly. I think several questions in this list are from organisations who did just that and are having trouble convincing their auditors of it. It's important to pick an auditor who truly understands your business - a valuable element of TickIT - so that you get firm but fair treatment.

> I've seen both models work effectively...and...I've seen them both fail
> miserably.
>
> The same can be said about Deming, Juran, Crosby, TQM.....
>
> For the most part, the model doesn't matter...change does....

Yes.

> For some organizations models can be handy, because they can hang their
> failure on choosing one particular model over another.....but that's
> another story.....
>

Regards,
Pat
 
Elsmar Forum Sponsor

Marc

Fully vaccinated are you?
Staff member
Admin
#2
From: ISO Standards Discussion
Date: Wed, 27 Jan 1999 09:17:08 -0600
Subject: Re: Tick-IT & ISO 9001 /.../Peter/Dey/Duong/Kohn

> I believe that CMM is better than ISO because CMM is dedicated to
> software. Whereas ISO was created for manufacturing first, adapted to
> software later.

Interesting argument, but I'm not terribly convinced. CMM doesn't provide much assurance to your customer beyond what your own word of honor could have provided.

If you look at the job market these days, especially in the IT industry, employers are beginning to really focus on whether job candidates are certified. Looking around this mailing list there are enough CQAs and QSLAs to fill 100 cans of alphabet soup.

Suppliers are not much different than people. To be sure that they are competent in meeting your needs as a purchaser, you need some assurances. Often, someone's word is enough; sometimes you need contractual protections; sometimes accredited certification to ISO 9001 or TickIT will give you the assurance you need. CMM just can't satisfy *that* need (though I feel it is surely better at satisfying the needs *it* was structured to meet.)

Brian

---------------snippo-----------------

From: ISO Standards Discussion
Date: Wed, 27 Jan 1999 09:18:30 -0600
Subject: Re: Tick-IT & ISO 9001 /../Dey/Deibler/Kohn

> Since you've made some general statements about the CMM and ISO, I'd
> thought I'd play a little devil's advocate with you. My point throughout
> this response is to offer up a theme. The theme is that models really
> don't amount to a hill of beans as much as a desire to change.

I think the biggest mistake folks pursuing ISO 9001 and TickIT could make is mistaking them for models for improvement. These two standards are tools for demonstrating to customers that you meet minimally acceptable standards for addressing quality. Use them however you wish; get out of them whatever you can from the standpoint of improving your business; but never forget that the point of the standards is to protect the customer.

Brian

----------snippo-----------

From: ISO Standards Discussion <[email protected]>
Date: Wed, 27 Jan 1999 09:20:08 -0600
Subject: Re: Tick-IT & ISO 9001 /Chen/Hale/Kohn

> The main thing you get with TickIT is an auditor with software
> qualifications.

Actually, you get an auditor with software qualifications when you get ISO 9001 registration services from any RAB- or RvA-accredited registrar, operating in compliance with procedures.

What you *do* get extra is assessment to a set of requirements that are either over-and-above ISO 9001, or simply more stringent or prescriptive than the corresponding requirement in ISO 9001.

> With Lloyd's Register Quality Assurance, the only extra you pay
> extra for is the $60 or so for the additional certification mark.

This is somewhat misleading. While this is perhaps true with Lloyd's, my experience is that the minimum number of assessor-days required to conduct a valid TickIT assessment are a bit more than the RvA requirements for the minimum number of assessor-days for an ISO 9001 assessment. That will make the costs proportionally more.

Brian
 
Thread starter Similar threads Forum Replies Date
T Directors View - Standards like ISO 9001 are "Just a TICK in the BOX" ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
C Measurement Uncertainty fluctuates half a tick mark (20 millionths of an inch) Measurement Uncertainty (MU) 1
V Tick IT - Regional representative for TICKIT at Chennai Software Quality Assurance 5
A ISO 9001 Internal Audits - No production right now due to furloughs Internal Auditing 3
D Question on Documented Calibration versus ISO 17025 Accredited Calibration ISO 13485:2016 - Medical Device Quality Management Systems 0
M Customer Property - ISO 13485:2016 Clause 7.5.10 ISO 13485:2016 - Medical Device Quality Management Systems 9
pbojsen ISO 13485 Requirements versus FDA product classification and GMP exemptions - Audits ISO 13485:2016 - Medical Device Quality Management Systems 3
D "certified" in ISO 19011, as well as IATF required? IATF 16949 - Automotive Quality Systems Standard 4
S ISO/IEC 15408 - Is this is Certifiable Standard? Other ISO and International Standards and European Regulations 2
D Lead time to schedule an ISO 13485 audit Auditing Quality and Environmental Management Systems 2
T Do we need an SOP for ISO 9001? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
K ISO 9001 Auditing in a Healthcare setting Auditing Quality and Environmental Management Systems 15
S Does anyone have a checklist to prepare for ISO 13485, Stage I audit? ISO 13485:2016 - Medical Device Quality Management Systems 1
H QMS ISO 13485:2016 - ISO14971 IEC60304 etc ISO 13485:2016 - Medical Device Quality Management Systems 2
Y How can i integrate ISO 13845 into ISO 27001? ISO 13485:2016 - Medical Device Quality Management Systems 4
vickyva ISO 14155:2020 CIP CIR templates Other Medical Device Related Standards 0
C ISO 9001:2015 8.3.2. h) Design and Development Planning - What is required? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4
B Employee Handbook in ISO 9001:2015 Section 7 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 19
B Operational Procedures for ISO 13485:2016 ISO 13485:2016 - Medical Device Quality Management Systems 7
Q ISO 9001/IATF 16949 Audit Finding Question - Document Retention IATF 16949 - Automotive Quality Systems Standard 10
D ISO 14971 applicability in ISO 13485 ISO 13485:2016 - Medical Device Quality Management Systems 7
G Trying to get a financially reasonable ISO Certification Body Registrars and Notified Bodies 8
L ISO 45001:2018 - Clause 5.4: Consultation and Participation of Workers Process Maps, Process Mapping and Turtle Diagrams 1
E ISO 13485 in Clinical Trial conduct: Applicable or No ISO 13485:2016 - Medical Device Quality Management Systems 2
G ISO 13485 Certification - Can we get the ISO 13485 certification prior to shipment of the device? ISO 13485:2016 - Medical Device Quality Management Systems 6
Richard Regalado Informational ISO/IEC DIS 27001:2021, to be published soon. IEC 27001 - Information Security Management Systems (ISMS) 0
Q Audit report template ISO 9001/14001 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 13
N Does anyone use SGS for ISO 13485 / CE certification Registrars and Notified Bodies 0
Q Process matrix examples of ISO 9001 & 14001 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
S Need ISO 15189:2012 Documentation toolkit. Document Control Systems, Procedures, Forms and Templates 0
chris1price Archiving of paper records - ISO 9001 7.5.3.1b Records and Data - Quality, Legal and Other Evidence 4
M Transferring ISO 17025 from one company to another ISO 17025 related Discussions 1
D Common practices in ISO 9001 deployment ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 17
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
P Audit check for IT company (ISO 9001) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
M Label Making & Printing Standards ISO / ASTM ISO 13485:2016 - Medical Device Quality Management Systems 5
Sidney Vianna Interesting Discussion Should ISO 9004 be changed from a guidance document to a requirements standard? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
Ed Panek ISO 13485:2016 Section 5.5.3 ISO 13485:2016 - Medical Device Quality Management Systems 3
Q Do these certificates of calibration meet ISO 9001 requirements for traceability to NIST? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
ebrahim QMS as per ISO 13485, Clause 4.2 Requirements for regulatory purposes for Medical Devices Authorized Representatives. ISO 13485:2016 - Medical Device Quality Management Systems 3
S ISO 2768-mk print call out Other ISO and International Standards and European Regulations 11
T ISO 17024, clauses 4.3.8. and 5.1.1. Other ISO and International Standards and European Regulations 4
C ISO 14001:2015 6.1.3 Compliance Obligations - Legal requirements monitoring ISO 14001:2015 Specific Discussions 0
C Requirement to link Quality Manual to ISO 9001 clause numbers? ISO 13485:2016 - Medical Device Quality Management Systems 13
D ISO 13485 scope (implantable) - Polymers for dental application EU Medical Device Regulations 9
W First time being audited (ISO 9001), asking for advice ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 9
John C. Abnet ISO 26262 ISO 26262 - Road vehicles – Functional safety 3
Marc ISO 26262- Road vehicles – Functional safety ISO 26262 - Road vehicles – Functional safety 0
John C. Abnet ISO 26262 IATF 16949 - Automotive Quality Systems Standard 0
A ISO/DIS 15223-1:2020 - Country of manufacture label (IEC 60417 No. 6049) - Which national law requires this symbol? Other Medical Device Related Standards 0

Similar threads

Top Bottom