Auditing for effectiveness

J

John C

#1
I notice a reluctance on behalf of Registrars to write up anything which is not clearly against a specific requirement of ISO 9000. In 4.17, the standard requires internal auditors to "determine the effectiveness of the quality system". Note that first we verify whether activities comply with the quality system - presumably the answer to that can be 'they do'. Then a jugement is asked as to whether this correctly implemented system is 'effective'.
This is obviously not limited to being objective. It becomes a judgement call. Yet it is of vital importance. I've heard a registrar say that a quality system may meet the requirements of the standard and yet produce junk. But we can't reconcile this requirement with anything but excellence and continuous improvement, which, I would think, was the purpose behind putting it in there.
Soon to be presenting myself as a consultant, I have to have a clear approach to this but it is probably the most difficult area to deal with. My conviction is that I should take the thing at face value and judge subjectively based on my own judgement of what is good and what is bad. But it looks like a minefield.
I'd appreciate the opinion and some feedback from the relevant experience of a cross section of correspondents; What do the consultants do? What do the in-house people and their colleagues expect? What do the registrars say?
thanks and rgds,
John C
 
Elsmar Forum Sponsor
D

Don Winton

#2
John,

You bring up an interesting subject, as always.

I notice a reluctance on behalf of Registrars to write up anything which is not clearly against a specific requirement of ISO 9000.
While some may dispute this statement, I am not surprised by it. The primary purpose of the registration assessment is (should be) to determine compliance, not noncompliance, to the standard. Therefore, logic would dictate that in order to 'write up anything,' there must be a area where compliance is not demonstrated.

Now, the 'gray' area.

In 4.17, the standard requires internal auditors to "determine the effectiveness of the quality system".
Under the assumption that an organization is ISO 900x compliant for the right reason, then the internal audit program is the key to effective implementation. The Registrar cannot (should not) determine effective implementation, just compliance. The internal audit program, then, is where the effective implementation must be determined. The internal auditors are presumably well versed on an organization's operations. This being the case, they would be the ones to determine effective implementation. One could assume, then, this is the reason the clause in 4.17 is there.

I've heard a registrar say that a quality system may meet the requirements of the standard and yet produce junk.
This is true. I have seen, and still see, it in action. I remember a discussion a while back that a company could produce concrete life jackets and still be ISO 900x compliant. After a study of the various responses and dialog, I had to agree. I find nothing in the standard that could prevent me from operating a concrete life jacket factory and still be ISO 900x compliant. But, this would be an organization that is ISO compliant for the wrong reason.

My conviction is that I should take the thing at face value and judge subjectively based on my own judgement of what is good and what is bad. But it looks like a minefield.
That would depend upon your role and the scope of your responsibilities. As a consultant, the contractor is paying you for a service. Now, which service is he paying for?

They want an ISO compliant system: In this case, judging subjectively would probably be the mine field you described. They may want to be compliant just to satisfy some customer's requirement. The right thing for the wrong reason. Avoid subjectivity and establish a compliant system.

They want to establish, implement and maintain an effective Quality Management System. In this case you are free to subjectively point out areas where their effective implementation is 'short' and make suggestions on improvement.

As strictly an assessor, you should not be subjective AT ALL, just objective, at least during the assessment. After the closing meeting, you could 'offer' areas where the effectiveness is short, but they would be under no obligation to take the advice and you cannot audit against it during future assessments. I STRONGLY advise against offering any suggestions if your role is as assessor.

Am I completely off base?

Excellent subject, BTW.

Regards,
Don


------------------
Just the ramblings of an Old Wizard Warrior.

Check Out dWizard's Lair:
www.ficom.net/members/donwinton/home.html
 
J

John C

#3
Don,
Not off base but right on target, I'd say. You blasted my concerns past the floodlights and they're still going. I see now how the minefield only happens if you go outside your brief - the customer is always right so make sure you really know what their expectations are. It reminds me of what a colleague who left here told me just a couple of weeks ago; She now works in a chemical plant on quality systems and said she has to listen for a long time before suggesting anything; If you want to change anything, they come over suspicious and worried. "What's wrong with it?", What happened?", almost to the point of saying; "What are you suggesting?" Not the same thing, exactly, and I'm not saying that all such plants are like that, but it seems like a place where to go outside objective observations could land you in the minefield. My preferred clients will be chemical plants - I'm planning to combine 9000 and 14000 in my product/service range, so it's good that I now can categorise this attitude as somewhere in between the 'right reasons' and the 'wrong reasons'. I guess it's a case of "the right reasons, but remember there's more at stake here than a quality certification", situation.
So my mind is put at rest on the 'effectiveness' issue and I can make my plans to deal with it. Thanks again.
rgds, John C
 

Marc

Hunkered Down for the Duration
Staff member
Admin
#4
I notice a reluctance on behalf of Registrars to write up anything which is not clearly against a specific requirement of ISO 9000.
Start worrying when they cite you for something that's not in ISO9001.
 
D

Don Winton

#5
... the customer is always right...
This is something I preach in my classes. WRONG! WRONG! WRONG!

The customer is not always ‘right;’ in fact they are seldom right. BUT, they are always the customer. It is our job to be able to explain the difference.

... know what their expectations are.
This is the key to what I was trying to expound upon above. Stay within the scope of the customer’s expectations.

So my mind is put at rest on the 'effectiveness' issue and I can make my plans to deal with it. Thanks again
Do not let the mind rest too easily. You have many a hard road ahead. Best of luck.

Start worrying when they cite you for something that's not in ISO9001.
Agreed.

Regards,
Don

------------------
Just the ramblings of an Old Wizard Warrior.

Check Out dWizard's Lair:
www.ficom.net/members/donwinton/home.html
 
D

Don Winton

#6
Start worrying when they cite you for something that's not in ISO9001.
Actual 'observation' from a recent Document Review:

"Although reference is made on individual pages in the Quality Manual to ISO standard’s paragraphs, there is no statement in the body of the policy or quality system manual or procedures of the intent of compliance to ISO 9002."

This was sited against 4.1.1. Thought I would share.

Regards,
Don

------------------
Just the ramblings of an Old Wizard Warrior.

Check Out dWizard's Lair:
www.ficom.net/members/donwinton/home.html
 
Thread starter Similar threads Forum Replies Date
B Auditing Senior Management to determine the Effectiveness of Management Processes IATF 16949 - Automotive Quality Systems Standard 6
M Auditing ISO9001/AS9100 7.3 Design and Development Quality System Effectiveness Manufacturing and Related Processes 12
V Auditing effectiveness of corrective actions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 1
L Internal Auditing & Assessing Effectiveness Internal Auditing 8
L Internal Auditing & Assessing Effectiveness Internal Auditing 8
C List of MDSAP Auditing Organizations Medical Device and FDA Regulations and Standards News 1
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 4
cscalise Suggestions for MDR Auditing tools EU Medical Device Regulations 1
J Auditing of Support Function IATF 16949 - Automotive Quality Systems Standard 9
D ISO 13485, FDA 21 CFR 820 and Auditing the Accounting Department ISO 13485:2016 - Medical Device Quality Management Systems 5
S Risk based internal auditing Internal Auditing 6
Randy Remote auditing (for disaster, disease, disturbance etc...) during the Neo Coronavirus Pandemic and Social Distancing Registrars and Notified Bodies 7
K ANVISA B-GMP Auditing requirements for Contract Manufacturers Other Medical Device Regulations World-Wide 0
F AS9100D Internal auditing requirements Internal Auditing 3
R Does any here use an internal auditing tool that works on different platforms? Internal Auditing 3
W Does anyone have an API Q2 checklist for internal auditing? Oil and Gas Industry Standards and Regulations 1
G Best Practices for IT auditing - Is a session-id necessary for a complete audit trail? IEC 27001 - Information Security Management Systems (ISMS) 0
I Questions to ask when auditing for Organizational Leadership and Planning for the QMS? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
C CE marking for general IVD (self-certified) & ISO 13485 QMS requirements - auditing EU Medical Device Regulations 6
blackholequasar Internal Auditing Inspiration - Getting volunteers to perform internal audits. Internal Auditing 22
W Internal Auditing carried out by a 3rd party - Review of previous audits AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 3
tony s What is the automotive process approach for auditing? IATF 16949 - Automotive Quality Systems Standard 2
S Internal Auditing for API Spec Q1 - auditor qualification requirements Oil and Gas Industry Standards and Regulations 6
R I've been auditing for a CB for 18 years General Auditing Discussions 10
P Consultant Auditing Qualifications Requirements ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 23
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 146
M We still have not received our certificate due to a 'backlog' with our auditing body Registrars and Notified Bodies 25
N Online Internal Auditing Course for ISO 13485 - Suggestions ISO 13485:2016 - Medical Device Quality Management Systems 8
A Agenda for 8D audit on Supplier's side - Auditing Corrective Actions General Auditing Discussions 5
U Internal auditing - Company employees or contract second party Internal Auditing 10
J Recomended Values - Auditing process in a supplier IATF 16949 - Automotive Quality Systems Standard 18
M Canada - Registrars that allow e-auditing for ISO 9001? Registrars and Notified Bodies 4
K Internal Auditing - Umbrella QMS and Multiple Standards Oil and Gas Industry Standards and Regulations 4
D Auditing Our Outsourced 2nd-3rd Party Internal Audit Company ISO 13485:2016 - Medical Device Quality Management Systems 6
supadrai Auditing Organization dragging their heels on issuing our MDSAP Surveillance Audit Confirmation Letter - everyone is nervous ... are we the only ones? Canada Medical Device Regulations 7
Ed Panek Supplier Auditing - No purchases from our key suppliers in the last 24 months ISO 13485:2016 - Medical Device Quality Management Systems 5
P Auditing "process validation" process 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 2
qualprod Effective Auditing advice needed ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 16
M Acceptance of remote auditing techniques - Can you help me with my research? General Auditing Discussions 0
GStough Auditing Against Criteria Unfamiliar to Auditee - Yea or Nay? General Auditing Discussions 11
qualprod Auditing Product and Services doubts ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
R Auditing support and management processes General Auditing Discussions 7
F It is acceptable moving remote locations staff to manufacturing plant for auditing? IATF 16949 - Automotive Quality Systems Standard 3
D MSDS / GHS Walk-through / Auditing Occupational Health & Safety Management Standards 6
Pmarszal Supplier Auditing Services (Audit Needed?) General Auditing Discussions 4
S ISO 9001 Audit Observations - Transitioning my career into auditing Career and Occupation Discussions 16
G AS9101 Rev F - Worksheets for internal auditing AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 11
N API Q2 clause 6.2.2.1 Auditing Outsourced Suppliers Oil and Gas Industry Standards and Regulations 5
M Auditing processes followed by employees placed on client's site Internal Auditing 4
S ISO 13485:2016 and MDSAP internal auditing ISO 13485:2016 - Medical Device Quality Management Systems 6

Similar threads

Top Bottom