An Internal Audit Challenge - Audit of the Quality System Effectiveness

C

cjssag

#21
Just because someone comes up with a ridiculous proposition doesn't mean one has to tear round in circles trying to actually do it.
To begin with, I want the thank those who contributed to this thread and to the many more who bothered to read it.

Now, let's see just how ridiculous (or not) this proposition was.

First, my challenge (forget the 15 minutes for a moment) was simply to assess the effectiveness of the QMS. It did not ask for a compliance assessment, as some responses seem to have presumed, nor does it ask for improvement opportunities, a valid goal of internal audits.

Now, why do I add the 15 minute restriction to my challenge? To demonstrate that, more often than not, we - internal auditors - have a very bad habit of losing sight of the prize. We get so "juiced up " over the trees that we can't see the forest. Are we so besotted by our own auditing skills that, like the pathologist who upon examining a heart he has just excised from a cadaver pronouces it "healthy"?

What is the prize?

It is the function of the QMS which, if you chose to use ISO 9001:2000, can be found at paragraph 1.1. (No I am not going to recite it here: look it up for yourselves. I'll bet it's the first time some of us have ever really read it.)

And whose QMS is it?

Para 5.1 and 5.3 pretty much spell it out for us. It's top management's system and, among other things, they need a quality policy and (measurable) objectives which, if we follow the logic, is the quality system's Mission Statement, i.e., what the QMS is expected to accomplish.

If the system is consistently realizing the quality policy as measured by the objectives, does that not constitute effectiveness?

So, how long will it take you to march into the boss' office, ask and get satisfactory answers to these questions?:
a) What is your quality policy?
b) Show me (i.e., objective evidence) that you are realizing it consistently?

The boss either can or can't provide the answers and evidence. The buck starts and stops with him/her. And you have at least 5 minutes left over for a coffee break.

If there is sufficient evidence that the system has been performing effectively over time, exactly what do you think your more detailed audits might uncover of any significance? Nonconformances to the standard or to their own documented procedures? Perhaps. But if the system is working, how troublesome might they be? (Please do not tell me that failure to comply with the letter(s) of the standard might jeopardize the "certificate" because - and here I go pouring gasoline onto the fire - certification has absolutely nothing to do with quality.)

Correspondingly, a qms that is foursquare "in accordance with all requirements of the ISO 9001:2000 standard" is absolutely no guarantee that a single customer is being satisfied. My answer to such a finding - if the system is not effective is - "Your system is broken" (actually my words tend to be more colorful than that since, in such cases, it's time to take out the 2x4 just to get the jackass' attention.)

Am I suggesting we should not do more detailed audits or not do compliance checks? Absolutley not. We have a job to do and we should do it: intelligently and efficiently. But I am more than suggesting that establishing the effectiveness (or not) of the system to consistently achieve its expressed purpose(s) is not just the first thing we should do but we must use the results of that assessment as a basis for our more detailed audit plan. Performance to measureable objectives is "objective evidence" and "Status and importance" really do have meaning here.

Yes, we must venture forth into the forest but we must not check our brains at the edge.

After all, internal auditing is a management activity and the language of management is money/results. Let's keep that ever at the front of our minds and K.I.S.S.

Someone asked if my challenge represented a real life situation. Yes. I pose it at the beginning of every on-site auditor training workshop I conduct and, again, at the end. Few get it right at the outset. All do at the end.

BTW, every one of my own internal audit cyles begins with the boss on the hot seat. The old saw about the head of the fish being the first part to die, is apropos.
 
Elsmar Forum Sponsor
#22
Nice idea, but like so many 'nice ideas' it's fundamentally flawed:mg::rolleyes:.........IMHO

I knew it was a just a game.....
 
Last edited:

Jim Wynne

Staff member
Admin
#23
Nice idea, but like so many 'nice ideas' it's fundamentally flawed.
It's no secret around here that I have no patience with people (consultants in particular) who take the bloody obvious and repackage it somehow, then claim to have had some sort of epiphany resulting in a Great New Idea. Here we go again. Not only is the OP's premise fundamentally flawed; it's DOA, because it makes the unfounded assumption that what's effective now will always be effective in the future. I think the OP might be surprised at the number of people here who not only have read the standard in its entirety, but are intimately familiar with it, else he would not have adopted the condescending tone.

It is no great secret that "top management" is ultimately responsible for the effectiveness of the system, but it's also no secret that (A) it's not unusual at all for top management to prevaricate about such things, and (B) many such managers are woefully ignorant. Suppose we do burst into the CEO's office and demand evidence, and wonder of wonders, she actually has some. Do you suppose it just dropped out of the sky? Do you think that the CEO gathered it all by himself? Do you think it can be trusted at face value? Do you think that faced with a prevaricating CEO, the hapless auditor will just pound his fist on the desk and demand corrective action???

If the CEO does have access immediate access to evidence, it came about as part of the management review process, which is based in part on audit results. I personally believe that internal auditing, as commonly practiced, could be immensely improved, and it's a subject we've discussed quite a bit here. But pointing out the fact that the CEO's office is where the buck is supposed to stop, and taking 1000 words to say it, isn't very helpful.
 
J

JaneB

#25
It's no secret around here that I have no patience with people (consultants in particular) who take the bloody obvious and repackage it somehow, then claim to have had some sort of epiphany resulting in a Great New Idea. Here we go again.
Well said, Jim. I fully share your lack of patience in this instance.

I also very much wish people/consultants wouldn't march in here and demonstrate this kind of approach. But then, this kind of approach is a very strong demonstration of their particular approach to consulting. And gives consultancy a bad name.

I wouldn't have anything to do with a consultant like this. But that doesn't mean all of us consultants do or are the same.
 
C

cjssag

#26
Jim and Jane, I appreciate your candor and obvious passion but you proved my point by missing the point.
I am not surprised.

My first duty as a consultant - and I have been doing it quite successfully for over 17 years - is to give my customer (i.e the boss) straight talk about the effectiveness of their business operating systems, of which the QMS is always the single most important part. I am well aware that the boss cannot do it all, that the boss has to depend upon many people to make the system work, that the boss is not the resident expert on all matters that make the system work, etc.

Jim cites instances where the boss is essentially clueless and I have seen more than my fair share of those situations. I have made it my practice to run away from them because if the boss really doesn't care, I am not going to waste my time trying to push a rope. You see, the sad truth is that when a system doesn't work, it doesn't matter whose fault it is, the bleep, bleep consultant usually gets blamed. I am not into self fulfilling prophesies.

I have made my living helping the bosses in all types of industries develop business operating systems that are designed to give them the results they want - using all the great folks that work for them. I have left behind me a trail of smiling faces - many of whom keep coming back for more straight talk- and none of them have ever so much as hinted that they never got less than they paid for. BTW, I know my system is effective by the repeat business I keep booking and by what my customers tell me. Is that wrong?

As I said at the outset, The Grouch is back. You can expect I will continue to state the obvious as long as folks keep losing sight of it. So if you think I used too many words to state the obvious or pose ridiculous questions or gored somebody's bloody ox... you're all big people, you'll get over it.

Jane, if I ever get to Australia I'll be sure to fly over to Melbourne on my broom to say hello.
 
J

JaneB

#27
Jim and Jane, I appreciate your candor and obvious passion but you proved my point by missing the point.
No. I don't think so. At all.

I'll refrain from a 'been consulting longer than you', because none of that is actually relevant.

I don't know anything about 'the grouch' being back - this post was a first for me.

But I stand by all my comments. And agree with Jim in all of his (with an exception on the consultant-bashing front).

I am not surprised.
I'm attempting to find a very polite way of saying that the tone of some of what you say sounds as though you are coming from a position of superiority and greater knowledge than everyone else (who have 'missed the point' according to you). That quote above is an example.

I too respect your obvious interest and passion. But not the flawed premise.
 

BradM

Staff member
Admin
#28
Hello everyone! I hope you don't mind if I join a bit late.

I would say that the Cove is always up to a challenge, laughter, good jokes, recipes, and the like. So I do say I appreciate the OP presenting a challenge. However, my thoughts seem to be in line with many of the previous posts.

There is a classic scene from M.A.S.H., where it was time to take inventory of lunch trays (I think). A massive amount of them were missing. So several people worked together on this scheme. They would show the "auditor" a stack of trays, then they would take them away. They would pass them around the other side to be inventoried again. Just a masterful job of trickery. Give me fifteen minutes, and I will do the same.

My point is that I don't believe you will be able to find out if any system is effective in fifteen minutes, other than the vending machines. The good auditors I have seen work their tails off, do lots of planning, and study. They look at things once, sometimes twice. I am not suggesting you are not good or effective; just stating what I have observed.

I would even suggest as an auditor, the evidence phase would take longer than fifteen minutes. Too, the more deficiencies you observe and the deeper the issues, the more services you can offer.

Auditing is likened to random sampling. So is the premise that 15 minutes will be as effective regardless the size/ scope/ complexity of the organization?

And please... you don't have to consider yourself a grouch, or feel you have to be a grouch for anyone here to engage your thoughts. We do enjoy the pursuit of quality, and appreciate your passion also:). Just be straight with us, throw it out there, and let's run together!
 
C

cjssag

#29
The Grouch seems to have gotten lots of juices flowing here and provided lots of opportunities for exercise what with all the jumping to conclusions we witnessed. (Yes, I told you I am The Grouch.)

(BTW If I am the OP and our moderator is Andy, do we also have an Aunt Bee, a Gomer and a Barny?)

Was it the idea of the 15 minutes that many took issue with? How long should it take to determine the effectiveness of a QMS? Fifteen hours, fifteen days? Why does it have to take longer than 15 minutes?
Is it time that makes a good audit?

How many internal auditors are not given sufficient time to conduct their assignments? How many of this list's readers come from organizations where their auditors are "volunteered" and instructed not to let auditing interfere with their "real job" yet are expected to come up with a meaningful report? Should we not be using our voices and experience to help them do what they need to do in the time given to them?

Why shouldn't one possible measure of the effectiveness of internal audits be the development of a system whereby QMS effectiveness can be objectively assessed in 15 minutes? Why should it take longer than a thorough physical examination or an annual auto inspection?

The answer (IMNSHO) is all there in 8.4. The tools for it are readily available.

Shouldn't the ultimate measure of our own contributions to our organizations be based on how well we, as auditors, can structure ourselves out of a job? If inspection does not add value to a product, does auditing add value to a QMS? Whether one believes it does or doesn't, should we not all be working toward development of a qms that does not need to be audited?

Would that be a good thing or a bad thing?

Now we come to the real challenge. If you were to design a QMS that did not need to be audited, that freely yielded up all its secrets and told you what you wanted to know without having to ask, what might it look like?
 
Thread starter Similar threads Forum Replies Date
Q ISO 9001-2015 Internal audit finding Internal Auditing 12
lanley liao How to understand this words that the planning of internal audit shall take into consideration the results of previous audits? Oil and Gas Industry Standards and Regulations 10
A Add MDSAP to Internal Audit Schedule Medical Device Related Regulations 0
J IATF 16949 Internal Audit question - Auditor's responsibility Internal Auditing 6
S IATF 16949 Internal Audit Example IATF 16949 - Automotive Quality Systems Standard 7
R AS9100D internal audit checklist or ISO 9001 2015 to AS9100 D AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
M ISO 13485:2016 internal audit checklist Medical Device and FDA Regulations and Standards News 5
A Internal Audit Questions ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
salaheddine96 Internal audit planning Internal Auditing 2
M ISO 9001 Major Nonconformance Internal Audit Schedule/COVID-19 ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 18
E MDR internal audit Internal Auditing 1
U Internal Auditor not trained but done Audit for some process Nonconformance and Corrective Action 5
B Looking for 10 Internal Audit Online Training Participants ISO 17025 related Discussions 2
H AS9100 Checklist for Internal Audit needed AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 2
A What are the pros and cons of using an audit software for internal auditing? General Auditing Discussions 7
F Internal Audit before Pre-Assessment ISO 17025 related Discussions 2
Q Internal audit plan template Internal Auditing 12
L Internal audit during COVID-19 restrictions ISO 13485:2016 - Medical Device Quality Management Systems 5
O ISO13485 implementation - Are internal audits expected before stage 1 audit? Design and Development of Products and Processes 3
B Using Unreleased Documents & Process Maps for Internal Audit purposes ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
N Small Company - Internal audit process - Who does the audit? Internal Auditing 16
J Does anyone have an excel IATF 16949 Internal Audit checklist I could use? IATF 16949 - Automotive Quality Systems Standard 7
G Addressing Non-Conformances from an Internal Audit that are not product related ISO 13485:2016 - Medical Device Quality Management Systems 11
S Internal audit discrepancy - We missed a few audits that were scheduled Internal Auditing 12
Raffy ISO 14001 9.2.2 Internal Audit Programme Content Internal Auditing 10
N Internal Audit Schedule – Who gets to set the schedule? Internal Auditing 16
V IATF 16949 9.2.2.1 Internal Audit Program - "Process Changes" IATF 16949 - Automotive Quality Systems Standard 11
G Non Conformance During ISO 9001 Audit - Not All Internal Audits Completed General Auditing Discussions 19
B Using external FDA and ISO 13485 audit as internal audit Internal Auditing 6
T Internal Audit Schedule when Hiring Out Internal Auditing 7
D ISO 9001:2015 Internal Audit Training Advice Internal Auditing 10
M Internal audit consultant ISO 13485 (English speaker) Consultants and Consulting 3
S Implementing a 45001 Health & Safety standard - Internal audit plan wanted Internal Auditing 1
F Internal Audit - Procedure example Internal Auditing 5
C Internal Audit - Process Clause Matrix / Audit Checklist ISO 13485:2016 - Medical Device Quality Management Systems 7
CPhelan Internal audit - Combine similar nonconformities in one or keep separate? Internal Auditing 6
M Internal Audit Plan in Retail Internal Auditing 10
D Management of NC after internal system audit IATF 16949 - Automotive Quality Systems Standard 7
A Purchasing - Internal Audit Questions Internal Auditing 8
N Comprehensive Compliance Matrix for Internal Audit Checklist Other Medical Device Regulations World-Wide 1
W Where to begin with an ISO 9001:2015 internal audit Internal Auditing 13
D Internal audit forms or checklists for a medical/veterinary laboratory General Auditing Discussions 5
E Informational Internal Audits - Wear multiple hats what can and can't I audit (so I'm not auditing my own work) Internal Auditing 149
E ISO 9001:2015 - Internal Audit Plan Clauses General Auditing Discussions 8
S Internal Audit Checklist for Application/Software development IEC 27001 - Information Security Management Systems (ISMS) 1
S Internal Audit - Risk and Opportunity (ISO 9001:2015 ) Internal Auditing 1
F API Spec Q1 9th Edition Surveillance Audit - Questions about internal audits. Oil and Gas Industry Standards and Regulations 22
Ashland78 IATF 16949 Internal Audit Checklist Manufacturing and Related Processes 11
Ed Panek Root Cause CAPAs from internal audit ISO 13485:2016 - Medical Device Quality Management Systems 16
K Student wanting Case Study about Internal Audit Report Internal Auditing 1

Similar threads

Top Bottom