Management & Internal Audits (Oil & Water?) Audits as a pass/fail test

[RCW]

I'm seeking some opinions here (I already formed mine and I hope the way my question is worded doesn't taint the opinions too much).

Question: Can Internal Audits be sucessful and/or useful if Management treats all audits (including internal) as a pass/fail test?

 

[Aaron Lupo]

Re: Management & Internal Audits (Oil & Water?)

RCW said:

I'm seeking some opinions here (I already formed mine and I hope the way my question is worded doesn't taint the opinions too much).

Question: Can Internal Audits be sucessful and/or useful if Management treats all audits (including internal) as a pass/fail test?

Question- Why does Management want audits to be pass/fail?

No matter. The audits can be a useful tool no matter how management looks at them pass/fail, scoring etc... What really matters is that they are looking at the results of those audits and making informed decisions on where to focus resources to improve the company. JMHO

 

[Claes Gefvenberg]

I agree with Johnny Bravo: I cannot see a reason for a fail/pass mentality? Hmmmm... Oil and water? Well, oil floats on water...

The audits can still can be worth something of course, but successful? Surely nowhere near as successful as if they are regarded as conveyors of improvement potential. Still, as long as appropriate action is taken...

Now then: Jim Wade recently asked a crucial question: how do you measure the degree of success of an internal audit?
Have a look at: "Measuring the results of ISO 9001 internal audits"

/Claes

 

[Randy Stewart]

Yes they Can

All I do is present an "Audit Summary" to the Exec Mgmt. Where the meat and potatoes (or beer & pizza) is at for IA's is working with the process owners for improvement. As internal auditors you can do consulting because think about it. Lets say process 1 is suppose to give me results "A" and it gives me "a", I've identified a nonconformance. Anything else is basically problem solving or consulting! Theres your pass/fail.
Now, as internal I can sit down with the process owners and look at how the failure came about. Better yet, I can help fix it! Do I care about the pass/fail criteria - NO. What I report is that 1 NC (or OFI) was found on/in process 1 and here is the Corrective Action.
I don't think it's a bad thing to be harder on yourself during internal audits, it protects you from failing externals. And there are more benefits in getting auditors involved in problem solving. They learn the processes better, they are viewed as resources instead of pains in the arses and they learn Root Cause Analysis.
Don't let Executive Management decide if the event was successful or not. The way I look at it is that if I can find something wrong and fix it, that's a success. If someone else points out the problem, I haven't done my job properly (in most cases).

 

[Al Dyer]

I believe that the pass/fail methodology is the best way to go.

A company is either doing what they say or not. All the "gray" areas of the system without P/F can only serve to undermine the integrity of the auditor and the continuity of the audit process. How many times can an internal auditor "overlook" a nonconformance before all respect is lost, for the auditor as well as the process!

That said, if there is a robust system of preventive action, continuous improvement, and corrective action the P/F works fine. The company needs to define the process for directing a nonconformance into one of the above 3 catagories. The more that are directed to continuous improvement the more an outside auditor will see a companies proactive approach to the use of internal audit.

Al...

 

[Claes Gefvenberg]

I wonder if we're seeing the same meaning in the expression "pass/fail"?

RCW, I sense that you have a problem here? Could you give us a bit more background info?

/Claes

 

[Tom Harris]

Originally posted by Al Dyer
I believe that the pass/fail methodology is the best way to go.

A company is either doing what they say or not.

Maybe that's a bit simplistic, Al?

I'd like to make two points:

1 Surely a company sometimes [often?] does what it says it will do only to some extent?

Examples: We say we put the component on the right way round, but from time to time we put it on the wrong way round. We say that our employees are our greatest assset but occasionally a rogue manager treats someone like shít. We say we adopt a PDCA philosophy but the second shift sometimes simply fixes the problem, again and again.

A pass/fail attitude to audits has its place, but only in connection with getting a certificate. For subsequent internal audits, it ain't helpful (except for that annoying new bit about conformity to the standard). IMHO.

2 The accent on "doing what they say or not" is a hangover from the bad old days: "say what u do, do what u say and prove it" those old-timers use to trot out, didn't they?

The more enlightened approach recognises that simply following the rules is sometimes [often?] more part of the problem than the solution. Hence, even ISO 9000 drops the 'document everything' requirements in favour of the 'improvement of a system of processes' approach. The internal audit must also adopt that approach. IMHO.

 

[M Greenaway]

Tom

Quite true to an extent, however auditing compliance to the standard is very useful because it will force you to ask the right questions of a process.

Also have you noticed that nowhere in ISO9001:2000 is there a requirement to work to your documented procedures, hence auditing for compliance to the standard will not drag you down the wasteful auditing for compliance to procedures.

As you (and a fella called Deming) say most problems are inherent in the system (or procedures if you wish), hence continued compliance ensures continued problems (or level of performance).

By auditing to the requirements of the standard we should not be blinkered by procedures, and should as important compliance questions like is there demonstrable continuous improvement, or are there objectives for this process, or is this process performance adequately measured, etc, etc, and all that other good ISO9001 compliance stuff.

 

[Randy Stewart]

Not Sure

Martin are you talking internal or external here?
If it's internal audits then I disagree. External, yes but internal audits should go deeper than what the standard requires.
You state:

auditing compliance to the standard is very useful

and

most problems are inherent in the system

Isn't the standard a "suggested" system outline?
If you are not "blinkered" by what my process maps or procedures are showing then how in the world are you going to know what we are trying to accomplish?? By knowing the standard? I think not.
How can an external auditor come in a tell me if my process is being adequately measured when they don't know what we are measuring. That calls for a lot of auditor personality, interpretation and background to come into play. Which is one reason why QS has failed so miserably.
Here's an example. ISO 14001 under 4.4.5 states the "The organization shall establish and maintain procedure for controlling all documents required by this standard. I had an auditor interpret this as every single piece of paper (MSDS sheets, etc.) that was referenced by our Environmental system, down to the crib requests for mineral spirits, was to be under doc control! NO F'ing WAY!!!!!!!!!! But that was being audited to their interpretation of the standard and not what our procedures, processes and Ops manual stated.
Maybe I'm taking what you said to literally here. But IMO auditing to only the standard is like doing only the maintenance called out by your cars owners manual.

 

[David Hartman]

I must agree with M. Greenaway.

I just finished our first internal audit and performed it as a compliance to the standard audit.

The first questions I asked dealt with the presence, general understanding, and attainment of the quality policy and quality objectives. Along with verifying that customer satisfaction was another of the driving forces within the overall organization.

I then verified that these "driving force" issues were understood and witnessed how they impacted the processes of each individual organization. This was followed by witnessing how each individual organization monitored or measured the effectiveness of their processes, and ensuring that they were taking actions to improve those processes (corrective and/or preventive).

Verifying that senior management was monitoring the health of the QMS, and was participating in establishing priorities and removing roadblocks to improvement, completed my audit efforts.

I feel like this activity not only assures me of compliance, but also provides evidence that the organization is on the road to customer satisfaction and continuous improvement

I had little interest in documented procedures (although I did ensure that processes and their interactions had been defined), and I did ask for defined competencies and how they ensured that the individuals filling a position met (and continued to meet) those competencies (including an examination of training records).

I believe that the new standard is written in such a fashion that not only can we develop our QMS to ensure accomplishment of company objectives, customer satisfaction and continuous improvement; but it can be audited against for compliance to verify that we are indeed accomplishing those goals.