Definition Performance Based Auditing - What is the definition?

D

dashmen

Hi everyone. It's my first post . I visit this forum almost every day for half a year and I think the forum is thoughtful and a great place to understand "quality". :read:
Now my question is, what's your understand of the "performance based auditing"?
 
W

WALLACE

Welcome Dashmen,
You'll find lots of invaluable resources at the Cove.
Dashmen, what is your definition of a performance based audit?
Your answer shall allow the Covers to get a baseline on your understandings and interpretations.
Wallace.
 
D

dashmen

Jennifer, I am studying the article. Thanks for your links.

Wallace, the situation is, one customer ask us have an audit on one of their suppliers, they give the checklist and ask that the audit must be a "performance based audit". I reviewed the checklist and feel that seems like the "process audit" . So I want some opinion and can ready for the audit.
 

Jen Kirley

Quality and Auditing Expert
Leader
Admin
dashmen said:
Jennifer, I am studying the article. Thanks for your links.

Wallace, the situation is, one customer ask us have an audit on one of their suppliers, they give the checklist and ask that the audit must be a "performance based audit". I reviewed the checklist and feel that seems like the "process audit" . So I want some opinion and can ready for the audit.
This is as I suspected.

One of the criticisms I have read of current registration standards (both ISO and the specialty standards that resemble its structure) is that audits of systems per these standards are based on evidence that the system is being followed. How effective and with what value are not questions for the auditor--the auditee is only responsible to show the system is functioning within specified parameters.

This has invited soul searching after problems such as the Ford-Bridgestone debacle, where tires blew on Explorer vehicles under certain condisions and people asked, "Why wasn't this caught before people died from it? Why didn't the audit process catch this problem?"

The basic answer is that registrars donot have the right to demand that performance data be included. Test results, design changes that result, etc. are considered sensitive, confidential data and can be kept secret.

Industry regulators have been the standard answer for this need for oversight, but they may be too few, thus cannot provide the kind of coverage necessary for effective oversight.

Those who favor less government intervention assert that free market forces can fill the gap; let flawed suppliers be exposed by their customers, and either be forced to change or suffer from shrinking market share. In other words, let the customer's demand for quality rule over practices, not just enforcement of law.

The performance audit seems to be an attempt to meet this objective.
However, you still wouldn't have the powers of a regulatory auditor. The majority of references I found, including the second link, involves government audits. Even when done internally, government entities would likely find it much easier to exert authority in the audit process than the free market auditor could expect.

If you feel the checklist does not address performance, you may want to negotiate that because it would still be very easy to hide poor performance behind graphs and charts if you are not allowed to dig for data that supports their claims of effectiveness. Your customer may or may not know all this, and the supplier may also understand it, or not.

It may be up to you to decide how approach drawing the parameters of this audit, negotiate the extent you will be held responsible for its results, and ensure the auditee will cooperate by supplying the data that shows performance, and not just that they are following the system.

Anyone else? I am ready to hear if I am on the mark or not, or if I am missing something.
 
D

D.Scott

Jennifer - I'm not sure where you are coming from when you say auditors can't verify the effectiveness of the process.

ISO 19011:2002 (applicable to all organizations needing to conduct internal or external audits of quality and/or environmental management systems) says in the introduction: "verifying the effective implementation of an organization's quality and/or environmental policy".

16949 states: "..... and maintain a quality management system and continually improve its effectiveness ....". How can an auditor determine compliance to this requirement if they are not allowed to verify that effectiveness?

Am I missing something or were you meaning something else?

Dave
 
W

WALLACE

Dashmen,
The attached visual may be of some help in clarifying audit focus and process approach regarding effectiveness of system.
Wallace.
 

Attachments

  • Process audit.zip
    327.5 KB · Views: 784
  • Process audit.pdf
    387.2 KB · Views: 966

Jen Kirley

Quality and Auditing Expert
Leader
Admin
D.Scott said:
Jennifer - I'm not sure where you are coming from when you say auditors can't verify the effectiveness of the process.

ISO 19011:2002 (applicable to all organizations needing to conduct internal or external audits of quality and/or environmental management systems) says in the introduction: "verifying the effective implementation of an organization's quality and/or environmental policy".

16949 states: "..... and maintain a quality management system and continually improve its effectiveness ....". How can an auditor determine compliance to this requirement if they are not allowed to verify that effectiveness?

Am I missing something or were you meaning something else?

Dave

My understanding is that in performance audits, the auditor can look at performance data and determine the system's effectiveness.

The complaints I have read are that auditors normally may not be able to see actual data like regulatory and accounting auditors do. Such data can be claimed off-limits due to competetive sensitivity. This has been the excuse of why the Bridgestone/Ford tire problems went on as long as they did.

I suspect it is why there is such lingering cynicism regarding ISO...is the standard really doing all it can to ensure quality? Many have argued it is not.

It has, according to the complaints, required auditors to be satisfied with the auditees' own assessments, including testing, internal audits, customer feedback and so on. Activities based on results of those assessments, including CARs, continuous improvement and design changes can be promoted as "proof" that the system is effective. But these show the system is functioning, not its effectiveness or value or performance (as I use the term "performance", that is). It does not "prove" quality, just raises the comfort levels.

The bulk of the sources I found regarded government entities. I am not sure how much the performance audit is being used in private or publicly owned companies.

Now I could be wrong about all this, but I base my comments more on long-term reading than my own, frankly more limited exposure to the practice. If I am wrong, I'd like to find out because I am here to learn too!
 
H

haifeng

Jennifer Kirley said:
My understanding is that in performance audits, the auditor can look at performance data and determine the system's effectiveness.

The complaints I have read are that auditors normally may not be able to see actual data like regulatory and accounting auditors do. Such data can be claimed off-limits due to competetive sensitivity. This has been the excuse of why the Bridgestone/Ford tire problems went on as long as they did.

I suspect it is why there is such lingering cynicism regarding ISO...is the standard really doing all it can to ensure quality? Many have argued it is not.

It has, according to the complaints, required auditors to be satisfied with the auditees' own assessments, including testing, internal audits, customer feedback and so on. Activities based on results of those assessments, including CARs, continuous improvement and design changes can be promoted as "proof" that the system is effective. But these show the system is functioning, not its effectiveness or value or performance (as I use the term "performance", that is). It does not "prove" quality, just raises the comfort levels.

The bulk of the sources I found regarded government entities. I am not sure how much the performance audit is being used in private or publicly owned companies.

Now I could be wrong about all this, but I base my comments more on long-term reading than my own, frankly more limited exposure to the practice. If I am wrong, I'd like to find out because I am here to learn too!

I agree with ur idea. That's situation is very popular in China. Most of the auditor conduct the audit according to the standard requirements, although according to 2000 EDITION, the audit should be conducted along the process of the auditee, but the actural performance datas are very difficult to confirm and verify. So, I think, in the future, a kind of combined audit will appear that's combine with the accounting audit and the quality system audit and other related audits. It's called by someone "the internal auditor."
 
D

dashmen

Thanks all your replies. :applause:

I am thinking this issue: Does effective process=>effective outcome? or Does effective outcome=>effective process?
And how we define performance within an quality audit? Does it mean product performance or system performance or process performance? Maybe all of them?

This article (https://www.nalga.org/qrtly/art97d1.html ,thanks for Jennifer's link,I dig into it) gave me some inspiration,especially the last sentence:"the procedures (conversion activities) are only the piece of the equation that facilitates the achievement of management's goals."

More opinion are welcome.
 
Top Bottom