Calculate FPY for Audit Results

[TacitBlue]

How do you calculate first pass yield for internal audits vs. CA?

 

[Jim Wynne]

Please tell us more about what you're trying to learn. Why do you want to do this? What does "internal audits vs. CA" mean?

 

[TacitBlue]

Trying to establish kpi for our compliance dept.

 

[Jim Wynne]

Trying to establish kpi for our compliance dept.

I asked two questions. Let me try again: What do you want to know about audit results? Why have you chosen FPY?

 

[TacitBlue]

Thanks Jim. I would like to know that we are seeing less and less CA findings over time during audits. I’m other words, we are having more successful audits than findings in each quarter.

 

[Jim Wynne]

Thanks Jim. I would like to know that we are seeing less and less CA findings over time during audits. I’m other words, we are having more successful audits than findings in each quarter.

This raises a lot more questions.

• Why would an audit that discovers nonconformities not be considered successful? Wouldn't it result in something being improved?
• What is your role in this? Are you a part of the compliance department? If not, why don't they (and the management people above them) determine their own KPIs?
• If you want to track the number of audit NCs, why don't you just do that? FPY won't help you.
• Is the number of NCs being generated now considered excessive?

Consider these to be rhetorical questions. I could go on, but I'll just say that I think your idea here is misguided. Find out what "success" means, why you're not achieving it, and whether achieving it is realistic and useful.

 

[Funboi]

Trying to establish kpi for our compliance dept.

Why?

 

[Bev D]

In addition to Jim’s great input I will add that FPY is mathematically non sensical for audits.
FPY is intended for ‘large’ volumes of things going through multiple pass/fail operations or inspections. Audits do not fit this.

KPIs for auditing are difficult. KPIs require you to “mathematize” outcomes and auditing/compliance is not easy to put into a mathematical algorithm. It’s more of a human logic thing you know when the system is good and when it’s not. There are so many intangibles. As Deming said some things that can be counted don’t matter and some that matter can’t be counted. your desire to reduce something to a number doesn’t make it possible or advisable.

I commend you for coming here and asking really good questions. There is so much misinformation and ‘rote’ applications of popularized approaches that are really just snake oil. Don’t accept anything just because ‘they’ said so. Seek first to understand.

 

[matthewjd24]

I think I understand where you're coming from. I'm not nearly as experienced as the other people in this thread, but let me give you my two cents. It appears you (or your org) views QA as being responsible for ensuring the rules are being followed, and that a successful audit is one where everyone audited was found to be following the rules. I wouldn't agree with that approach.

If you compare QA to the police, you know that the police are responsible for enforcing the rules. But you wouldn't hold the police responsible if someone breaks the law. So measuring police performance by expecting less and less arrests over time doesn't make much sense. Really, an effective police department is one that catches a lot of criminals, issues a lot of tickets, etc. So lots of these indicates a well performing police department.

I think that is the thinking that has guided the responses in this thread. I wouldn't view an audit with findings as unsuccessful, just the opposite. Measuring QA performance in the way you've described is like expecting the police to make fewer and fewer arrests over time. It doesn't make much sense. It should be the personnel and their supervisors that are held accountable when audit finds they are not following procedure, and QA should be patted on the back for discovering it. Often times, it sparks a conversation around the procedure- whether it's practical or needs changing.

 

[Funboi]

Thanks Jim. I would like to know that we are seeing less and less CA findings over time during audits. I’m other words, we are having more successful audits than findings in each quarter.

You seem to be confusing terms here. What’s the purpose of doing internal audits? According to ISO 9001, it’s to provide information about the QMS. That information would be based on the findings of the auditor(s). Do you equate “findings“ with bad results?