Do AS9100 Registrar Auditors have nonconformity quotas?

[Mike S.]

During our recent AS9100 audit, the registrar auditor said that soon auditors will have a sort of "quota" or minimum number of NC's that they have to write during audits or else they will be warned and put on some sort of probation, and eventually fired if they don't meet the requirements. I can't recall for sure, but I think this was supposedly coming not from the regsitrar, but from higher up the food chain.

Has anyone else heard of this?

 

[jmech]

This may have been incorporated into AS9104/3A, which was published 2020-11-11. I don't have a copy of the standard, but a slide from a September 2019 IAQG presentation on changes to AS9104/3 includes "Nonconformities per audit day" as part of new "Auditor Performance Monitoring" requirements:


Presentation:
https://iaqg.org/wp-content/uploads/2019/10/9104-3-Ballot-Draft-DSM-September-2019_Rev-2.pdf

 

[Sidney Vianna]

but a slide from a September 2019 IAQG presentation on changes to AS9104/3 includes "Nonconformities per audit day" as part of new "Auditor Performance Monitoring" requirements:

Presentation:
https://iaqg.org/wp-content/uploads/2019/10/9104-3-Ballot-Draft-DSM-September-2019_Rev-2.pdf

Great find.

I also note the following slide from the same presentation:

​

In the IAQG world, the OEM's are under continual "pressure" from the regulatory authorities (FAA, EASA, etc...) to demonstrate that the ICOP certification is robust. Too many cases of serious quality scapes and, when investigated, data shows that supplier certified to AS9100 for 10+ years and CB audit reports in all of these years show ZERO nonconformities.

While non-conformity quota/auditor is an asinine idea, monitoring of NC's/auditor over time is not.

 

[Mike S.]

I admit that 10 years of AS9100 audits showing 0 nonconformities would seem quite unusual.

It would be interesting to see the data on the number of NC's written in different manufacturing sectors, different types of organizations, witnessed vs. un-witnessed audits, etc.

As a client, I can see auditors pushing, pushing to meet their "quota" and perhaps sometimes even creating NC's that don't really exist (there is too much of that occurring already). I can also see clients thinking about leaving a few low-hanging fruits (minors) for the auditor to find so he/she can meet their quota and relax a bit.

 

[jmech]

While non-conformity quota/auditor is an asinine idea, monitoring of NC's/auditor over time is not.

My initial instinct is to agree, but I have a little concern about the monitoring. Either it is done subjectively (auditors are asked for an explanation if the CB thinks their NCR numbers "look fishy" based on their gut feeling [unwritten and inconsistent criteria]) or it is done objectively (based on written criteria that looks a lot like quotas). I think some monitoring likely needs to be done, but I don't know how you do it in a way that won't ever encourage auditors to write NCRs for the sake of meeting the monitor's (implicit or explicit) expectations. Maybe you make the criteria loose enough that an auditor will only get in trouble if they have many audits with NCR numbers that seem completely implausible - this catches the most incompetent auditors, but won't affect the vast majority and won't catch the occasional bad audit. I don't think there is a perfect solution here - maybe this, combined with thorough review of audit reports, is the best one available.

As a client, I can see auditors pushing, pushing to meet their "quota" and perhaps sometimes even creating NC's that don't really exist (there is too much of that occurring already). I can also see clients thinking about leaving a few low-hanging fruits (minors) for the auditor to find so he/she can meet their quota and relax a bit.

These are the obvious problems that quotas (or monitoring with an implied but unwritten quota) will cause.


We'd heard rumors that our (non-AS9100) CB set auditor quotas or KPI expectations for NCRs and sometimes got the feeling during audits that the auditor might have a quota. When asked, our CB very strongly denied that they set any auditor quotas or KPI expectations for NCRs.

 

[Sidney Vianna]

As a client, I can see auditors pushing, pushing to meet their "quota" and perhaps sometimes even creating NC's that don't really exist (there is too much of that occurring already). I can also see clients thinking about leaving a few low-hanging fruits (minors) for the auditor to find so he/she can meet their quota and relax a bit.

Yep. I can also see CB auditors “replicating” unresolved internally reported nonconformities. Since the organization should be doing corrective actions anyway, “cut & paste” NC’s could be a thing for CB auditors.

 

[Randy]

Yep. I can also see CB auditors “replicating” unresolved internally reported nonconformities. Since the organization should be doing corrective actions anyway, “cut & paste” NC’s could be a thing for CB auditors.

That quota garbage is a disaster waiting in the wings. Quota's remove objectivity and impartiality from the process and anyone that's says different is taking in too much CO2. Quota's will predispose auditors to document bogus, worthless krap that could have the potential to do more harm than good.....And I can't think of any good.

Replicating already identified NC's that are already being corrected is another krap job, I've seen it done and when I'm a Lead I don't allow it.

The whole thought of quota's or minimums strikes me the same as police officers, traffic tickets and small town politics (been there done that and I refused to do it)

 

[jmech]

That quota garbage is a disaster waiting in the wings. Quota's remove objectivity and impartiality from the process and anyone that's says different is taking in too much CO2. Quota's will predispose auditors to document bogus, worthless krap that could have the potential to do more harm than good.....And I can't think of any good.

I agree. I think the only argument for quotas is that they encourage/force an auditor to write an NCR for a legitimate issue that the auditor would otherwise ignore. However, if the auditor would ignore a legitimate issue, you've already got a serious problem with auditor competency/integrity, and quotas don't address the root cause of this problem while also causing new problems.

 

[Mike S.]

Who, and how, will they determine the number of NC's that "should" be written?

I wonder if they will use a control chart to plot the number of NC's by auditor to see if any special causes are at work? Nah, probably something way more subjective than that....

 

[pilchard]

They all have had quotas for years, nature of the beast.