How to Increase Registrar Audit Rigor?

M

MIREGMGR

We originally started with NB "X", then concluded that their business model was too oriented toward giving us a certificate no matter what. So we asked some of our larger multinational customers who was tough, they recommended NB "Y", and we switched.

In another thread pertaining to ISO 9001 and other standards, Sidney Vianna commented regarding audit rigor:

I vehemently disagree with the position that audits are snapshots which can not be validated. That kind of mindset (shared by many people in the CB community) is one of the fundamental problems we have in the certification sector. Many insiders like to hide behind the disclaimer that audits are limited, sample-based, time-constrained exercises and the absence of written non-conformities does not mean a healthy management system, blah blah blah. Anyone trying to defend that positions leaves the door open for people to question the value of such audits, if no sensible conclusion can be reached, after reading an audit report.

CB audit results are supposed to be representative of the management system efficacy. If aerospace suppliers with chronic quality escape problems undergo AS9100 audits and are consistently given high scores, and no nonconformities reported, we have an OBVIOUS mismatch between audit results and reality.

Furthermore, CBs (if they are doing their job in a proper manner) should not have disconnected "snapshot in time" audits, but a program of continuous and consistent audits, where, each time, we learn more and more about the registered organization, and have a chance to delve deeper in the effectiveness of the system being assessed.

I apologize for this quote being out of context, but it addresses an important point for me.

We seem to have topped out in our NB's rigor capability within their normal business model, and they certainly haven't yet attained a systematic grasp of what we're doing. Our last two audits clearly were snapshots, with the auditor not impressing us with having attained a particularly deep grasp of what we do and where our structural shortcomings may be. I'm not confident that they're going to get there.

We're unusually complicated...we have something over 3000 medical-device products under our name and that we make on a private label business, and at least a thousand or two products that we make as a contract manufacturer. We also have non-medical activities. We've been in business for 30+ years, so many products have legacy design and manufacturing histories. Our QMS is also US FDA and HC focused, so a number of its methods and structural approaches are esoteric.

I can't do anything about our complexity level. The NB just has to deal with it.

We've talked to our NB about additional services, but they don't seem to be able to make their business structure respond to our non-standard inquiries. How would we get an NB to "delve deeper into the effectiveness of the system being assessed"? If we were to be interested in switching NBs again, how would we objectively identify one that could deal with us effectively?
 

QMMike

Involved In Discussions
Re: How to Increase the Audit Rigor?

First let me say that it is not often you read a post or an article written by someone who desires their 3rd party auditor to delve deeper into their systems/processes. That says a lot about you and your organization and should be applauded -

However, if your processes are as complex as it sounds (to me anyways) I don't think you will get the depth and satisfaction you desire from a third party registrar. Perhaps the best plan of attack is to save the money you would spend on contracting someone else to dig into your systems and hire a more robust internal auditing body. Who knows your systems and processes more than your own people.
 
T

tomvehoski

Re: How to Increase the Audit Rigor?

I don't think the third party registration scheme is the place you should be looking for this kind of feedback, although it is a great thing to be looking for. Why not find a consultant who is the best of the best in the field and hire them on a recurring basis? A consultant is required to give you advice and can participate on an ongoing basis. A registration auditor is barred from giving advice and consulting.

How about setting a goal to recruit some top talent from your competitors? That will bring another level of expertise and the "out of the box" thinking that you might need to jump start more improvements.

You could also look at pursuing the Malcolm Baldridge Award. I have not looked at that one myself in a long time.

The registration scheme is designed for, and limited to, showing compliance. It is not about sharing best practices, evaluating your performance against goals, etc.
 
M

MIREGMGR

Re: How to Increase the Audit Rigor?

Perhaps the best plan of attack is to save the money you would spend on contracting someone else to dig into your systems and hire a more robust internal auditing body. Who knows your systems and processes more than your own people.

At present all of our internal auditing is done by senior operational personnel, mostly hourly, who are trained as auditors in addition to their normal functions. Of course, they don't audit their own areas, but there's considerable crossover of awareness of formal vs. real procedures, problems due to customers, etc.
 

AndyN

Moved On
Re: How to Increase the Audit Rigor?

On the other hand, I do agree that is what a CB/NB should be doing! If any CB/NB auditor is failing to grasp a complex business operation and, as a result does superficial audits, I'd be asking if they possess the capabilities that I require of my supplier. And, as such, I'd either be asking to work with them to develop an approach staffed with the competent resources necessary to deliver, or I'd be searching for an organization who could do that for me!

The whole CB world needs to be challenged by clients or be challenging their clients to provide this type of in-depth assessments, rather than the same old vanilla compliance variety!
 
D

Duke Okes

I agree that you may be expecting more of your registrar than they are authorized to provide. There are many other ways to get a good systemic assessment, as others have mentioned.
 

Sidney Vianna

Post Responsibly
Leader
Admin
Re: How to Increase the Audit Rigor?

The registration scheme is designed for, and limited to, showing compliance. It is not about sharing best practices, evaluating your performance against goals, etc.
While there are constraints on what registrar auditors can do without creating a threat to their impartiality and stepping into a conflict of interest, evaluating a system against self-directed goals MUST be one of the BASIC things a CB auditor is doing.

There are a number of threads in this forum about auditors (internal and external) going beyond conformance and challenging processes and systems for effectiveness (and efficiency, for 1[sup]st[/sup] and 2[sup]nd[/sup] party audits). It is time for (serious) auditees to expect more from auditors. Proper auditing has never been and will never be limiting oneself to the 'say-what-you-do-do-what-you-say" check nonsense.
 

Sidney Vianna

Post Responsibly
Leader
Admin
How would we get an NB to "delve deeper into the effectiveness of the system being assessed"?
Have you tried providing unsolicited feedback to your NB/CB (let's not confuse Notified Body with Certification Body)? Have you told them explicitly that the assessors are not challenging your system for effectiveness? They, as suppliers of a service to you, must be interested in feedback and should react accordingly.

But, be careful with what you ask for. If your CB/NB believes that you want the thoroughest possible auditor, they might send you one of those auditorsauruses that will check if you have crossed all the t's and dotted all the i's (erroneously thinking that is "in-depth" auditing"). Make sure you communicate very clearly what your expectations are. And see how they will respond.

Unfortunately, most people believe that the least communication they have with their NB/CB and auditor, the better. But in order for that business relationship to be mutually beneficial, an open communication channel should exist.
 

AndyN

Moved On
Re: How to Increase the Audit Rigor?

The registration scheme is designed for, and limited to, showing compliance. It is not about ......... evaluating your performance against goals, etc.

Since this IS a requirement of ISO 9001 et al, it HAS to be part of a CB audit! Otherwise compliance with the standard has not been established.

Maybe you've had bad luck, but unless your CB auditors have been doing this, I'd suggest that you didn't get what you paid for! I'd also question why, if you've never demonstrated or been asked to demonstrate effectiveness, why you allowed the CB/auditor to do less than what's required...
 

ScottK

Not out of the crisis
Leader
Super Moderator
Is there a CB/NB that specializes in your particular business.

My last company makes pressure equipment. Our registrar/NB focused on that niche - Pressure Equipment Directive and gas transport. They are also a testing body. So they are intimate with the pressure relief and control devices the company makes and the rules governing them... as well as ISO9001, 13485, and MDD as it applies to hospital gasses.

And they were rigorous and challenging. Very much so.

Are you looking internationally?
 
Top Bottom