International Auditor Calibration

T

tyker

#1
I've been volunteered, despite protests, to lead my employer to registration to EN AS 9100 within the next year or so. We have plants in the UK, Ireland, France, USA and Mexico and all will use a common management system by the time I've finished with them. Our manufacturing, and other processes, are identical across the globe.

I've had bad experiences previously with inconsistent approaches from auditors in different countries working for the same CB and want to avoid that frustration this time.

We'll shortly be going down the route of choosing a CB and I'm curious to know what is currently considered best practice amongst that community for ensuring that their auditors in different countries/continents take the same line. I'm only interested in using a reputable CB with international capability and those with regular contributors to Elsmar such as BSI, DNV and NQA will be included on the shortlist whether they like it or not.

I don't want different interpretations of the standard or for situations to be accepted in one plant and criticised in another. I don't want weak auditing in one country and nit picking in another. I want the same modus operandi.

So CBs, how do you ensure I'll get the same level of auditing across international boundaries?
 
Elsmar Forum Sponsor

Sidney Vianna

Post Responsibly
Staff member
Admin
#2
Tyker, while your expectations are fair, I doubt you will get a lot of concrete answers to your central question. Even large CB's are structured in ways that don't make for easy "calibration" of auditors working for different geographical groups.

At best, large Regions or countries do some training of their respective auditors, but I doubt any major CB could prove to you that they do that on a global level.

For the AS/EN Scheme, obviously all auditors must be approved in the OASIS database. To that effect, all of them must have attended the IAQG-endorsed AATT course, which, to a level must have "calibrated" the auditors about the 9101D audit process. As for "calibration" of interpretations and depth of auditing, nobody is there yet.

In our case, we have tackled some of these challenges with the provision of Project Leads, people who were responsible for being audit team leaders in most (if not all) the audits. Having a few "calibrated" individuals acting as Team Leaders on site, minimize the variation of findings (NC's, OFI's, observations, etc.) Even though that might require flying auditors between continents, it might be a wise investment; at least early on in the process.

Another tool we use is the development of Project Brief Documents in which certain agreements (administrative, technical, interpretative, etc.) are captured and agreed upon with the registrant. Then, the Project Brief is forcefully communicated to all auditors and support staff so they are aware of the contents of the document. The assignment of an account manager and a single point of contact at the registrant are also key in identifying variation in the service delivery, so it can be acted upon.

Cheers.
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#3
What I did in the past when working with several *big* companies (large multi-nationals) was have the registrar assign one person as a "lead auditor" for the entire company. That person had to be at every audit in every facility on every continent. The other auditors in the group could be local. The typical size of the group of auditors was 4 to 6 people (which included the Lead Auditor), and audits were from 4 to 8 days depending upon the facility size. I went through that several times years ago.

The key in these cases was they were large multi-nationals so they could place some "demands" upon their registrar.

What you could do is send a representative from your company as a "Guide" to follow along on every audit.

The problem was they were allowing simultaneous audits at different locations (Singapore, Ireland, etc.) and we ended up with "interpretation" issues. When we got the reports/findings flowing into our "central war room" was in part what you fear - Extremely inconsistent nonconformances which were not due to procedures. That is, a facility in Singapore would have the exact same process in an Ireland facility (or a US facility or wherever) and the different auditors had different interpretations and "understandings". In one example it was calibration of an instrument in a process where one place got a nonconformance and the other place didn't even though both were audited. In one case the auditor insisted that a specific process indicator was critical and thus had to be calibrated, while in another the auditor listened to the reason the facility didn't calibrate that process instrument and agreed it didn't need to be. It was starting to get ridiculous.

NOTE: In the case mentioned part of the problem was different locations had different implementation consultants. I remember two plants decided *all* process instruments had to be calibrated while the other plants had looked at each process and defined which instruments were critical and why (so they had at ready their process evaluation sheets where they had made their decision and why).

In the case mentioned we provided data showing that there was no evidence of internal nonconformities at that process and we also provided data showing that there were no customer returns, warranty replacements/ returns/repairs or other evidence that the process in question had failed. *Data is your Friend!*, not to mention being prepared, as in this case, to back up your reasoning for not having a process indicator calibrated (a process instrument determined to be "non-critical" to the process).

The aftermath was there was better coordination between plants on a number of issues. In this case the 2 plants which were calibrating instruments that were not critical to the process had to re-evaluate their "local" calibration requirements (which saved some significant $$$ in unnecessary calibration costs) and some of their processes.

I doubt this would be a problem with ISO 9001 these days, but AS9100 is a bit more "subjective" in some areas.

But - A lot of time has passed since I was involved with multi-nationals and my assumption is things have improved. The 1990's was sort of a "wild west" in some senses in so far as interpretations of the requirements go.

As Sidney says, the assignment of an account manager and a single point of contact at the company being audited are key aspects. You need to be able to narrow it down to a 1 to 1 sometimes to deal with discrepancies.

If it was me - I'd assign a person from your company to tag along on every audit. I know travel is expensive, but after it is all over that person would be in a position to give your company's upper management an evaluation of all the different plants. My experience has been that even though the same process(es) is (are) employed at different locations (same procedures, same work instructions, etc.) each plant/location has it's own idiosyncrasies (for lack of a better word off hand). It would be an excellent opportunity for one person to see each facility. I'd use it as an opportunity for improvement, so to speak.

Just some thoughts.

NOTE: As evidence that interpretation issues *still* come up these days even with ISO 9001 requirements, read: Is a Quality Representative (Management Representative) Designee Required?

If you want to relive the Wild West days, see: Word On The Street Is.....

If you'd like a quick laugh, see one of my old QS-9000 rants.
 
Last edited:
M

Markaich

#4
Auditor consistancy...the Holy Grail of the Quality Profession...

In a smaller environment (it may be scalable) as Marc says, we used to insist that the Lead Assessor for the company led each visit to the five sites within the company to try and get that common approach. We also insist that of CB's auditors are all appropriately registered (IRCA or IEMA in our case). This still doesn't guarantee consistancy.

As Sydney says, it is unlikely that any global CB will have a common auditor training and devlopment programme throughout the world...sitting on our CB's 'Governing Body', this is a conversation we have had a couple of time, with no resolution.

As to 'exactly the same processes', this is very unlikely irrespective of what the documentation says. As I said, we had five sites all within the UK apparently complying with the same management system. Unfortunately, while each site was essential completing the same work, the local interpretaion of the company system was quite different.

We are now smaller, only two sites, but the problems with interpretation are similar (although not as extreme)...it is easier to maintain consistancy as the audit team for both sites is the same, but even then, if they are having a bad day...

I suspect there is no easy answer to the problem; that's if there is an answer at all.

I suggest all you can really do is take a proactive approach, taking up inconsistancies with audit outcomes with the CB and inconsistancies with operations with the local management. In this way you can help to reduce the variations across the patch.

M
 
P

Peter West

#5
Hi Tyker

We are facing exactly the same issue across our offices in Europe, Middle East, Asia and Australia. Its looking like we are going to pursue separate certificates for each region (so as not to endanger long-standing certification in the UK and China) but use the same body to carry it out, and request the same auditor. We are not particularly large (maximum of 300 people in our London office), so do not envisage too much difficulty in getting by with just one auditor (who knows our business). This will ensure a uniform approach to auditing across the business regardless of region.

The only concern is cost, but as we would probably have to incur hotel costs for the audits anyway, its only an extra £2k to £4k a year on flights, which split over 4 regional cost centres is a negligible cost.

Good luck in pushing a uniform approach to operations. I have had to re-structure our management systems so it is uniform in policy and mandatory procedures only. SOPs differ slightly (in content) subject to the regions and business regulations there.
 
B

bazboy

#6
My previous employer was registered by BSI. They assigned the same auditor for all there locations.


Barry Quinlan
 

John Broomfield

Staff member
Super Moderator
#7
I've been volunteered, despite protests, to lead my employer to registration to EN AS 9100 within the next year or so. We have plants in the UK, Ireland, France, USA and Mexico and all will use a common management system by the time I've finished with them. Our manufacturing, and other processes, are identical across the globe.

I've had bad experiences previously with inconsistent approaches from auditors in different countries working for the same CB and want to avoid that frustration this time.

We'll shortly be going down the route of choosing a CB and I'm curious to know what is currently considered best practice amongst that community for ensuring that their auditors in different countries/continents take the same line. I'm only interested in using a reputable CB with international capability and those with regular contributors to Elsmar such as BSI, DNV and NQA will be included on the shortlist whether they like it or not.

I don't want different interpretations of the standard or for situations to be accepted in one plant and criticised in another. I don't want weak auditing in one country and nit picking in another. I want the same modus operandi.

So CBs, how do you ensure I'll get the same level of auditing across international boundaries?
tyker,

Make consistency of AS9100 (and AS9101D) interpretation with respect to your global management system one of the criteria for selecting or reselecting your registrar.

You cannot solve this problem for them.

Focus instead on what you can do:

  1. Use the reselection criterion as specified by you above.
  2. Brief your counterparts not to accept recommendations (sometime disguised as OFIs) for more rigor in deciding if the evidence indicates conformity or nonconformity.
  3. Brief your leaders not to celebrate the absence of nonconformity but to welcome well-crafted nonconformity statements that add value.
  4. Measure the value of nonconformity statements in terms of the price of each nonconformity (annualized cost - see Phil Crosby) as reported by employees, supervisors, managers, leaders and auditors.
I wish you well in improving the performance of one of your suppliers: your accredited registrar.

John
 
T

tomvehoski

#8
Until robots and computers can perform audits, you will have inconsistency. Each individual has their own experience, motivation, pet peeves, opinions, etc. You can't train human nature out of people completly.

Paying to have the same auditors travel as mentioned is one solution. There is still the possiblity of the same auditor changing over time, so the intrepretation on Audit 1 could be different by the time they get to Audit 3.

The registration bodies are going to tell you what you want to hear during the quoting process. Of course all of their auditors are trained 100% the same and have the same intrepretations. You can try getting references of other multi-national companies they audit and check references, but still not an exact science.
 
A

Antyp

#9
Curently we are (company i work with) certified with one CB for all corporate facilities/companies/sites.
For last 4 years i saw a little evolution in this subject. First we had one lead auditor for all of sites. It was till like 2 years ago, while "new rules" for certification were issued, and auditors team had to be changed.
New solution is we called "mutual agreements", which contain all "specific" items and all corporate documents that must be same at all companies. Its signed both by CB and corporate quality manager together.
Aslo there is always someone from headquarter to "support" local audits and also to verify if auditors are well calibrated. After all planned audts are performed, there take place another meeting with CB, where results of auditors work is evaluated, and new "mutual agreements" are set for next year.

One more input companies inside corporation got separate certificates, other words its not a "corporate certification" but rules and focus areas are defined by headquarter and agreed by CB.
 
T

tyker

#10
Thanks to all of you for the constructive replies.

I'll just tell you why I'm concerned and then move on. My last job involved gaining TS16949 registration at sites in the UK, Poland and Turkey. Our CB allocated, at my request, a UK based lead aufditor with local team members. For the initial assessment this was fine, the problem started with the routine surveillance visits which were conducted by the locals. The UK guy carried on as before. The Polish auditor created havoc by raising nonsensical issues as major nonconformities (the subject of another thread a year or three back) and trying to get extra auditing days (he was a contractor). This was dealt with by some constructive phone calls between me and the head of schemes at the CB. The Turkish bloke refused to issue nonconformities to fellow Turks. Also frustrating and informally reported back to the CB. I couldn't complain formally because our management were delighted at the lack of findings and held the Turkish plant in high regard. This completely undermined the internal audit findings which suggested significant problems.

Anyway, I'm moving on from that and want a constructive relationship with the CB in my new place. I like Sidney's suggestion of a project manager within the CB who can at least review all the reports, be available for feedback from me and contribute to a balanced series of audits. The company will have a central contact for Europe (me) with my opposite number in the USA taking that role for North America. We'll be visiting each other at least once per year to audit each other's sites, help to internally calibrate processes and identify best practice. We're also working closely together standardizing process documentation to allow for flexibility of production throughout the group. Somehow, though, I've been landed with most of the work as I have the best systems knowledge/experience.

We're not the biggest operation. Although the company is spread out, that's to easily service our customers so, in total we only have maybe 300 people. I have no doubts about our ability to implement and maintain EN AS 9100. I don't want a CB that sends in auditors with ambitions to be consultants preaching best practice and telling me my job. I do want to have a fair, ethical assessment by people I can trust and bury, once and for all, my negative feelings towards 3rd party registration.

Again, thanks for all the replies.
 
Last edited by a moderator:
Thread starter Similar threads Forum Replies Date
Jerry Eldred Calibration Methods - Auditor wants federal or international calibration procedures? General Measurement Device and Calibration Topics 33
E International Railway Industry Standard (IRIS) Auditor Training Requirements Other ISO and International Standards and European Regulations 13
S What are the International Forging Requirements for Nitrogen Strengthened Stainless Steel? EU Medical Device Regulations 3
D ISO 9001 certificate issued by QMS International for 10 years - legit? Registrars and Notified Bodies 17
J Leveraging FDA 510k Clearance for International Registrations Other Medical Device Regulations World-Wide 2
Sidney Vianna Informational IAF and ILAC Seek Contractor for Establishment of a Single International Organization for Accreditation ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 2
M Class I - International Registrations EU Medical Device Regulations 3
E Packaging Specifications Mercedes-Benz U.S. International IATF 16949 - Automotive Quality Systems Standard 4
F Mig Welded Components - IMDS International Material Data System RoHS, REACH, ELV, IMDS and Restricted Substances 1
M Informational Publication of Draft Health Canada Implementation Guidance for the International Medical Device Regulators Forum Table of Contents Format Medical Device and FDA Regulations and Standards News 0
M Informational RIVM – Summary International Expert Meeting on breast implant-associated lymphoma Medical Device and FDA Regulations and Standards News 0
Sidney Vianna Medical Device News International exposé of the harm caused by poorly tested medical devices. Other Medical Device Regulations World-Wide 12
Wes Bucey International Manufacturing Technology Show - 10 September 2018 - Chicago (meet up?) Coffee Break and Water Cooler Discussions 6
Sidney Vianna Optimizing Standards for Regulatory Use - International Medical Dev. Regulators Forum Other Medical Device Regulations World-Wide 0
qualprod Which standard to follow, local or international? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
S Referencing International Standards as Design Input Design and Development of Products and Processes 5
N International / Traceable Standards - Procedures for Calipers, Micrometers, Etc. ISO 17025 related Discussions 2
B ISO 17100 (Requirements for Translation Services) International Translations Other ISO and International Standards and European Regulations 4
S Where is a good place to get IMDS (International Material Data System) Training? IATF 16949 - Automotive Quality Systems Standard 3
Marc IAF (international accreditation forum) - Mandatory Documents (MD Series) ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 0
Marc "Elsmar 90" International Quality Standard anyone? Other ISO and International Standards and European Regulations 33
F Future Plan for IMDS (International Material Data System) RoHS, REACH, ELV, IMDS and Restricted Substances 5
J International Standards governing Test Method Validations ISO 13485:2016 - Medical Device Quality Management Systems 1
R Determination of IP (International Protection Marking) Rating IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
U ISO 9001 Certification for Highly Distributed International Company ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
M Calibration Procedure Traceable to National/International Standards (API Q1 9th Ed) Oil and Gas Industry Standards and Regulations 3
M Basic International Standards required for a Manufacturing Industry (Optical Media) Other ISO and International Standards and European Regulations 4
M Differences between international and regional standards (EN, AAMI, CSA...) Other Medical Device Related Standards 6
D International Standards (ISO or MIL) for MTBV Reliability Analysis - Predictions, Testing and Standards 4
automoto International Problem Solving Guide Problem Solving, Root Cause Fault and Failure Analysis 2
AnaMariaVR2 The International Pharmacopoeia (Ph. Int.) WHO US Food and Drug Administration (FDA) 1
I International Sales Legal/Regulatory Gray Area? CE Marking (Conformité Européene) / CB Scheme 3
M FDA Acceptance of ISO 13485 Reports from a Recognized International Program Other US Medical Device Regulations 4
I International Accreditation Board of Bristol, UK ASQ, ANAB, UKAS, IAF, IRCA, Exemplar Global and Related Organizations 11
Jen Kirley This international consultant thing sure is a ride! Career and Occupation Discussions 13
S IATF (International Automotive Task Force) Rules 4th edition changes IATF 16949 - Automotive Quality Systems Standard 1
P IMDRF (International Medical Devices Regulators Forum)guidance on standalone software Medical Information Technology, Medical Software and Health Informatics 1
J Rayong Thailand - Best Hotel and closest international Airport Travel - Hotels, Motels, Planes and Trains 3
J Domestic and International Medical Device IFU (Instructions for Use) Other US Medical Device Regulations 3
E Are non-SI units International? General Measurement Device and Calibration Topics 7
S The days before International Standards - Has ISO done what it was supposed to? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 7
D RABQSA International (SUSPENDED) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
Icy Mountain Combining ISO9001 and ITAR (International Traffic in Arms Regulations) ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 12
M Measurement Standards Traceable to International or National Measurement Standards AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 4
B What a mess!!! Nebula Glass International (d.b.a. Glasslam) vs. Wacker Chemicals Quality Manager and Management Related Issues 1
T International Medical Data Storage Requirements Other Medical Device Regulations World-Wide 4
H Painting Quality Requirement International Standard Manufacturing and Related Processes 6
D FDA Cease to Manufacture Order in USA and our International Interests 21 CFR Part 820 - US FDA Quality System Regulations (QSR) 11
T Are there any International Conferences related to ISO/IEC 27000 series standards IEC 27001 - Information Security Management Systems (ISMS) 1
A Instructions for Use - Best Practice to meet all International Regulations ISO 13485:2016 - Medical Device Quality Management Systems 6

Similar threads

Top Bottom