Rating Auditors - A 'Clearinghouse' for Auditors or a Way to Rate Auditors?

Wes Bucey

Quite Involved in Discussions
#1
Just some musings:

The auditor action which triggered this thread really depresses me. We know there are "strong personalities" (like Marc, Randy, Wes Bucey, Rob Nixon, Bill Pflanz, et al) who would have thrown this "would-be auditor" out on his ear if he tried that in our shop.

I would have demanded all fees which had anything to do with such a clown back from the registrar. My fellow executives and stockholders would have backed my play.

What happens, though, when no one screams FOUL! at such an action? I know why they wouldn't -
  • little top management support to begin with;
  • customers who DEMAND the registration as the price of doing business;
  • the "no big deal" thing (since it didn't result in a nonconformance)
  • uncertainty about whether we, the organization, or the auditor is correct
  • and so on.
In another thread, Sidney suggested a customer should be aware of the quality of the registrar and its auditors who are attesting to the conformance of a QMS. I said something to the effect (ala "Chico and the Man") - "That's not my job!"

This thread, though, has caused me to modify my position a little. It may NOT be my job, but perhaps we need a central clearing house which "names names" so that customers and potential auditees have a little stronger information base to search about issues like this one or others which never make it to the Cove. I'd sure like to hear "the rest of the story" including the auditor's reasoning and his point of view.

I don't know about you, but I sure wouldn't want that auditor coming to my shop until he had a "serious attitude adjustment."

The main problem I see with such a clearing house is that many of the complaints would be invalid or improperly documented. I'm definitely reminded of product rejections and CARs sent by customers which proved to be completely unfounded, but which caused a lot of wasted time and motion plus many hurt feelings. The truth is, there are organizations which do not deserve to be registered to an ISO Standard.

So, here's the quandary:
  1. How do we protect ourselves from getting an incompetent auditor?
  2. What can we do to alert others about "suspicious" characters in the auditing business without leaving ourselves and our organizations open to libel and slander suits?
  3. What can we do when we are being victimized by an incompetent auditor, but don't have the personality and status within our organizations to get management backing for a protest?
  4. I don't think the accrediting bodies have an adequate mechanism to deal with and publicize the complaint and the subsequent resolution.
 
Elsmar Forum Sponsor

Sidney Vianna

Post Responsibly
Staff member
Admin
#2
The power of the WEB

Wes Bucey said:
So, here's the quandary:
  1. How do we protect ourselves from getting an incompetent auditor?
  2. What can we do to alert others about "suspicious" characters in the auditing business without leaving ourselves and our organizations open to libel and slander suits?
  3. What can we do when we are being victimized by an incompetent auditor, but don't have the personality and status within our organizations to get management backing for a protest?
  4. I don't think the accrediting bodies have an adequate mechanism to deal with and publicize the complaint and the subsequent resolution.
Great post, Wes.
You know, there is a website creating a lot of interest and discussion called www.ratemyteachers.com in which students can grade their respective teachers all over the country. Maybe Marc could create something like ratemyauditor.com eek.gif

Some of the problems would be liability, transparency, confidentiality and also the fact that, many, many people would give great reviews to mediocre auditors hoping to artificially boost that auditor's rating just to show to their bosses that they have the "best auditor in the Universe" omg.gif . Something similar happened to the "rate the registrars surveys". On the other extreme, people being very appropriately written up for poor systems could use this as a way of getting back at the (good) auditors.
 

Wes Bucey

Quite Involved in Discussions
#3
Sidney Vianna said:
Great post, Wes.
You know, there is a website creating a lot of interest and discussion called www.ratemyteachers.com in which students can grade their respective teachers all over the country. Maybe Marc could create something like ratemyauditor.com eek.gif

Some of the problems would be liability, transparency, confidentiality and also the fact that, many, many people would give great reviews to mediocre auditors hoping to artificially boost that auditor's rating just to show to their bosses that they have the "best auditor in the Universe" omg.gif . Something similar happened to the "rate the registrars surveys". On the other extreme, people being very appropriately written up for poor systems could use this as a way of getting back at the (good) auditors.
As I said,
The main problem I see with such a clearing house is that many of the complaints would be invalid or improperly documented. I'm definitely reminded of product rejections and CARs sent by customers which proved to be completely unfounded, but which caused a lot of wasted time and motion plus many hurt feelings. The truth is, there are organizations which do not deserve to be registered to an ISO Standard.
the rating alone would not be adequate - too easy to skew the results.

There definitely would have to be a mechanism with "depth" (point by point reasons behind a rating) and some protection for auditor against libel, slander by vindictive types as well as some mechanism to protect person doing the rating against retaliation by lawsuits.

In my opinion, the accrediting body should be MUCH MORE OPEN AND COMMUNICATIVE with the public about the process of dealing with complaints.

What is the process, for example?
  1. If RosieA complained to the registrar for whom the auditor works, how soon should such a complaint be resolved to RosieA's satisfaction before RosieA goes to the accrediting body?
  2. When RosieA ultimately goes to the accrediting body, what do they do?
    (write an inquiry back to the registrar like the Better Business Bureau?)
  3. If they investigate, how do we know it's not a "whitewash"?
    (Should there be a detailed, reasoned report? Who should be allowed to see reports? Can there be a public record of the reports?)
  4. How soon should RosieA expect results?
  5. Should the accrediting body have power of arbitration? (like NASD or SEC dealing with bad stockbrokers?)
I think I'm going to watch some mindless entertainment on TV - this is too much like work!
 
R

Rob Nix

#5
Wes,

First of all, let me make two things perfectly clear, 1) I am not a crook, and 2 ) it is Nix, not Nixon. :lol:

Second, this is a very good thread. I think I’ll add a few points whether they answer any of the questions or not.

There is a mechanism within each registrar’s organization that helps them get feedback about their auditors: a survey that allows your organization to express itself; to disagree with the findings of the auditor. My registrar also has both a continuous improvement survey and a customer satisfaction survey on its web site as well (I’m sure other registrars have something similar). NOTE: RosieA - Did you send your survey in with your concerns expressed?

I used the surveys (and supporting E-Mails) twice with the same auditor and eventually had him removed from the list of auditors auditing our facility. When I moved on to another company, I used the same registrar with the stipulation that they not assign that particular auditor.

The two incidents involved were likewise trivial.
1) A Shipping and Receiving person used a tape measure to identify one product from another similar one. There was a two inch difference. The auditor tended to be intimidating and often used leading questions. In this case he asked, “So, you use that tape measure to inspect the two similar parts, right?” The Receiving guy stammered, “Uh, yeah, I guess”. The auditor said that since the tape measure was an “inspection” device, it had to come under calibration control - which it did not - and a minor was written (I got it removed).
2) A sound meter was written up by the auditor for not having epoxy affixed to the adjustment screw “to prevent inadvertent adjustment”. I explained that the screw was placed in a recessed hole by design for that very purpose! Same result as incident number one. The big problem was that this guy intimidated our people, making them nervous and unsure what he was driving at; they’d “slip up”, and he prided himself on “catching them”.

While this method only addresses the concerns Wes raises to the registrar level, it is something. Perhaps an expansive list of trivial issues could be built somewhere that could be forwarded to RAB. They could filter out the “valids” from the “invalids” and send letters of clarification downstream to the registrars and auditing organizations. It doesn’t directly deal with the offending auditors, but perhaps they’ll receive this information and modify their practices. Like I said earlier, this may not answer the original question, they’re simply my rambling thoughts.
 
B

Bill Pflanz

#6
I have still not decided whether I have been insulted or complimented by the strong personality description. At least I am in good company. :D

I have challenged bad auditing findings by working directly with the auditor's manager. Unfortunately, the same auditor was auditing other parts of my company and their ISO representatives were also complaining. Finally, one of them threatened to file a formal complaint with the registrar if she was not removed as an auditor. I think they just stopped using her at our company and sent her to another client. It solved our problem but some poor quality manager out there got stuck with her.

I wonder how many ISO auditors get their auditing certifications revoked so they cannot impose themselves on other poor, unsuspecting clients of the same registrar or another.

Bill Pflanz
 
R

RosieA

#8
Rob Nix said:
NOTE: RosieA - Did you send your survey in with your concerns expressed?
Hi Rob,
Now that you mention it, this Registrar, one of 4 I've worked with, does NOT hand out satisfaction surveys at the end of the audit. I haven't been to their website to see if there is one posted there, but I will. Good point.

Rosie
 

Wes Bucey

Quite Involved in Discussions
#10
sal881vw said:
No offence intended but............
I'm not the only one with brain freeze:rolleyes:
If you were to shout beacon you may get a different reaction.(my italics)
I think "bacon" is the intent.
 
Thread starter Similar threads Forum Replies Date
M Reduce occurrence rating based on the PMS data and customer complaint data ISO 14971 - Medical Device Risk Management 2
W FMEA - Current control and occurrence rating FMEA and Control Plans 3
D FAA inspection at my repair station - Rating Check Federal Aviation Administration (FAA) Standards and Requirements 0
S Complexity Rating - CB adding another audit day for "high complexity" AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 6
P Design FMEA - Detection Rating criteria ISO 14971 - Medical Device Risk Management 3
S Power rating for medical devices in the US - Radiotherapy devices US Food and Drug Administration (FDA) 0
P Detection rating for a user (surgeon) related failure mode in DFMEA APQP and PPAP 3
M Should Potential Customer Complaint Outcome Define Registrar NC Rating? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 8
R Determination of IP (International Protection Marking) Rating IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
M Supplier 8D Report-Rating Procedure Problem Solving, Root Cause Fault and Failure Analysis 10
A Supplier Rating from SAP QM Module Data Supplier Quality Assurance and other Supplier Issues 2
B Third Party Service Provider Risk Rating Spreadsheet Risk Management Principles and Generic Guidelines 1
S FMEA Visual Inspection Detection Rating FMEA and Control Plans 5
V Where to find Red/White Rust Rating Requirements? APQP and PPAP 4
L Flammability Rating - is FT4 a better rating than FT1? Other US Medical Device Regulations 2
B FMEA Ranking Rating Best Practices FMEA and Control Plans 10
P Occurrence Rating in AIAG FMEA Manual 4th Edition FMEA and Control Plans 5
R How much Protective Current Rating of Building Branch Circuit should be considered? IEC 60601 - Medical Electrical Equipment Safety Standards Series 2
W Severity Rating CORRECTIVE ACTIONS - Opinions Wanted Please Nonconformance and Corrective Action 15
P Risk Rating approaches (OHSAS 18001 - For Office Environments) Occupational Health & Safety Management Standards 6
S Smartphone based software with automatic "rating" from remote servers Medical Information Technology, Medical Software and Health Informatics 4
T Enclosure Flammability Rating of Molded-In Custom Color Liquid Concentrates IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
J Determining Fuse Rating and Labeling - Medical Device Crash Cart IEC 60601 - Medical Electrical Equipment Safety Standards Series 3
M FMEA Occurrence Rating for Customer Claims (Complaints) FMEA and Control Plans 2
D Flame Rating for Internal Plastic Support inside of a Fire Enclosure IEC 60601 - Medical Electrical Equipment Safety Standards Series 4
K Supplier Performance and Supplier Rating Process - AS9100 Requirements AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 12
J How to Increase Customer Satisfaction Rating? Customer Complaints 20
M Supplier Rating Form with Quality, Cost, Delivery wanted Document Control Systems, Procedures, Forms and Templates 4
C Supplier Performance Index (Spi) - Supplier Rating Supplier Quality Assurance and other Supplier Issues 10
S How to determine what rating an article falls under - 14 CFR Part 145 Federal Aviation Administration (FAA) Standards and Requirements 5
L Gap in FMEA Occurrence Rating Standard FMEA and Control Plans 2
S About the rating of fuse - Can I employ 2 fuses of different rating in my equipment? IEC 60601 - Medical Electrical Equipment Safety Standards Series 9
A PFMEA (Process FMEA) Detection Rating - Actions to Reduce RPN FMEA and Control Plans 3
I Service Supplier Rating where Objective Pass/Fail Data is not Available Service Industry Specific Topics 7
V In Design FMEA, how to give Occurrence Rating? FMEA and Control Plans 4
P Detection Rating in PFMEA and Process Controls FMEA and Control Plans 6
M FMEA Rating Scales for the Insurance Industry FMEA and Control Plans 4
Q FMEA: Visual Detection Rating IATF 16949 - Automotive Quality Systems Standard 25
N All-or-Nothing Rating System - Internal Audit Checklist - ISO 9001 Internal Auditing 5
Chennaiite FMEA Severity Rating for Potential Manufacturing Effect mentioned in the FMEA manual FMEA and Control Plans 9
R Supplier Rating System Ideas and Suggestions Quality Manager and Management Related Issues 3
R Supplier Quality Rating Scorecard Template (excel spreadsheet) Supplier Quality Assurance and other Supplier Issues 4
D How to use the Supplier Rating Calculator Supplier Quality Assurance and other Supplier Issues 2
S Supplier Quality Rating - What to evaluate Supplier Quality Assurance and other Supplier Issues 11
Chennaiite PFMEA rating - Occurence Rating and Detection Rating - Some clarification FMEA and Control Plans 15
A Vendor Rating System - IS:12040-1987 (India Standard) - Where to buy ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
S OHS Risk Analysis - What does the 'Risk Rating' refer to? Total or Residual? Occupational Health & Safety Management Standards 5
D Tools for CMMI Rating the SCAMPI approach Software Quality Assurance 1
I FMEA Detection Rating - What is Soft or Hard? FMEA and Control Plans 7
J Need Advice on Supplier Performance Rating System Supplier Quality Assurance and other Supplier Issues 9

Similar threads

Top Bottom