Survey for internal use to Evaluate Internal Auditors

S

Steven Sulkin

#1
I am putting together a survey for internal use to evaluate auditors. My idea is to have the (short) survey handed out to auditees to give the auditor feedback. I will collect these surveys and use them to develop coursework and direct auditor meeting discussions.

I would appreciate some ideas and questions. Here is what I have so far...
I would appreciate your opinion on this. I am trying to develop a satisfaction survey for auditors. Here is the list so far....

The auditor knows the standard.
The auditor gave me some valuable suggestions on how to improve my area.
The auditor gave a good feel for how my system is operating from the perspective of the standard.
The auditor’s questions were logical and clear.
I felt like the auditor was here to help me.

and an overall...

The audit was worth the time invested.

Thanks,

Steve.
 
Elsmar Forum Sponsor
D

Dusty

#2
The auditor gave me some valuable suggestions on how to improve my area.
Steven,
Confused on this statement/question. During our class on lead auditing, we were told NOT to consult. Have just recently gone through this class, so granted have not the experience as others...just wondering about this.

------------------
Dusty Rhoads
(Chief Dummy)

[This message has been edited by Dusty (edited 18 October 1999).]
 

Marc

Hunkered Down for the Duration
Staff member
Admin
#3
Lead Auditor course is distinct from internal auditing.

I don't have any advice on the survey off hand. I can say that "The auditor knows the standard" isn't really valid - how do they know if you know the spec? Few auditees would have any idea. Same for "The auditor gave a good feel for how my system is operating from the perspective of the standard." I have seen "The auditor gave me some valuable suggestions on how to improve my area" piss people off because they don't see the connection - sounds like someone who doesn't even work in the area is telling them how do do their job (or whatever).

I applaud the feedback idea, however more common is where it's hard enough to get the audits done period, much less time for a survey. Let us know how this works out for you!

I'm not really trying to be negative - just some thoughts.
 

barb butrym

Quite Involved in Discussions
#4
YUP...Lead auditor is a different animal..."consulting" does not apply to 1st party stuff. An internal auditor is a great way to add "value".

The survey....hmmmmm.. I have to agree with marc on the "know the standard" stuff..and I too like the idea of feedback.

Maybe an independant observer would provide the feedback you need on true understanding of the standard...maybe each auditor is observed once for each "round" Or even an evaluation of the report, if the report is comprehensive, would show that.

Do the auditors need to understand the standards in depth, or is the 'standard/scope' your internal system? If its the internal system, then the auditee may be able to answer the question. I would guess its both...the more experienced auditors will know the 'standard' as well as the internal system requirements.

Every company is different.....I say try it, then work it to fit your company culture.

dusty...you're no dummy, chief or not. A course is just the 1st step...experience will make the difference. AND the only truley dumb question is the one not asked!!!


[This message has been edited by barb butrym (edited 19 October 1999).]
 
S

Steven Sulkin

#5
I am thinking about a short survey that should take no more than a few minutes (7questions?). The internal audits are a pain in the butt sometimes. I believe the auditees would appreciate the opportunity to shape the function into something more valuable for themselves.

Thanks for your points. I would agree with Marc- internal audits are different from 3rd or 2nd party.

To your comment about some people being sensitive to advice from outsiders--

Our folks have been audited for going on six years now. Management has bought into the idea, and the employees see how much we have changed (for the better) with ISO9001. I am very happy to say we are over that (not completely of course. There will always be 5% that dont want to change). We expect the auditors to be experts on the standard. We expect that they will show us how to improve by using it to our advantage. I am going to be investing in more training for them to be able to do that. I need this tool in place to provide a baseline and feedback through the development process.

Good feedback on knowing the standard. I didnt think of that. I want the auditors to know how to use the standard as a tool. We have all had the experience of implementing a beaurocratic policy to meet the standard. Eventually we learn how to meet the intent and add value to the company. This is where I want them to be. Besides training I also want to participate in benchmarking opportunities locally to help them think outside the box. Wouldnt you agree that most auditees would be delighted if you could show them a better way- to meet the intent and still add value?

I hope so. We will see.
 
S

Steven Sulkin

#6
To Dusty's comment... you are correct, 2nd and 3rd party auditors should not tell the auditee how to do their job. Unfortunately, many 2nd party folks do. But I have been very happy with auditors who have given examples of how other companies have met the same intent (as an example only).

Good luck in your new role. Dont spend all those frequent fliers in one place!
 

Kevin Mader

One of THE Original Covers!
Staff member
Admin
#7
*The auditor knows the standard.

Is the auditee in a position to make this distinction?


*The auditor gave me some valuable suggestions on how to improve my area.

On an internal audit, the auditor may offer advice. However, as mentioned above, the auditee may interpret this differently and do more harm than good. My opinion, internal auditors should not give advice as Dusty pointed out.

*The auditor gave a good feel for how my system is operating from the perspective of the standard.

Feedback is important, especially positive feedback. This can be a dangerous area though, as the feedback may go beyond the scope of the audit or standard ( the auditor may end up offering advice).

*The auditor’s questions were logical and clear.

A good question. The purpose of the audit is not to fool or confuse the auditee, but to get down to suitability, compliance, and effectiveness.

*I felt like the auditor was here to help me.

Another good question. This is a continuous improvement tool and not an inquisition.

Back to the group...
 
L

Laura M

#8
At a large company, "internal customer satisfaction" is a useful tool and I think that is what your survey idea is getting at. Typically "support functions" (i.e. quality auditors) have a hard time making the manufacturing side appreciate their existance (my experience, anyways). So the survey idea is a good one from that point of view.
I always felt that my internal auditors were. in a sense, "training" during the audit. If something didn't comply, then the auditor needed to site the element, or company procedure (which hopefully tied to an element or company objective!) That can be how you get at the "auditor knows the standard." Maybe a question on "the auditor fully explained non-conformance received" - something the auditor may walk away thinking they did, but maybe the area didn't quite "get it".
I always liked a question that had to do with being cordial, as well. This comes somewhat from a 3rd party experience, but works for internal audits as well. If I can dig up the one I used (in a previous life) I'll post it here for everyone.
 
J

John C

#9
Steven,
When you set out to do something, it's always a good idea to consider why you are doing it. In fact, it's better to have identified a need and then looked at the optional ways of achieving it.

What is your objective? To do a survey? or to find out how good your audit is?

Now I think there are a number of ways to find out how good your audit is. The best is to judge the effectiveness of the audit by identifying objective improvement data, if that is possible. The next, I'd say, is at the audit closing meeting, where you can hear and see the response of your auditee management team. Do they mean it when they thank you? Or are they just showing relief it's all over for another couple of months?

You can ask them at the meeting, when you have the most leverage - It helps if you make sure their boss or their bosses boss is there. You can ask them to gather the concerns and positive points of their people who were audited.

But I would keep individuals and personalities out of it. Stick to judging only the process, as we auditors do, and avoid threatening or upsetting the audit team.

The method of feedback you propose seems to come from operator or supervisor level. For reasons given in responses above, eg Dusty says 'don't consult'(if you do you take responsibilty for all the trouble that happens in future). 'The auditees don't know the standard'. 'Your auditors knowledge of the standard is limited and when questioned will show cracks' (like the rest of us).

My main concern is that you may be storing up a heap of trouble and getting little in return;

No-one likes being reviewed, least of all by someone neither qualified, nor in a position of authority to do so. Your auditors are going to feel threatened every time they audit. Already they feel under pressure to produce results (ie NCs), now every action and word will be judged. The auditees will have being given a weapon to fight back.

Will the auditors avoid audits in future? Not volunteer? Look for another job?
Will they respond by being aggressive? or too lenient? Avoid troublesome areas? Go back and get revenge next time?
Will the auditees settle old scores? Respond to NCs by arguing and turning the tables on the auditor by pointing out their weaknesses? Will they finish up insulting each other and throwing eggs?
Will your superiors see all this stuff in the reports and decide that you would be better off doing something else? maybe somewhere else?

I don't like it. Sounds like a lot of hassle, even a minefield, and even if you guide it wisely and your organisation is the sort of place where it can be used, I see no obvious gains to come from it.

Certainly look for feeback but don't lose the upper hand. Be prepared to defend your area against attack if it is seen as the opportunity to do so, after all, it's your job to make the decisions and, when you're interfering in other people's areas, they'll soon enough put you down if the opportunity arises. Work closely with them. Don't put questions in their mouths and doubts in everyone's heads. Tell them how good your team is and help them understand how their input might make it even better.

rgds,
John C
 
S

Steven Sulkin

#10
Interesting feedback. Good point about the closing meeting. Now that I think about it, we really dont take good advantage of closing meetings. I will look at changing the meeting format.

Per the survey- I have had a lot of success (with previous employer) on internal departmental surveys. Its like a 360 degree performance appraisal, except it applies for departments. Instead of surveying all auditees I may look at auditing only main internal customers.

Bottom line is, I have to know if the group is adding value. I have to know that my main internal customers feel like they are getting good service. The auditing function needs to be accountable for the value they add just like every other area. The survey should give us good feedback on how we are doing, what we need to improve, and if we are getting better. Assuming, of course, the tool is well developed.

If we were to put together such a survey, one that is distributed to main internal customers (e.g. manufacturing manager, VP-Research and Development, Lab Manager, etc), what questions would you recommend to evaluate the auditor function? Again, my objective is to determine if the audit is adding value and what can be improved.

Can we come up with seven questions?

To start this off- we would need one overall question to gauge satifaction over time.

How about-
"The audit was worth the time invested."
 
Thread starter Similar threads Forum Replies Date
S Need to prepare Internal Customer Satisfaction Survey form for QA department Misc. Quality Assurance and Business Systems Related Topics 5
M Internal and External Customer Survey Procedure and Form examples wanted ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
V Internal Audit Satisfaction Survey Examples wanted Internal Auditing 16
L Sample Of Internal Clients Survey Needed Document Control Systems, Procedures, Forms and Templates 6
P ISO 13485:2016 MDSAP Certification Fee Survey ISO 13485:2016 - Medical Device Quality Management Systems 6
S Need help with analysing a survey on minitab Using Minitab Software 1
M Informational From RAPS – 27% Will be Compliant? Survey Highlights Lack of Readiness for EU MDR Medical Device and FDA Regulations and Standards News 0
M Informational Team-NB published a press release regarding the survey run among its members to analyse the certificates being issued Medical Device and FDA Regulations and Standards News 0
optomist1 Survey of Tier 1 & Tier 2 suppliers APQP and PPAP 6
B [Help] Customer Engagement Survey for MBA Project - due Apr19 Manufacturing and Related Processes 0
B Help with Customer Engagement Survey for MBA Project - due Apr19 General Measurement Device and Calibration Topics 0
T Likert Analysis of Data from a Survey Statistical Analysis Tools, Techniques and SPC 6
A Efficacy of ISO 9001:2015 - Survey ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 10
T Assessment / Audit / Survey for Indirect Suppliers or Material ISO 13485:2016 - Medical Device Quality Management Systems 3
N Customer Survey Question Has Me Stumped - Compliance with Laws and Regulations Customer and Company Specific Requirements 4
K Temperature Uniformity Survey (TUS) - Vacuum Furnace General Measurement Device and Calibration Topics 1
D Alternatives to Customer Feedback Survey in ISO 9001 Quality Manager and Management Related Issues 4
N Interesting Discussion How to Deal with Suppliers Who Refuse to Complete our ISO Survey? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 51
N Survey to Customers Regarding Our Complaint Response Time Customer Complaints 7
L 5 minutes only for this Survey on the Value of PDCA and Your EMS Miscellaneous Environmental Standards and EMS Related Discussions 1
S Conduct Customer Satisfaction Survey for every submitted Bid? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 3
N Load Thermocouple requirement for TUS (Temperature Uniformity Survey) AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 8
J Looking for a Customer Satisfaction Survey Form Other Medical Device Related Standards 2
Sean Kelley How to upload GM's MMOG survey version 4 to GM Powersupply Customer and Company Specific Requirements 3
C Vendor Self-Survey to Qualify Vendors/Suppliers - AVL Question ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
Richard Regalado 2014 Information Security Breaches Survey by PWC IEC 27001 - Information Security Management Systems (ISMS) 1
Richard Regalado Weathering the storm - The 2013 Business Continuity Management Survey Business Continuity & Resiliency Planning (BCRP) 2
R Female auditors earn 18% less than male auditors - IRCA Salary Survey 2014 Career and Occupation Discussions 0
S What is the minimum samples required for Customer Survey? Quality Manager and Management Related Issues 3
John Broomfield Survey on Management Quality Manager and Management Related Issues 8
Z Customer Satisfaction Survey Analysis Quality Tools, Improvement and Analysis 2
M Temperature Uniformity Survey Sensors AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 7
C Customer Survey Assessment Feedback Customer Complaints 11
M Customer Satisfaction Survey - Poor Response ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 11
N Survey about Attribute Gage and Sight Checking R&R Gage R&R (GR&R) and MSA (Measurement Systems Analysis) 5
T A survey on problems during ISMS implementation - need help IEC 27001 - Information Security Management Systems (ISMS) 2
P Should I send survey questionnaire prior to Supplier EMS Audit? Miscellaneous Environmental Standards and EMS Related Discussions 14
D U.S. Department of Commerce, Bureau of Industry and Security survey Various Other Specifications, Standards, and related Requirements 1
W IR Camera - Heat Survey of parts during dynamic testing General Measurement Device and Calibration Topics 5
S Airborne Survey Legal Compliance - OHSAS 18001 Occupational Health & Safety Management Standards 2
N What Questions Are On Your Customer Survey? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 2
B What Customers Return Survey Results? Customer Complaints 1
B What is the Appropriate Sample Size for a Customer Survey? Customer Complaints 5
B Your thoughts on Communicating to Customers the Actions on Customer Survey Customer Complaints 4
T Brief Survey for OEM's about Outsourcing Supplier Quality Assurance and other Supplier Issues 5
R Supplier Satisfaction Survey about Working with us as a Customer Supplier Quality Assurance and other Supplier Issues 7
I Request to participate in ISO 14001 survey for Masters Thesis ISO 14001:2015 Specific Discussions 3
B Writing a Customer Survey......Any tips? Customer Complaints 13
T A little survey on ISMS Implementation - Need help IEC 27001 - Information Security Management Systems (ISMS) 12
I Survey - The impact of ISO 9001 Certification on Organisational Peformance ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 4

Similar threads

Top Bottom