Survey for internal use to Evaluate Internal Auditors

  • Thread starter Thread starter Steven Sulkin
  • Start date Start date
S

Steven Sulkin

I am putting together a survey for internal use to evaluate auditors. My idea is to have the (short) survey handed out to auditees to give the auditor feedback. I will collect these surveys and use them to develop coursework and direct auditor meeting discussions.

I would appreciate some ideas and questions. Here is what I have so far...
I would appreciate your opinion on this. I am trying to develop a satisfaction survey for auditors. Here is the list so far....

The auditor knows the standard.
The auditor gave me some valuable suggestions on how to improve my area.
The auditor gave a good feel for how my system is operating from the perspective of the standard.
The auditor’s questions were logical and clear.
I felt like the auditor was here to help me.

and an overall...

The audit was worth the time invested.

Thanks,

Steve.
 
Elsmar Forum Sponsor
The auditor gave me some valuable suggestions on how to improve my area.

Steven,
Confused on this statement/question. During our class on lead auditing, we were told NOT to consult. Have just recently gone through this class, so granted have not the experience as others...just wondering about this.

------------------
Dusty Rhoads
(Chief Dummy)

[This message has been edited by Dusty (edited 18 October 1999).]
 
Lead Auditor course is distinct from internal auditing.

I don't have any advice on the survey off hand. I can say that "The auditor knows the standard" isn't really valid - how do they know if you know the spec? Few auditees would have any idea. Same for "The auditor gave a good feel for how my system is operating from the perspective of the standard." I have seen "The auditor gave me some valuable suggestions on how to improve my area" piss people off because they don't see the connection - sounds like someone who doesn't even work in the area is telling them how do do their job (or whatever).

I applaud the feedback idea, however more common is where it's hard enough to get the audits done period, much less time for a survey. Let us know how this works out for you!

I'm not really trying to be negative - just some thoughts.
 
YUP...Lead auditor is a different animal..."consulting" does not apply to 1st party stuff. An internal auditor is a great way to add "value".

The survey....hmmmmm.. I have to agree with marc on the "know the standard" stuff..and I too like the idea of feedback.

Maybe an independant observer would provide the feedback you need on true understanding of the standard...maybe each auditor is observed once for each "round" Or even an evaluation of the report, if the report is comprehensive, would show that.

Do the auditors need to understand the standards in depth, or is the 'standard/scope' your internal system? If its the internal system, then the auditee may be able to answer the question. I would guess its both...the more experienced auditors will know the 'standard' as well as the internal system requirements.

Every company is different.....I say try it, then work it to fit your company culture.

dusty...you're no dummy, chief or not. A course is just the 1st step...experience will make the difference. AND the only truley dumb question is the one not asked!!!


[This message has been edited by barb butrym (edited 19 October 1999).]
 
I am thinking about a short survey that should take no more than a few minutes (7questions?). The internal audits are a pain in the butt sometimes. I believe the auditees would appreciate the opportunity to shape the function into something more valuable for themselves.

Thanks for your points. I would agree with Marc- internal audits are different from 3rd or 2nd party.

To your comment about some people being sensitive to advice from outsiders--

Our folks have been audited for going on six years now. Management has bought into the idea, and the employees see how much we have changed (for the better) with ISO9001. I am very happy to say we are over that (not completely of course. There will always be 5% that dont want to change). We expect the auditors to be experts on the standard. We expect that they will show us how to improve by using it to our advantage. I am going to be investing in more training for them to be able to do that. I need this tool in place to provide a baseline and feedback through the development process.

Good feedback on knowing the standard. I didnt think of that. I want the auditors to know how to use the standard as a tool. We have all had the experience of implementing a beaurocratic policy to meet the standard. Eventually we learn how to meet the intent and add value to the company. This is where I want them to be. Besides training I also want to participate in benchmarking opportunities locally to help them think outside the box. Wouldnt you agree that most auditees would be delighted if you could show them a better way- to meet the intent and still add value?

I hope so. We will see.
 
To Dusty's comment... you are correct, 2nd and 3rd party auditors should not tell the auditee how to do their job. Unfortunately, many 2nd party folks do. But I have been very happy with auditors who have given examples of how other companies have met the same intent (as an example only).

Good luck in your new role. Dont spend all those frequent fliers in one place!
 
*The auditor knows the standard.

Is the auditee in a position to make this distinction?


*The auditor gave me some valuable suggestions on how to improve my area.

On an internal audit, the auditor may offer advice. However, as mentioned above, the auditee may interpret this differently and do more harm than good. My opinion, internal auditors should not give advice as Dusty pointed out.

*The auditor gave a good feel for how my system is operating from the perspective of the standard.

Feedback is important, especially positive feedback. This can be a dangerous area though, as the feedback may go beyond the scope of the audit or standard ( the auditor may end up offering advice).

*The auditor’s questions were logical and clear.

A good question. The purpose of the audit is not to fool or confuse the auditee, but to get down to suitability, compliance, and effectiveness.

*I felt like the auditor was here to help me.

Another good question. This is a continuous improvement tool and not an inquisition.

Back to the group...
 
At a large company, "internal customer satisfaction" is a useful tool and I think that is what your survey idea is getting at. Typically "support functions" (i.e. quality auditors) have a hard time making the manufacturing side appreciate their existance (my experience, anyways). So the survey idea is a good one from that point of view.
I always felt that my internal auditors were. in a sense, "training" during the audit. If something didn't comply, then the auditor needed to site the element, or company procedure (which hopefully tied to an element or company objective!) That can be how you get at the "auditor knows the standard." Maybe a question on "the auditor fully explained non-conformance received" - something the auditor may walk away thinking they did, but maybe the area didn't quite "get it".
I always liked a question that had to do with being cordial, as well. This comes somewhat from a 3rd party experience, but works for internal audits as well. If I can dig up the one I used (in a previous life) I'll post it here for everyone.
 
Steven,
When you set out to do something, it's always a good idea to consider why you are doing it. In fact, it's better to have identified a need and then looked at the optional ways of achieving it.

What is your objective? To do a survey? or to find out how good your audit is?

Now I think there are a number of ways to find out how good your audit is. The best is to judge the effectiveness of the audit by identifying objective improvement data, if that is possible. The next, I'd say, is at the audit closing meeting, where you can hear and see the response of your auditee management team. Do they mean it when they thank you? Or are they just showing relief it's all over for another couple of months?

You can ask them at the meeting, when you have the most leverage - It helps if you make sure their boss or their bosses boss is there. You can ask them to gather the concerns and positive points of their people who were audited.

But I would keep individuals and personalities out of it. Stick to judging only the process, as we auditors do, and avoid threatening or upsetting the audit team.

The method of feedback you propose seems to come from operator or supervisor level. For reasons given in responses above, eg Dusty says 'don't consult'(if you do you take responsibilty for all the trouble that happens in future). 'The auditees don't know the standard'. 'Your auditors knowledge of the standard is limited and when questioned will show cracks' (like the rest of us).

My main concern is that you may be storing up a heap of trouble and getting little in return;

No-one likes being reviewed, least of all by someone neither qualified, nor in a position of authority to do so. Your auditors are going to feel threatened every time they audit. Already they feel under pressure to produce results (ie NCs), now every action and word will be judged. The auditees will have being given a weapon to fight back.

Will the auditors avoid audits in future? Not volunteer? Look for another job?
Will they respond by being aggressive? or too lenient? Avoid troublesome areas? Go back and get revenge next time?
Will the auditees settle old scores? Respond to NCs by arguing and turning the tables on the auditor by pointing out their weaknesses? Will they finish up insulting each other and throwing eggs?
Will your superiors see all this stuff in the reports and decide that you would be better off doing something else? maybe somewhere else?

I don't like it. Sounds like a lot of hassle, even a minefield, and even if you guide it wisely and your organisation is the sort of place where it can be used, I see no obvious gains to come from it.

Certainly look for feeback but don't lose the upper hand. Be prepared to defend your area against attack if it is seen as the opportunity to do so, after all, it's your job to make the decisions and, when you're interfering in other people's areas, they'll soon enough put you down if the opportunity arises. Work closely with them. Don't put questions in their mouths and doubts in everyone's heads. Tell them how good your team is and help them understand how their input might make it even better.

rgds,
John C
 
Interesting feedback. Good point about the closing meeting. Now that I think about it, we really dont take good advantage of closing meetings. I will look at changing the meeting format.

Per the survey- I have had a lot of success (with previous employer) on internal departmental surveys. Its like a 360 degree performance appraisal, except it applies for departments. Instead of surveying all auditees I may look at auditing only main internal customers.

Bottom line is, I have to know if the group is adding value. I have to know that my main internal customers feel like they are getting good service. The auditing function needs to be accountable for the value they add just like every other area. The survey should give us good feedback on how we are doing, what we need to improve, and if we are getting better. Assuming, of course, the tool is well developed.

If we were to put together such a survey, one that is distributed to main internal customers (e.g. manufacturing manager, VP-Research and Development, Lab Manager, etc), what questions would you recommend to evaluate the auditor function? Again, my objective is to determine if the audit is adding value and what can be improved.

Can we come up with seven questions?

To start this off- we would need one overall question to gauge satifaction over time.

How about-
"The audit was worth the time invested."
 
Back
Top Bottom