Internal Audit - Grading Scheme?

E Wall

Just Me!
Trusted Information Resource
Currently we hold internal audits and report results (good and bad) then take corrective action against the nonconformities and leave the (preventive action and continuous improvement) seeds we planted for process or systems management with the managers to discuss at mgmt review.
I have heard of others using a grading scheme, but do not like the idea of them being based on the number of nonconformities. Does anyone that has a grading system (not exclusively based on nonconformities) in place feel like sharing an outline of the system along with some of the pros/cons (regarding using a grading scheme)?

Grateful for a bit of help :)
Eileen
 
A

Al Dyer

Eileen,

I'm not a fan of grading systems for internal audit. They seem to be too subjective in defining what result get what score and brings in too many variables. A pass/fail system works well and leaves very little room for the interpretation of scores and is simple to maintain.

Has anybody else had good results with a grading system?

ASD...
 
G

goose

I had to deal with a rating scheme in a former life until I could change it.

My experience was that there was more emphasis on the "score" than there was on addressing the nonconformance.
 
I

ISO Chick

Totally agree with what you are all saying about grading Internal Audits, management tends to get hung up on the numbers and reducing them without actually analysing exactly what the problems are and dealing with them..... bad idea
 

E Wall

Just Me!
Trusted Information Resource
Thanks gang!
I'm printing this out so if it comes up - I can show them objective comments from other professionals.
 

Marc

Fully vaccinated are you?
Leader
I've never seen a grading system. Maybe I'll have to consider addressing it in the Guide ( http://Elsmar.com/Audit/ ).

Will someone describe a grading scheme for me? I'm hearing the Cons. Any 'Pro's?
 

E Wall

Just Me!
Trusted Information Resource
Thanks Alf. Something we changed might help you...
We too had numerous delays in addressing C/PARs because when found they were addressed to the area supervisor - who might not be the best person to resolve the problem. So while combining what used to be separate CARs and PARs into CPARS we also set up a timeline for responses based on the quality impact (which must be determined by the QA Manager) and designating who the C/PAR goes to is up to the Plant and/or QA Manager (both if available).
The time lines we use are:
* Low Impact - within 4 weeks
* Moderate - within 3 weeks
* High - within 2 week
The Short term solution must be defined immediately.
The Root Cause and Long Term Action Plan must be turned in by the timeline above which is reviewed by the originator (usually an internal auditor, myself or the QA Manager).
If accepted, then the follow-up is scheduled from the datae of receipt (again using the same timeline).

Weekly open reports go to the QA Manager and copies to anyone associated with open C/PARs.
 
A

Alf Gulford

Eileen-
If you’re looking for ammunition, try this. I have to report major/minor/pass counts for each production supervisor each month and the results are generally inaccurate as well as skewed:

1. Everyone has a different idea about when the month ends for reporting purposes. Some of the count gets lost between the cracks.

2. Sometimes I find that I have to re-issue a non-compliance to another supervisor so the count for the previous month is (was?) wrong on two counts. The fewer N/Cs I write, the greater effect this has on their percentages.

3. A minor non-compliance for a very small issue is counted the same as a minor non-compliance for a larger issue.

4. There’s no recognition for supervisors that take quick, effective corrective action and even preventive action as a result (as opposed to those that are dragged kicking and screaming into compliance).

On the plus side…, well, I’ll have to get back to you on that one.

Alf



[This message has been edited by Alf Gulford (edited 26 June 2001).]
 

Marc

Fully vaccinated are you?
Leader
Some thoughts from the ISO ListServe:

****************

From: Nancy Jennejohn <[email protected]>
Date: Tue, 24 Jul 2001 14:44:29 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Antonov

From: Boncho

Darryl asked:
> I am interested in determining what a common practice is for
> internal audit reporting. (apart from nonconformances or observations
> or findings) i.e. do internal auditors in general, create a
> formal "report" or is the output of their process just the
> observations/findings/nonconpliances?

Hi, Darryl,

I usually recommend the audit team to prepare official report. The report is addressed to the top management, Quality Manager, Head of the audited department and other interested internal parties/persons.

The report usually include:
- audit identification part (date, number, auditors, goals, audited
departments, other);
- the statement of the team - is the audit goals achieved;
- overall conclusion for the level of conformance of the audit area;
- brief description of NC detected and related CA prescriptions;
- other observations and remarques related to the audit performed, which are
not NC, but should be considered for the improvements in the affected area.

The report may include copies of NC notes as attachments.

As document the report usually should be written "in the language of management" - i.e. short and clear statements, clear suggestions and proposals, conclusions - they shell direct management for making appropriate decisions.

Hope this answers your question.

Regards,
Boncho Antonov

Date: Tue, 24 Jul 2001 14:48:14 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Naish

From: PNaish

Darryl,

I think the value in your internal audit process is what information you gain from it to make it better. Part of that is knowing not only what did not work right but also what is working well.

As a general rule the following items are the items we collect during an internal audit:

Person/people audited; area/dept.; auditor/s; date; documents audited such as section of QA manual, section or subsection of standard, procedure, work instruction or the like; quality records (normally being specific such as PO numbers), work order numbers, or date of receiving inspection report; equipment numbers where applicable; and what the findings were. Findings include that Karen has a good knowledge of the parts she purchases and follows the procedure exactly for steps 10 through 25 that I audited her on. Or Tom followed most of the steps but he skipped step 16 and 17 and said he did not know them. All the other people audited followed these steps exactly. Or Susy did not completely fill out PO 167825 and 167939 and photocopy and attach so the manager knows what she did not fill out. The other 15 I looked at were complete.

My feeling is that the more a manager or supervisor of an area know the more they have a better chance of determining what to fix. Since the other people followed the procedure and Tom did not, as a manager I can find out why Tom does not know and see if anyone besides the people audited did not know. If I simply say someone did not know as a manager I may never find out that it is Tom who did not know and I can not help him by getting him training.

For Susy I can see what part of the form was not completed. Then I can help Susy if she understands the importance of it or I may find it is not information needed any more and get the form changed.

I personally have seen a number of audits that did not specify the person or the steps in a process or the forms or the places on the form. By the time the manager went to fix the problem no one seemed to know exactly what the details were so the problem recurred due to incomplete problem solving information.

The other side of it is follow up. If I don't know who or what or where it is hard for me to ensure that particular situation was corrected. And per the examples not all of an area or all of the documents were incorrect so if I audit the ones that were correct before I will not be able to verify the non conformances have been corrected.

Phyllis

Date: Tue, 24 Jul 2001 14:51:01 -0500
Subject: Re: Internal Audit Reporting /VanDorp/Braaten

From: Scott

Darryl inquired:

> I am interested in determining what a common practice is for
> internal audit reporting. (apart from nonconformances or observations
> or findings) i.e. do internal auditors in general, create a
> formal "report" or is the output of their process just the
> observations/findings/nonconpliances?

I like to have each auditor record evidence of conformance. I review the evidence to ensure that the auditor understood the requirements and the intents of my checklist. I'll write a formal report including a summary of the sample taken, the nonconformances and observations, as well as percent conforming. The percent conforming can be more accurately calculated from the conforming evidence, discounting any requirements which the auditor determined to be "N/A" for the sample. If the conforming evidence is useful to the rest of the organization you can include it in the report, however most are too busy to go through 8 pages of "conformance" to see if they have any assignments.

The report and completed checklists are maintained as records of the audit, and I refer back to them when designing the next checklist for the same audit.

Hope this helps,
Scott

Date: Tue, 24 Jul 2001 14:52:29 -0500
Subject: Re: Internal Audit Reporting/VanDorp/Hoelker

From: Patricia

A report should be created stating the objective evidence used to arrive at the output of either positive observations, observations, minor or major noncompliance. The more detailed the report the easier it is for the process owner to know exactly what the issue is and to determine the correct actions to take.

Pat Hoelker

Date: Tue, 24 Jul 2001 14:56:01 -0500
Subject: Re: Internal Audit Reporting/VanDorp/Madden

From: Lori

Darryl,

We have an exit meeting after each internal audit which includes supervisors, VPs, President, and anyone else who wants to be there. In addition to presenting the findings, I write an Audit Report which includes the scope, the auditors, the findings, positive aspects and opportunities for improvement. Each attendee at the meeting gets a copy, and the report is also posted on our information bulletin board. If we just presented the findings alone, audits are perceived as all negative. But pointing out what people are doing right is a motivator.

Lori
 

Raffy

Quite Involved in Discussions
Six Months ago, we had our surveillance audit and part of the recommendations coming from our Registrar, that we must do some grading scheme. Currently we are using a Rating System in one of our Checklist Form. And Here's how it works. During Our Internal Quality Audit. We use a Checklist Form (with three columns)wherein we list number of questions on the first column. Then, there answers on the second column. And on the third column, we list the rating. The Rating System is composed of numbers from 1-4. Each number has a corresponding equivalent.
1. - Element Fulfilled
2. - Element Partially Fulfilled
3. - Planning On stage
4. - Element Not Fulfilled.
But in our specs for Internal Quality Audit, we didn't mention on how to use this form. Our Registrar had said that we must develop this Rating System. I don't know if this could be the correct Rating System that we must used.
Raffy
[email protected]
 
Top Bottom