Driving Objectivity of Review Mechanism during Internal Audits

[v9991]

So help us out...what is your objective? It will, IMHO, be no 'small effort' but will make a 'rod for your back' to try to develop such indicators. It sounds to me as if your management don't want to invest in internal audits. Have the auditors already had class time? What are you seeking to accomplish with more class time? If you find the root of the issue it'll be better to address that - if management don't support IA, then THAT's what you must address - and we can help you!!

You got it right...I was struggling to state it...
for pharma industry where i am working, Internal audit is often/always a matter of "regulatory audit-compliance"...Its not that often to be used a tool to drive improvement!.

Parallelly, I was trying to device a way through improvising the system. such that it demonstrates the "state of affairs + probably provide an target/objective to the internal auditor". Yes i take the caution that checklist will only help as much as the skill of auditor...but that is minimum starting point for aiding as pointers to SOP(aiding auditor-memory. )

+ when we start measuring with above indicated metrics, we could reflect on the areas of improvements OR provide a basis for telling management to act(invest) in people training.

in short, my objective of posting above concern...
1) share my approach on improving "internal-audit-objectivity/effectiveness"
2) take your feedback on possible metrics
3) find out any better/simpler ways of doing it

thank you all for your patience

 

[AndyN]

So, really, your challenge is to get the audit programme to be viewed differently by management. Once you have done that, you will be more than half way - you can focus on developing auditors' skills.

I'm going to suggest that the first answer lies in your audit programme management. How do you go about selecting the audit scope, criteria, frequency etc?

 

[v9991]

I'm going to suggest that the first answer lies in your audit programme management. How do you go about selecting the audit scope, criteria, frequency etc?

Audit Scope & Criteria :-
Its focussed on each specific function viz., warehouse operations, production, engineering, and criteria is most of the times compliance to set procedures
(+ certain common QA procedures compliance such as training - deviations, etc.,)
Team is aided by a rudimentary checklist which covers overall compliance to set SOPs. (i am proposing to re-structure/elaborate on the checklist, and change the mode of questionions/probing auditor must perform...in order to improve the auditor effectiveness.!!!! atleast by a small increment!!!)

Frequency :-Quarterly; (this is ensured for completion (compliance )!!!)

Team :- Mainly comprising of QA + additional functional expert (this thing seldom seems to happen)

Documentation :- Audit Report & CAPA verification/review procedures. (here, i am proposing to track the number&kind of observations!!! as mark of leading indicators...)

( informal )assessment :-of course there is focus on critical gaps; but internal observation are linked with the performance of department personnel... and performance of internal-auditor is linked with the critical observations of external/regulatory audits or complaints. ( this is reverse tracked,,, internal audit reports are retrospectively reviewed for corresponding regulatory/external audit observations,complaints ) (here, i am proposing to strengthen the link between the audited processes/observations with complains-external audit observations )

 

[AndyN]

It seems that your management's perception of audits is spot on! You may wish to modify your approach to doing audits to a different set of criteria (not audit criteria, this time) but based on 'status and importance'. I've posted this type of suggestion before, so I'd encourage you to look here: http://www.nqa-usa.com/resources/articles_detail.php?id=48

Instead of 'pushing' audits over a calendar of 12 months, why not use a different tactic and audit the things management are measured on/worried about?

 

[Bill Pflanz]

Although audit findings may drive improvement efforts I do not feel that it is a primary objective of an audit. Obviously an audit may identify process weaknesses but it would still be up to the management responsible for the system to change the process not the auditors. It would be helpful if the audit could demonstrate consistent failures in a process so there is evidence that an improvement is needed. Even with that it is sometimes get management to realize an improvement is needed.

Bill Pflanz

 

[AndyN]

Totally agree, Bill! As much as people like to talk up a storm about audits being used as an improvement mechanism, I'd rather put my money on Lean or 6 Sigma!

IMHO, audits are done to validate for management if the process gave the results observed against the objectives - by following the defined process!

If the results have been poor, then the auditors should be reporting where and why the process is a contributor to that performance. The same goes for good process performance, of course!

Only when management actively participate in the use of audits as a tool within an overall risk management context can we truly speak of them being used for 'improvement'...

 

[v9991]

Only when management actively participate in the use of audits as a tool within an overall risk management context can we truly speak of them being used for 'improvement'...

Could you pl. elaborate on this aspect...probably an example could help me here...

Totally agree, Bill! As much as people like to talk up a storm about audits being used as an improvement mechanism, I'd rather put my money on Lean or 6 Sigma!

IMHO, audits are done to validate for management if the process gave the results observed against the objectives - by following the defined process!

Yes management drive and senior-team's focus is must --> will create all the difference in success of new initiatives.; From pharma perspective, any new concept brought-in, is, seen as alien/additional process!!! but, I personally haven't heard in from any of my colleagues as a great deal...but then its couple of organizations in part of country!!!...

 

[AndyN]

Could you pl. elaborate on this aspect...probably an example could help me here...

I'm going to assume that you have a calendar of audits that someone puts together - based on some need to cover all of the ISO requirements/departments etc.

This is scheduling audits on a 'push' basis. What's needed is a 'pull' system , where audits are focused on aspects of the business which present risk to management - as described in the article - and what the standard refers to as 'status and importance'. If you choose certain members of management who can be 'trusted' to participate in determining what's a risk, then agreeing to an audit to determine if the process(es) of the QMS are being followed etc. you will be able to demonstrate the usefulness of audits to the other members of management. As an example, I know of an audit team who discovered that as a result of a process breakdown between HQ and the manufacturing plant, they were unable to ship product on time, risking (retailer) customer dissatisfaction, loss of revenue as well as non-availability of product in stores for the 'ultimate' customer...loss of revenue etc.
THAT got management's attention!

 

[Jen Kirley]

I'm going to assume that you have a calendar of audits that someone puts together - based on some need to cover all of the ISO requirements/departments etc.

This is scheduling audits on a 'push' basis. What's needed is a 'pull' system , where audits are focused on aspects of the business which present risk to management - as described in the article - and what the standard refers to as 'status and importance'. If you choose certain members of management who can be 'trusted' to participate in determining what's a risk, then agreeing to an audit to determine if the process(es) of the QMS are being followed etc. you will be able to demonstrate the usefulness of audits to the other members of management. As an example, I know of an audit team who discovered that as a result of a process breakdown between HQ and the manufacturing plant, they were unable to ship product on time, risking (retailer) customer dissatisfaction, loss of revenue as well as non-availability of product in stores for the 'ultimate' customer...loss of revenue etc.
THAT got management's attention!

Absolutely right. And some standards ask Though how they do so is not always obvious) for the audit schedule to be sensitive to things like "hot spots". In registration audits I have often been asked to show how my schedule includes areas that need more attention for reviewing NC effectiveness or other considerations.

 

[PE-2011]

Hi v9991

Very good thinking and good initiative. Definitely questionnaire is good idea, but never questionnaire will not be tick mark option. Every audit if MR / auditor collect different - different questionnaire filled from employees, management can measure the effectiveness of system, can analyse training requirement etc. Employees also start reading, learning and take improvement steps.

Best wishes.

Its like frustating... Being aware of the situation and able to do nothing about it...
I agreee, No amount of procedures/checklist/metrics could replace the individual's objectivity/sense-of-purpose to any activity.

But this is a small effort from my side to nudge the team towards right direction. Moreover, management is going to ask, what have you done in-house before recommending people for audit-course; + how do you ensure that after training& learnings (people might move on...), are internalized & sustained & measured. Above efforts are in that direction. Of course they will need to evolved specific to situations evolving over time.



They are supposed to be providing, an indicators of effectiveness/progress of the initiatives. leading indicators are something like, knowing how much of desired progress is made (assessment before writing an exam!!!)... lagging indicators are measuring the effect/impact due to initiatives.